Non-contact monitoring of photoplethysmography using radar
The non-contact radar-based PPG method using artificial neural networks effectively addresses the limitations of conventional PPG by generating accurate waveforms from radar reflections, enhancing scalability and convenience in health monitoring.
Patent Information
- Application Number
- JP2024531312
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2021-11-23
- Filing Date
- 2022-11-22
- Publication Date
- 2024-11-15
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing photoplethysmography (PPG) methods require physical contact and are limited in scalability and convenience for widespread health monitoring.
A non-contact method using radio frequency radar to irradiate and reflect energy from a person, processed by an artificial neural network to generate PPG waveforms without physical contact, employing self-attention and encoder-decoder models to extract cardiac and respiratory signals.
Enables accurate estimation of PPG signals from radar reflections, overcoming signal-to-noise ratio challenges and environmental complexities, allowing for scalable and convenient health monitoring.
Smart Images

Figure 2024542570000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to photoplethysmography (PPG), and more particularly to monitoring PPG using radar. [Background technology]
[0002] Regular monitoring of human physiological systems, such as respiratory and cardiac activity, is important both in hospitals and at home, as it is important in medical diagnostics as well as patient monitoring. One of the gold standards for such monitoring is PPG.
[0003] PPG is an optical technique that detects changes in blood volume via a pulse oximeter, which illuminates the skin and measures changes in light absorption. The ability to monitor PPG easily and on a large scale allows for better pre-screening of many health conditions and also improves an individual's overall well-being. It is widely used to monitor hypertension, measure cardiac output, predict cardiovascular disease risk, and for early screening of various pathologies. Additionally, various features derived from PPG are used as diagnostics for conditions such as arteriosclerosis, predicted risk of coronary heart disease, and the presence or absence of atherosclerotic lesions. Summary of the Invention
[0004] In one aspect, a non-contact method for photoplethysmographic monitoring of a person includes irradiating the person with radio frequency energy from a transmitter without the patient contacting the transmitter, detecting with at least one antenna the radio frequency energy reflected back from the person, and generating a photoplethysmographic waveform based on the reflected energy using an artificial neural network. [Brief description of the drawings]
[0005] To easily identify a particular element or act in the description, the most significant digit(s) in a reference number indicates the number of the drawing in which that element is first introduced. [Figure 1] 1 is a block diagram illustrating a PPG system, in accordance with some examples. [Diagram 2] 1 is a schematic diagram of a processing environment, according to some examples. [Diagram 3] FIG. 1 is a block diagram illustrating an artificial neural network, in accordance with some examples. [Figure 4] 1 is a chart illustrating the effect of bandpass filtering, in accordance with some examples. [Diagram 5] FIG. 1 is a block diagram illustrating an encoder-decoder model of an artificial neural network, in accordance with some examples. [Figure 6] FIG. 1 illustrates a multipath scenario, according to some examples. [Figure 7] FIG. 1 is a block diagram illustrating a self-attention model, according to some examples. [Figure 8] 1 illustrates a method for photoplethysmographic monitoring of a person, according to some examples. [Figure 9] FIG. 2 is a block diagram illustrating a software architecture in which the present disclosure may be implemented, according to some examples. [Figure 10] FIG. 1 is a schematic diagram of a machine in the form of a computer system upon which a set of instructions may be executed to cause the machine to perform any one or more of the methods described herein, according to some examples. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0006] Examples disclosed herein provide a radio frequency based non-contact approach to accurately estimate PPG signals (also interchangeably referred to as PPG waveforms) using radar for stationary participants. Changes in blood volume that appear in the PPG waveform are correlated to the physical motion of the heart that the radar can capture. To estimate the PPG waveform, examples use a self-attention architecture to identify the most informative reflections in an unsupervised manner, and then use an encoder-decoder network to convert the radar phase profile into a PPG sequence.
[0007] One use case for PPG is to monitor the cardiac cycle, which involves the pumping of blood from the heart to the body. PPG captures the changes in blood volume at the skin during the diastole and systole phases of the cardiac cycle. During diastole, the myocardium relaxes, the ventricles fill with blood, and blood pressure drops. Meanwhile, during systole, the myocardium contracts, pushing blood to the various organs, and blood pressure rises. Thus, the changes in blood volume seen in the PPG waveform correlate with the physical movement of the heart captured by the radar.
[0008] Because the signal-to-noise ratio (SNR) of the signal reflected from the heart is extremely small, the radar signal at the antenna may only show systolic and diastolic motion at some time points. For example, systolic motion may only be visible for a small portion of one cardiac cycle, or only after several cycles. Similarly, because there may be multiple antennas, some motion may be more visible at one antenna compared to other antennas at any given time. Deep learning networks such as Convolutional Neural Networks (CNNs) are used to tease out this property by using spatial filters to extract distinct patterns that correlate with systolic and diastolic motion.
[0009] In addition, the deep learning model exploits the diversity of multipath reflections since each multipath has a distinct angle with cardiac motion. The deep learning model also uses both extrapolation and interpolation. If the prediction window of the deep learning model is long enough to include multiple cardiac cycles, the model can learn to extrapolate information from one cardiac cycle to another. Similarly, the model can learn to interpolate by filling in missing motion patterns for a given cardiac cycle.
[0010] FIG. 1 is a block diagram illustrating a PPG system 102, according to some examples. The PPG system 102 includes a processor 106 that hosts an artificial neural network 300 (e.g., a deep learning-based encoder-decoder model), a radar 108 with one or more of a transmitting antenna and a receiving antenna, and an optional PPG sensor 110. The PPG system 102, in one example, the radar 108 includes a frequency modulated carrier wave (FMCW) radar that transmits radio frequency signals and receives reflections of the transmitted radio frequency signals from the person 104. When the person or people 104 are stationary, the main changes in the radar signal are caused by small breathing and heartbeat movements. The optional PPG sensor 110 can be used during a training phase, as described further below. The PPG sensor 110 is wearable and includes a light source and a photodetector that are attached to the surface of the skin to measure volumetric variations in blood circulation. The PPG system 102 uses a deep learning-based encoder-decoder model that converts these small movements contained in the radar signal into a PPG waveform, as described further below.
[0011] Referring now to FIG. 2, a schematic diagram of a processing environment 200 is shown including a processor 204, a processor 206, and a processor 106 (eg, a GPU, a CPU, etc., or a combination thereof).
[0012] The processor 106 is shown connected to a power source 202 and includes the following modules (either permanently configured or temporarily instantiated): an artificial neural network 300, a radar module 208, and a PPG sensor module 210. The artificial neural network 300 operatively generates a PPG waveform based on data received from the radar 108, the radar module 208 operatively generates radio frequency signals using the radar 108 and receives reflected signals for analysis by the artificial neural network 300, and the PPG sensor module 210 operatively generates PPG data using the PPG sensor 110 for training the artificial neural network 300. As shown, the processor 106 is communicatively connected to both the processor 204 and the processor 206, and the modules may be distributed between the processors.
[0013] FIG. 3 is a block diagram illustrating an artificial neural network 300, according to some examples. The artificial neural network 300 includes pre-processing 302, background removal 312, self-attention selection 314, and an encoder-decoder model 316. In pre-processing 302, the artificial neural network 300 receives continuous streams of data from both the radar 108 and the PPG sensor 110. Pre-processing 302 prepares small synchronous chunks of these streams so that they can be fed to the encoder-decoder model 316. To prepare the radar data, the artificial neural network 300 estimates a round trip length (RTL) profile 304, which indicates the RTL of each return received at the radar 108. The artificial neural network 300 then estimates the phase of the RTL profile over a time window to obtain a phase profile 306. Because the phase of the radar signal is affected by small movements of the chest and heart, the phase profile 306 can capture these movements. The artificial neural network 300 then applies bandpass filtering 308 on both the radar phase profile 306 and the ground truth PPG signal from the PPG sensor 110 to obtain the respiration and heart rate signals of both modalities. The motivation for applying bandpass filtering 308 is to ensure that the signals from the two modalities look as similar as possible and to remove high frequency noise to aid learning. The final pre-processing step is to apply a data sanity check, e.g., data sanitization 310, to ensure that the encoder-decoder model 316 does not train on erroneous data instances, such as when the person 104 is moving or does not correctly wear the ground truth PPG measurement PPG sensor 110.
[0014] Background Removal 312 distinguishes the main participant (person 104) from background participants and removes background reflections, if present. To remove reflections from background participants, the artificial neural network 300 first identifies all RTL bins from stationary participants using a periodicity heuristic. The artificial neural network 300 then marks the closest RTL bin from the stationary participant as a representative bin and uses Dynamic Time Warping (DTW) to measure the similarity of the representative bin to all other stationary RTL bins. Finally, the artificial neural network 300 filters the background RTL bins by setting them to zero in the input radar representation.
[0015] Self-attention selection 314 reduces the number of RTL bins because many of these bins do not contain useful signals but rather represent only noise. To obtain the reduced representation, the artificial neural network 300 computes an attention map and then projects the attention map onto the radar input to obtain a representation that includes only the most informative RTL bins. The attention map can be a scalar matrix that represents the relative importance of each RTL bin with respect to the target task of predicting the output PPG signal. Instead of using heuristics to select the appropriate RTL bins, the artificial neural network 300 learns to transform the input radar representation into a reduced representation of selective RTL bins using a self-attention based learning architecture integrated within the overall model.
[0016] The encoder-decoder model 316 converts the downscaled radar phase profile sequence obtained in the previous step into an output PPG time series sequence. The artificial neural network 300 uses a convolutional encoder-decoder architecture, where both the encoder and the decoder are implemented using CNNs. The convolutional encoder captures progressively higher level features as the receptive field of the network increases with the encoder depth. In contrast, the decoder progressively increases the spatial resolution of the feature maps by upsampling.
[0017] There are at least three technical challenges faced by the artificial neural network 300. The first challenge is to design a suitable loss function for the learning network. A straightforward loss that calculates the element-wise distance between the ground truth and the predicted PPG sequence does not perform well for two reasons. First, there is a small synchronization error between the PPG sensor 110 data and the radar 108 data that prevents the model from converging. To address this challenge, the artificial neural network 300 uses a synchronization-invariant loss function, which slides the target PPG sequence with different offsets, calculates the l1 loss on each slide, and then discards the rest to select the minimum loss value. Second, while the PPG signal is flip-invariant, the radar signal is not flip-invariant. This is because when a participant turns to face away from the radar, the radar phase signal also flips. However, at the same time, the position of the person does not have any effect on the PPG signal. To address this challenge, the artificial neural network 300 modifies the loss function to preserve this inversion invariance property.
[0018] The second challenge is that the majority of the RTL bins in the radar phase profile 306 do not contain any reflections from the person 104, but rather represent only noise. Thus, training the Encoder-Decoder Model 316 with all the RTL bins would not only unnecessarily increase the complexity but would also be prone to overfitting. To address this challenge, the Self-Attention Selection 314 learns to identify the RTL bins that are most informative with respect to the target task of predicting the output PPG signal. Moreover, the Self-Attention Selection 314 itself learns and is integrated within the Encoder-Decoder Model 316, thus avoiding adding unnecessary complexity.
[0019] A third challenge is that there may be multiple participants in the environment other than the main participant that the PPG system 102 is tracking. To address this challenge, the artificial neural network 300 identifies all RTL bins from stationary participants and then uses a dynamic time warping (DTW) technique to measure the similarity of the different RTL bins with the representative bin closest to the PPG system 102. The artificial neural network 300 then filters out the background RTL bins by setting them to zero in the input radar representation. However, another challenge that arises with this approach is that it is difficult to generate a large dataset from background participants. To address this challenge, the artificial neural network 300 uses an augmentation technique that randomly sets some RTL bins to zero. Thus, the artificial neural network 300 can simulate multiple person scenarios even when there is a single person in the environment.
[0020] The radar 108 transmits a sinusoidal chirp and receives reflections from the environment. The frequency of the transmitted chirp increases linearly with time at a rate m. At the receiver, a mixer downconverts the signal received at time t by multiplying it with the signal transmitted at time t. If the received signal consists of L direct and multipath reflections (where the RTL of the i-th reflection is d i), the received signal at time t after passing through the mixer is
[0021]
number
[0022] where f i is m*d i / c, and φ T is the initial phase of the transmitted signal, and φ i is the received phase of the i-th reflection. This formula is i Each reflection that travels along the RTL of 1 meter causes the downconverted signal y(t) to have a frequency of f i =m*d i / c. Thus, the magnitude of the FFT of y(t) has L peaks, each representing a frequency introduced into y(t) by one of the L reflections. The complex-valued FFT of y(t) is represented as y^(f), which means that for each frequency f in this FFT i is c*f i / m, which represents the RTL profile of the environment, given the frequency f k , that is, y^(f=f k ) any value of this RTL profile will be k represents the RTL bins that quantify the magnitude and phase of the signal with j (f) can be obtained, where 1≦j≦N.
[0023] After estimating the RTL profile, the artificial neural network 300 calculates the RTL profile bin y^(f = f k We proceed by extracting the phase of the RTL profile bin for a given antenna at a time instance t, φ j(t,f) and is given by 2π*d(t) / λ, where λ denotes the wavelength of the transmitted signal and d(t) is the round-trip distance between person 104 and PPG system 102. Because d(t) changes during inspiration and expiration, as well as during different periods of the heartbeat, φ j (t,f) captures these motions. Exemplary sliced phase profiles for several fixed values of j and f are shown in FIG.
[0024] The pre-processing 302 makes the two representations (PPG sensor 110 data and radar 108 phase profile data) as similar as possible. For example, if the two signals have different sampling rates, the artificial neural network 300 will use a more complex model that first learns to resample the two signals. Therefore, to avoid unnecessarily complicating the model, the artificial neural network 300 first learns to resample the two signals at a common sampling frequency f, which in one example is set to 20 Hz. s We resample both signals with . Furthermore, the breathing harmonics dominate in the radar phase profile, while the heartbeat harmonics dominate over the breathing harmonics in the PPG signal. This can be seen in the unfiltered radar and PPG signals shown in Figure 4.
[0025] In Fig. 4, the top row (from left to right) shows the φ j (t,f), respiratory radar phase profile φ b j (t,f), and the cardiac radar phase profile φ h j (t,f). The bottom row (from left to right) shows the PPG signal p(t), the respiratory PPG profile p b (t), and the cardiac PPG profile p h (t) is shown.
[0026] Therefore, if the learning model is directly trained on the unfiltered radar phase profile, it may not be very effective. Instead, for each window, the artificial neural network 300 obtains two band-pass filtered signals for both the radar phase profile and the PPG signal, respectively. By using band-pass filtering 308, similar breathing and heartbeat signals for the radar and PPG are obtained, which the encoder-decoder model 316 can then learn to transform. φ b j (t,f) and φ h j Let (t,f) denote the respiratory radar phase profile and the cardiac radar phase profile, respectively. To obtain these profiles, the artificial neural network 300 used a Butterworth bandpass filter with cutoff frequencies of [0.2,0.6]Hz and [0.8,3.5]Hz, respectively. The Butterworth filter provides a maximally flat response in the passband. Similarly, if the PPG signal is denoted as p(t), the respiratory PPG signal and the cardiac PPG signal are denoted as p(t), respectively. b (t) and p h The composite respiration and heart rate signals for both the radar and PPG are expressed as φ j b|h (t,f) and p b|h (t), respectively. Figure 4 shows these signals after bandpass filtering for both radar and PPG. Thus, the objective of the encoder-decoder model 316 is to obtain the following transformation:
[0027]
number
[0028] The aim is to obtain Finally, another benefit of these bandpass filters is that they remove high frequency noise that may be specific to the sensor or environment, which, if not removed, may adversely affect the performance of the Encoder-Decoder Model 316 by overfitting it. Returning to FIG. 3, the final pre-processing 302 step of the artificial neural network 300 is data cleansing 310 to check data sanity to ensure that the encoder-decoder model 316 is not trained on erroneous data. There are three sanity checks that the artificial neural network 300 performs in this step. First, the artificial neural network 300 verifies that the person 104 generating the data for training the model is actually wearing the PPG sensor 110. To perform this check, the artificial neural network 300 discards data samples where the dynamic range of the PPG signal p(t) is below a certain threshold, as this indicates that the PPG signal does not change over time. Second, the artificial neural network 300 ensures that the person is stationary by discarding any data samples where the dynamic range of the PPG signal is above a certain threshold. These thresholds are sensor specific, so their values may be calibrated by experimentation with the particular PPG sensor used in the implementation. The third and final sanity check is to ensure that the person is within range and view of the radar 108. To perform this check, the artificial neural network 300 uses a periodicity heuristic to determine whether the dominant movement in the radar signal is due to respiration.
[0029] 5 is a block diagram illustrating an encoder-decoder model 316 of an artificial neural network 300, according to some examples. The artificial neural network 300 is j b|h It takes (t,f) as input and outputs PPG sequence p b|h Predict (t). φ j b|hRecall that the shape of (t,f) is (W,N,F,2), where F is the number of RTL bins, and the last dimension represents the bandpass filtered signals of respiration and heart rate. Similarly, p b|h The shape of (t) is (W,2). In one example, F is set to 64, which means that the last RTL bin indicates a distance of about 2.5 m. However, most of these RTL bins do not contain any reflections from the person 104; rather, they only represent noise. Thus, training the Encoder-Decoder Model 316 with all the RTL bins would not only unnecessarily increase the complexity, but would also be prone to overfitting. An approach to address this issue is to use a heuristic that identifies the location of the person 104 and then selects a single corresponding RTL bin. However, as will be shown below, this would result in losing a lot of information that the Encoder-Decoder Model 316 could potentially utilize. Furthermore, heuristic-based single RTL bin selection tends to be error-prone and not generalize well to different environments. Thus, the artificial neural network 300 trains a model of self-attention selection 314 to learn to identify the top RTL bins that contain the most useful information, and then feed only those RTL bins to the Encoder-Decoder Model 316, as will be further explained below. Self-Attention Selection 314 to F a Assume there are RTL bins, where F a is a design parameter that will be explained further below. Therefore, the input φ j b|h The shape of (t,f) is (W,N,F a ,2). The final preparation step is to merge the antenna dimensions with the RTL dimensions, which may result in better verification performance. Thus, the final input dimensions fed to the encoder-decoder model 316 are (W,N×F a ,2).
[0030] The encoder-decoder model 316 comprises an encoder 502 that takes an input sequence and creates a dense representation of it, called an embedding. The embedding conveys the essence of the input to the decoder 504, which forms a corresponding output sequence. The artificial neural network 300 uses a convolutional encoder-decoder architecture, as shown in FIG. 5, where both the encoder 502 and the decoder 504 are implemented using CNNs. The convolutional encoder 502 shown in FIG. 5 captures progressively higher level features as the receptive field of the network increases with the depth of the encoder 502. At each step, the encoder 502 gradually reduces the spatial resolution of the CNN feature maps by average pooling, which performs a downsampling process. In contrast, the decoder 504 progressively increases the spatial resolution of the feature maps by upsampling. In each layer of the encoder 502 and the decoder 504, the artificial neural network 300 uses residual connections that provide alternative paths for the gradients to flow, allowing the encoder-decoder model 316 to converge faster.
[0031] Returning to FIG. 3, the loss between the target PPG signal and the predicted PPG signal is calculated using the l1 loss function 318. h|b (t)-m |h|b (t), where m |h|b (t) is the predicted model output of dimension (W,2), and p h|bwhere (t) is the ground truth PPG target of the same dimension. However, there are several challenges in using this loss. The first challenge is that although the artificial neural network 300 takes care in data collection to synchronize the radar sequence and the PPG sequence, there is nevertheless a small synchronization error that still remains. In experiments, it has been observed that the two sequences are misaligned from each other by as much as 300 ms. With such an offset, the encoder-decoder model 316 does not converge with the l1 loss. To address this issue, the artificial neural network 300 uses a sliding loss, which scales the target PPG sequence p h|b We slide (t) by offsets ranging from -S to +S, calculate the l1 loss on each slide, then discard the rest and select the minimum loss value. With this modification, the loss L can be expressed as
[0032]
number
[0033] where S is the maximum offset amount, which in one implementation is set to 300 ms. The second challenge is that while the PPG signal is inversion invariant, the radar phase profile is not inversion invariant. To understand this property, consider the case where the person 104 faces the radar 108 and then turns around to face away from the radar 108. Because the radar signal phases capture breathing and cardiac motion relative to the radar 108, these phases are inverted when the person 104 turns around to face the other direction. However, the position of the person has no effect on the PPG signal. To address this challenge, the loss function is modified to have this inversion invariant property. Specifically, the artificial neural network 300 calculates the loss for both the original target signal and the inverted target signal, and then selects the loss with the smaller value, as shown by the following equation:
[0034]
number
[0035] The third challenge is to derive the first and second derivatives from the PPG signal, which can be used to extract many informative features. However, the l1 loss does not strictly penalize errors in the predicted first and second derivatives of the PPG signal. Therefore, we modify the loss function 318 to include a term that directly penalizes both the first and second derivatives. For simplicity, let l(x,y) be expressed as:
[0036]
number
[0037] Then the final expression for L including the derivatives becomes:
[0038]
number
[0039] In one example, an encoder-decoder model 316 was trained using the RMSProp optimizer for 300 epochs. The learning rate annealing routine was set to 1e for the first 5 epochs. -4 We start with a warm start learning rate of and increase it to 1e for the next 195 epochs. -3 , 2e for the last 100 epochs -4 The training further used batch normalization after each convolutional layer to obtain a stable distribution of the inputs throughout training. For regularization, the training used a dropout layer with probability of 0.2 after each layer of the encoder-decoder model.
[0040] With reference to the self-attention selection 314, the encoder-decoder model 316 converts the radar phase profile sequence into a corresponding PPG sequence. However, instead of using all F RTL bins, the artificial neural network 300 first reduces the number of bins to F a The motivation for this reduction is to select only RTL bins that contain either direct or multipath reflections from person 104. Before describing the architecture for selecting these RTL bins, we explain why multipath reflections are critical.
[0041] Consider two cases: (i) the person's chest is pointing towards the antenna of the radar 108, and (ii) the person's chest is perpendicular to the antenna of the radar 108. These two cases are illustrated in FIG. 4(ab), where the lines d c The expiratory and inspiratory positions of the chest are indicated by d1 and d2, respectively, and the arrows indicate the direction of chest movement. The distances of the two reflections from the expiratory and inspiratory chest positions are indicated as d1 and d2, respectively, and d c Let denote the actual amount of chest motion. Also recall that the phase of the RTL profile bin is given as 2π*d(t) / λ. Here, in the first case of Fig. 6, the person's chest motion d cis the same as |d1-d2|. Due to this change in the reflection distance between exhalation and inhalation, there is a substantial variation in the phase over time due to breathing and heart motion. However, in the second case of Figure 6, the movement of the person's chest does not change the magnitude of |d1-d2|. Thus, the direct radar reflection from the person in the second scenario does not contain any information about the person's breathing or heart motion. Now consider the scenario of Figure 6 where there is an additional multipath reflection that first hits the person's chest and then reflects off a nearby wall before reaching the radar antenna. In this case, there is a change in |d1-d2| and therefore a change in the phase of the signal depending on the angle of arrival of the multipath reflection. From these observations, we show that the encoder-decoder model 316 can potentially exploit multipath reflections to significantly improve its performance. We next describe self-attention selection 314, which allows the model to select the most informative RTL bins.
[0042] FIG. 7 is a block diagram illustrating a self-attention model architecture 700, according to some examples. In one example, the self-attention selection 314 uses the self-attention model architecture 700. The self-attention model architecture 700 generates an attention map and then projects it onto the radar input to obtain a representation that contains the most informative RTL bins. The attention map is a scalar matrix that represents the relative importance of each RTL bin with respect to the target task of predicting the output PPG signal. Intuitively, it is expected that an RTL bin is informative if it contains respiration and heartbeat dominated signals, and uninformative otherwise.
[0043] The self-attention model architecture 700 includes an attention encoder 702 and an attention projector 704. The goal of the attention encoder 702 is to create an input representation of the input using convolutional layers, while the goal of the attention projector 704 is to project the attention map onto the input to obtain a reduced radar phase profile representation. The encoder comprises multiple convolutional layers that apply convolutional filters across the time dimension W, while keeping the other input dimensions intact. The intuition behind this choice is to learn features across each RTL bin independently. Each convolutional layer is constructed similarly to the encoder-decoder model 316.
[0044] The attention projector 704 converts the attention coding into a dense representation of shape (F,F a ) followed by a softmax layer that normalizes the output of the dense layer to generate an attention map. The attention map is then converted to D mn (1≦m≦F, 1≦n≦F a ) will be expressed as F a Recall that F denotes the number of reduced RTL bins that we want to keep after the projection step. In one exemplary implementation, to define a rough upper bound on the number of multipaths in an indoor environment, F a was set to 4. In addition, F a We evaluated the model using different values of F a = 4 gave the best performance. In the following description, F a is called the attention head. Attention Map D mnThe entries of indicate the relative importance of the mth RTL bin to the nth attention head. Finally, the artificial neural network 300 multiplies the input representation with the attention map to obtain a reduced radar phase profile representation. Since the self-attention model architecture 700 is part of the artificial neural network 300, the self-attention model architecture 700 is trained together with the encoder-decoder model 316 using the same loss function as described above.
[0045] Returning to Figure 3, optional background removal 312 can be used to remove radar returns associated with background people other than person 104, if any are present. In this step, the artificial neural network 300 identifies RTL bins that belong to stationary participants, i.e., RTL bins that represent returns from stationary participants. Radar input φ j b|h Recall that the shape of (t,f) is (W,N,F,2), where F is the number of RTL bins, set to 64 in one implementation. Before identifying the RTL bins that belong to stationary participants, the artificial neural network 300 performs two modifications to the input representation for this identification step. First, the artificial neural network 300 considers only the respiratory waveform, since it has a higher SNR compared to the heartbeat waveform. This is because the movement of the chest during breathing is significantly larger compared to the movement of the heart during the cardiac cycle. Second, the artificial neural network 300 pools the antenna dimensions by summing the signals from all N antennas, since each antenna has independent measurements, and adding up the measurements improves the SNR. Thus, the modified input representation φ b (t,f) has shape (W,F).
[0046] To identify RTL bins belonging to stationary participants, when a participant is stationary, the dominant movement is caused by respiratory activity. Thus, if we take the Fourier transform of the phase changes of RTL bins belonging to stationary participants over a time window, the dominant harmonics of the FFT should be in the respiratory cycle range, i.e., 0.2-0.6 Hz. To implement this insight, the artificial neural network 300 uses a heuristic that (i) checks that the highest peak of this FFT is within the respiratory cycle range, and (ii) verifies that the ratio between the first highest peak of the FFT and the second highest peak is greater than a periodicity threshold. The purpose of the latter check is to ensure that there are no other dominant movements, such as limb or arm movements. In one example,
[0047]
number
[0048] provides a good tradeoff between high true negatives (filtering RTL bins that do not belong to stationary participants) and low false negatives (filtering RTL bins that actually reflect stationary participants). b After implementing this heuristic for each RTL bin in (t,f), the artificial neural network 300 selects F that satisfies the heuristic check. ~ Identify the RTL bins.
[0049] F ~ The artificial neural network 300 calculates the similarity of each RTL bin in F with a representative RTL bin F′ and then marks each bin as either a foreground RTL bin or a background RTL bin. ~ The smallest bin in is selected as the representative RTL bin, denoted as F', since it defines the dominant participant as the one closest to the device. Before scoring the comparisons, the artificial neural network 300 performs a multiplication of the input φ b Normalize (t,f) to the scale [-1,1], where f ∈ F ~where F' is the RTL bin that ... j b|h At (t,f), we set all background RTL bins to zero. After filtering the background RTL bins, the artificial neural network 300 feeds the radar input to the encoder-decoder model 316 to generate the PPG output.
[0050] 8 illustrates a method for photoplethysmographic monitoring of a target, according to some examples. At block 802, the method 800 illuminates a target (e.g., a human and / or an animal such as a pet) with radio frequency energy from a transmitter without contacting the transmitter. At block 804, the method 800 detects radio frequency energy reflected back from the target using at least one antenna. At block 806, the method 800 generates a photoplethysmographic waveform based on the reflected energy using at least one processor (e.g., running an artificial neural network).
[0051] 9 is a block diagram 900 illustrating a software architecture 904 that may be installed on any one or more of the devices described herein. The software architecture 904 is supported by hardware, such as a machine 902 that includes a processor 920, memory 926, and I / O components 938. In this example, the software architecture 904 can be conceptualized as a stack of layers, with each layer providing a specific function. The software architecture 904 includes layers such as an operating system 912, libraries 910, frameworks 908, and applications 906. Operationally, the applications 906 invoke API calls 950 through the software stack and receive messages 952 in response to the API calls 950.
[0052] The operating system 912 manages hardware resources and provides common services. The operating system 912 includes, for example, a kernel 914, services 916, and drivers 922, where the kernel 914 serves as an abstraction layer between the hardware and other software layers. For example, the kernel 914 provides memory management, processor management (e.g., scheduling), component management, networking, and security configuration, among other functions. The services 916 can provide other common services for the other software layers. The drivers 922 are responsible for controlling or interfacing with the underlying hardware. For example, the drivers 922 can include a display driver, a camera driver, a BLUETOOTH® or BLUETOOTH® Low Energy driver, a flash memory driver, a serial communication driver (e.g., a Universal Serial Bus (USB) driver), a WI-FI® driver, an audio driver, and a power management driver.
[0053] Libraries 910 provide a low-level common infrastructure used by applications 906. Libraries 910 can include system libraries 918 (e.g., standard C libraries) that provide functions such as memory allocation functions, string manipulation functions, mathematical functions, etc. Additionally, the libraries 910 may include API libraries 924, such as media libraries (e.g., libraries for supporting the presentation and manipulation of various media formats, such as Moving Picture Experts Group-4 (MPEG4), Advanced Video Coding (H.264 or Advanced Video Coding (AVC)), Moving Picture Experts Group Layer-3 (MP3), Advanced Audio Coding (AAC), Adaptive Multi-Rate (AMR) audio codec, Joint Photographic Experts Group (JPEG or JPG), or Portable Network Graphics (PNG), graphics libraries (e.g., OpenGL framework used for rendering in two-dimensional (2D) and three-dimensional (3D) graphics content on a display), database libraries (e.g., SQLite for providing various relational database functions), web libraries (e.g., Web Kit for providing web browsing functions), etc.). The libraries 910 can also include a wide variety of other libraries 928 to provide the applications 906 with many other APIs.
[0054] The framework 908 provides a high-level, common infrastructure used by the applications 906. For example, the framework 908 provides a variety of graphical user interface (GUI) functionality, high-level resource management, and high-level location services. The framework 908 can provide a wide range of other APIs that can be used by the applications 906, some of which may be specific to a particular operating system or platform.
[0055] In some examples, the applications 906 may include a wide variety of other applications, such as a home application 936, a contacts application 930, a browser application 932, a book reader application 934, a location application 942, a media application 944, a messaging application 946, a game application 948, and a third party application 940. The applications 906 are programs that perform functions defined in the program. Various programming languages can be employed to create one or more applications 906 structured in various manners, such as object-oriented programming languages (e.g., Objective-C, Java, or C++) or procedural programming languages (e.g., C or assembly language). In certain examples, the third party applications 940 (e.g., applications developed using the ANDROID or IOS Software Development Kit (SDK) by an entity other than the vendor of a particular platform) may be mobile software that runs on a mobile operating system, such as IOS, ANDROID, WINDOWS Phone, or another mobile operating system. In this example, the third party application 940 can invoke API calls 950 provided by the operating system 912 to enable the functionality described herein.
[0056] FIG. 10 is a schematic diagram of a machine 1000 on which instructions 1010 (e.g., software, programs, applications, applets, apps, or other executable code) may be executed to cause the machine 1000 to perform one or more of the methods described herein. For example, the instructions 1010 may cause the machine 1000 to perform one or more of the methods described herein. The instructions 1010 transform a general unprogrammed machine 1000 into a specific machine 1000 programmed to perform the functions described and illustrated in the manner described. The machine 1000 may operate as a standalone device or may be coupled (e.g., networked) to other machines. In a networked arrangement, the machine 1000 may operate as a server machine or a client machine in a server-client network environment or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 1000 may include, without limitation, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), an entertainment media system, a cellular phone, a smartphone, a mobile device, a wearable device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of executing, sequentially or otherwise, instructions 1010 that specify operations to be performed by the machine 1000. Additionally, although a single machine 1000 is shown, the term "machine" may include a collection of machines that individually or jointly execute the instructions 1010 to perform one or more of the methods described herein.
[0057] The machine 1000 may include a processor 1004, a memory 1006, and I / O components 1002, which may be configured to communicate via a bus 1040. In some examples, the processor 1004 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, a processor 1008 and a processor 1012 that execute instructions 1010. The term "processor" is intended to include a multi-core processor, which may include two or more independent processors (sometimes referred to as "cores") that can execute instructions simultaneously. Although FIG. 10 shows multiple processors 1004, the machine 1000 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.
[0058] The memory 1006 includes a main memory 1014, a static memory 1016, and a storage unit 1018, all of which are accessible to the processor 1004 via a bus 1040. The main memory 1006, the static memory 1016, and the storage unit 1018 store instructions 1010 that embody any one or more of the methods or functions described herein. The instructions 1010 may also reside, in whole or in part, within the main memory 1014, within the static memory 1016, within a machine-readable medium 1020 in the storage unit 1018, within the processor 1004 (e.g., within a cache memory of the processor), or any suitable combination thereof during execution by the machine 1000.
[0059] The I / O components 1002 may include various components for receiving input, providing output, generating output, transmitting information, exchanging information, or capturing measurements. The particular I / O components 1002 included in a particular machine will depend on the type of machine. For example, a portable machine such as a cell phone may include a touch input device or other such input mechanism, while a headless server machine likely does not include such a touch input device. The I / O components 1002 may include many other components not shown in FIG. 10. In various examples, the I / O components 1002 may include an output component 1026 and an input component 1028. The output component 1026 may include a visual component (e.g., a display such as a plasma display panel (PDP), a light emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), an acoustic component (e.g., a speaker), a haptic component (e.g., a vibration motor, a resistive mechanism), or other signal generator. The input components 1028 may include an alphanumeric input component (e.g., a keyboard, a touch screen configured to receive alphanumeric input, a photo-optical keyboard, or other alphanumeric input component), a point-based input component (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or another pointing device), a tactile input component (e.g., a physical button, a touch screen that provides the position and / or force of a touch or touch gesture, or other tactile input component), an audio input component (e.g., a microphone), etc.
[0060] In further examples, the I / O component 1002 may include a biometric component 1030, a motion component 1032, an environmental component 1034, or a position component 1036, among a wide range of other components. For example, the biometric component 1030 includes components for detecting facial expressions (e.g., hand expressions, facial expressions, vocal expressions, body gestures, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweat, or brain waves), or identifying a person (e.g., voice identification, retina identification, face identification, fingerprint identification, or brainwave-based identification). The motion component 1032 includes an acceleration sensor component (e.g., an accelerometer), a gravity sensor component, and a rotation sensor component (e.g., a gyroscope). The environmental components 1034 include, for example, one or more cameras, an illuminance sensor component (e.g., a light meter), a temperature sensor component (e.g., one or more thermometers to detect ambient temperature), a humidity sensor component, a pressure sensor component (e.g., a barometer), an acoustic sensor component (e.g., one or more microphones to detect background noise), a proximity sensor component (e.g., an infrared sensor to detect nearby objects), a gas sensor (e.g., a gas detection sensor to detect concentrations of harmful gases for safety or to measure pollutants in the air), or other components that may provide an indication, measurement, or signal corresponding to the surrounding physical environment. The location components 1036 include a location sensor component (e.g., a global positioning system (GPS) receiver component), an altitude sensor component (e.g., an altimeter or barometer to detect air pressure from which altitude may be derived), an orientation sensor component (e.g., a magnetometer), etc.
[0061] The communication may be implemented using a wide variety of technologies. The I / O component 1002 further includes a communication component 1038 operable to couple the machine 1000 to the network 1022 or the device 1024 via respective couplings or connections. For example, the communication component 1038 may include a network interface component or another suitable device for interfacing with the network 1022. In further examples, the communication component 1038 may include a wired communication component, a wireless communication component, a cellular communication component, a near field communication (NFC) component, a Bluetooth component (e.g., Bluetooth Low Energy), a Wi-Fi component, and other communication components for communicating via other modalities. The device 1024 may be another machine or any of a wide variety of peripheral devices (e.g., a peripheral device coupled via USB).
[0062] Further, the communication component 1038 may detect the identifier or may include a component operable to detect the identifier. For example, the communication component 1038 may include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., an optical sensor for detecting one-dimensional barcodes such as Universal Product Code (UPC) barcodes, multi-dimensional barcodes such as Quick Response (QR) codes, Aztec codes, Data Matrix, Data glyphs, Maxi Codes, PDF417, Ultra Codes, UCC RSS-2D barcodes, and other optical codes), or an acoustic detection component (e.g., a microphone for identifying tagged audio signals). In addition, various information such as location by Internet Protocol (IP) geolocation, by triangulation of Wi-Fi signals, or by detection of NFC beacon signals that may indicate a particular location may be derived via the communication component 1038.
[0063] Various memories (e.g., main memory 1014, static memory 1016, and / or memory of processor 1004) and / or storage units 1018 may store one or more sets of instructions and data structures (e.g., software) that embody or are used by one or more of the methods or functions described herein. These instructions (e.g., instructions 1010), when executed by processor 1004, cause various operations to be performed to implement the disclosed embodiments.
[0064] The instructions 1010 may be transmitted or received over the network 1022 using any one of a number of well-known transfer protocols (e.g., Hypertext Transfer Protocol (HTTP)) via a network interface device (e.g., a network interface component included in the communications component 1038) using a transmission medium. Similarly, the instructions 1010 may be transmitted or received using a transmission medium via a coupling (e.g., a peer-to-peer coupling) to the device 1024.
[0065] Working Example In view of the above disclosure, various embodiments are described below. It should be noted that one or more features of the embodiments, either alone or in combination, should be considered within the scope of the disclosure of the present application.
[0066] Example 1 1. A non-contact method for photoplethysmographic monitoring of a target, such as a human and / or an animal (e.g., a pet), comprising: illuminating the target with radio frequency energy from a transmitter without contacting the target with the transmitter; detecting, with at least one antenna, the radio frequency energy reflected back from a target; and generating a photoplethysmography waveform based on the reflected energy using at least one processor (e.g., running an artificial neural network).
[0067] Example 2 2. The method of claim 1, wherein the processor includes a convolutional encoder-decoder model. Example 3 The method of any preceding embodiment, further comprising training the model using reflected radio frequency data and photoplethysmography sensor data collected substantially simultaneously from one or more targets.
[0068] Example 4 The method of any preceding example, wherein the training step further includes estimating a round trip length of the irradiated energy to generate a round trip length profile, acquiring a phase profile over a time window of the estimated round trip length profile, and applying band pass filtering to the acquired phase profile and the collected photoplethysmography sensor data.
[0069] Example 5 The method of any preceding example, further comprising resampling the reflected radio frequency data and the photoplethysmography sensor data at a common frequency prior to training.
[0070] Example 6 13. The method of any preceding embodiment, further comprising estimating a round trip length profile for the reflected energy, generating a phase profile from the estimated round trip length, and band pass filtering the phase profile.
[0071] Example 7 13. The method of any preceding embodiment, further comprising using an attention encoder and an attention projector to perform self-attention selection on the phase profiles.
[0072] Example 8 13. The method of any preceding embodiment, wherein self-attention selection selects radar phase profiles having multipath returns over direct returns.
[0073] Example 9 The method of any preceding embodiment, further comprising removing background reflections that are not reflected from the target.
[0074] Example 10 The method of any preceding embodiment, further comprising applying a loss function to the detected reflected high frequency energy to compensate for target inversion.
[0075] Example 11 1. A non-contact photoplethysmography detection device, comprising: a radio frequency transmitter configured to irradiate a target, such as a human and / or an animal (e.g., a pet), with radio frequency energy without the target contacting the radio frequency transmitter; at least one antenna configured to detect radio frequency energy reflected back from the target; and at least one processor configured (e.g., running an artificial neural network) to generate a photoplethysmography waveform based on reflected energy.
[0076] Example 12 12. The apparatus of example 11, wherein the processor includes a convolutional encoder-decoder model. Example 13 2. The apparatus of any preceding embodiment, wherein the convolutional encoder-decoder model is trained using reflected radio frequency data and photoplethysmography sensor data collected substantially simultaneously from one or more targets.
[0077] Example 14 The apparatus of any preceding embodiment, wherein the training step further includes estimating a round trip length of the irradiated energy to generate a round trip length profile, acquiring a phase profile over a time window of the estimated round trip length profile, and applying band pass filtering to the acquired phase profile and the collected photoplethysmography sensor data.
[0078] Example 15 2. The apparatus of any preceding embodiment, wherein the at least one processor is further configured to resample the reflected radio frequency data and the photoplethysmography sensor data at a common frequency prior to training.
[0079] Example 16 11. An apparatus as in any preceding embodiment, wherein the at least one processor is further configured to estimate a round trip length profile for the reflected energy, generate a phase profile from the estimated round trip length, and band pass filter the phase profile.
[0080] Example 17 13. The apparatus of any preceding embodiment, wherein the at least one processor is further configured to perform self-attention selection on the phase profiles using an attention encoder and an attention projector.
[0081] Example 18 13. The apparatus of any preceding embodiment, wherein self-attention selection selects radar phase profiles having multipath returns over direct returns.
[0082] Example 19 13. An apparatus according to any preceding embodiment, wherein the at least one processor is further configured to remove background reflections that are not reflected from the target.
[0083] Example 20 An apparatus as in any preceding embodiment, wherein the at least one processor is further configured to apply a loss function to the detected reflected radio frequency energy to compensate for target inversion.
[0084] Example 21 1. A non-contact photoplethysmography detection device, comprising: At least one processor; and a non-transitory memory storing instructions for causing at least one processor to perform the method of any one of Examples 1 to 10.
[0085] Example 22 A non-transitory computer-readable memory storing instructions for causing a computer to carry out the method of any one of Examples 1 to 10.
[0086] Glossary A "carrier signal" refers to any intangible medium capable of storing, encoding, or carrying instructions for execution by a machine, including digital or analog communication signals, or other intangible media for enabling communication of such instructions. Instructions may be transmitted or received over a network using a transmission medium via a network interface device.
[0087] A "communications network" refers to one or more portions of a network, which may be an ad-hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), the Internet, a portion of the Internet, a portion of the public switched telephone network (PSTN), a plain old telephone service (POTS) network, a cellular telephone network, a wireless network, a Wi-Fi network, another type of network, or a combination of two or more such networks. For example, a network or portion of a network may include a wireless or cellular network, and a connection may be a code division multiple access (CDMA) connection, a Global System for Mobile Communications (GSM) connection, or other type of cellular or wireless connection. In this example, the connection may implement any of various types of data transfer technologies, such as Single Carrier Radio Transmission technology (1xRTT), Evolution Data Optimized (EVDO) technology, General Packet Radio Service (GPRS) technology, Enhanced Data Rates for GSM Evolution (EDGE) technology, Third Generation Partnership Project (3GPP) including 3G, Fourth Generation Wireless (4G) networks, Universal Mobile Telecommunications System (UMTS), High Speed Packet Access (HSPA), Worldwide Interoperable Microwave Access (WiMAX), Long Term Evolution (LTE) standards, others defined by various standards setting bodies, other long-range protocols, or other data transfer technologies.
[0088] A "component" refers to a device, physical entity, or logic having boundaries defined by function or subroutine calls, branching points, APIs, or other techniques that provide partitioning or modularization of a particular processing or control function. A component may be combined with other components through interfaces to perform a machine process. A component may be a packaged functional hardware unit designed for use with other components, and may be a part of a program that typically performs a particular function of multiple related functions. A component may constitute either a software component (e.g., code embodied on a machine-readable medium) or a hardware component. A hardware component is a tangible unit capable of performing a particular operation, and may be configured or arranged in a particular physical manner. In an example, one or more computer systems (e.g., a stand-alone computer system, a client computer system, or a server computer system) or one or more hardware components of a computer system (e.g., a processor or a group of processors) may be configured by software (e.g., an application or application portion) as a hardware component that operates to perform a particular process as described herein. A hardware component may also be implemented mechanically, electronically, or any suitable combination thereof. For example, a hardware component may include dedicated circuitry or logic that is permanently configured to perform a particular operation. A hardware component may be a dedicated processor, such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC). A hardware component may include programmable logic or circuitry that is temporarily configured by software to perform a particular operation. For example, a hardware component may include software executed by a general purpose processor or other programmable processor.Once configured by such software, a hardware component is no longer a general-purpose processor, since it is a specific machine (or a specific component of a machine) uniquely tailored to perform the function for which it is configured. The decision to implement a hardware component mechanically, with dedicated permanently configured circuitry, or with temporarily configured circuitry (e.g., configured by software) may be made based on cost and time considerations. Thus, the phrase "hardware component" (or "hardware-implemented component") should be understood to encompass a tangible entity that is physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a particular manner or to perform a particular process as described herein. Considering examples in which hardware components are temporarily configured (e.g., programmed), each of the hardware components need not be configured or instantiated at any one time. For example, if a hardware component comprises a general-purpose processor configured by software to be a special-purpose processor, the general-purpose processor may be configured as different special-purpose processors (e.g., with different hardware components) at different times. Thus, software may configure a particular processor or processors, for example, to configure a particular hardware component at one time and to configure a different hardware component at a different time. Hardware components can provide information to and receive information from other hardware components. Thus, the described hardware components may be considered to be communicatively connected. When multiple hardware components are simultaneously present, communication may be achieved through signal transmission between two or more of the hardware components (e.g., via appropriate circuits and buses).In examples where multiple hardware components are configured or instantiated at different times, communication between such hardware components may be realized, for example, through the storage and retrieval of information in a memory structure accessible to the multiple hardware components. For example, one hardware component may perform an operation and store the output of that operation in a communicatively connected memory device. An additional hardware component may then later access the memory device to retrieve and process the stored output. A hardware component may also initiate communication with an input or output device and operate on a resource (e.g., a collection of information). Various operations of the exemplary methods described herein may be performed, at least in part, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the associated operations. Whether temporarily or permanently configured, such processors may constitute processor-implemented components that operate to perform one or more processes or functions described herein. As used herein, a "processor-implemented component" refers to a hardware component that is implemented using one or more processors. Similarly, the methods described herein may be at least in part processor-implemented, with the particular processor or processors being an example of hardware. For example, at least some of the processes of the methods described herein may be performed by one or more processors 1004 or processor-implemented components. Additionally, one or more processors may operate to support execution of associated processes in a "cloud computing" environment or as "software as a service" (SaaS). For example, at least some of the processes may be performed by a group of computers (as examples of machines that include a processor), which processes are accessible over a network (e.g., the Internet) and via one or more suitable interfaces (e.g., APIs).The execution of certain processes may be distributed among processors and spread across multiple machines as well as being present within a single machine. In some examples, a processor or processor-implemented component may be located in a single geographic location (e.g., in a home environment, an office environment, or a server farm). In some examples, a processor or processor-implemented component may be distributed across multiple geographic locations.
[0089] "Computer-readable medium" refers to both machine storage media and transmission media. Thus, these terms include both storage devices / media and carrier wave / modulated data signals. The terms "machine-readable medium," "computer-readable medium," and "device-readable medium" mean the same thing and may be used interchangeably in this disclosure.
[0090] A "machine storage medium" refers to a single or multiple storage devices and / or media (e.g., centralized or distributed databases, and / or associated caches and servers) that store executable instructions, routines, and / or data. The term includes solid-state memory, including memory internal or external to a processor, and optical and magnetic media. Specific examples of machine storage media, computer storage media, and / or device storage media include, for example, non-volatile memory, including semiconductor memory devices (e.g., erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), FPGAs, and flash memory devices), magnetic disks, such as internal hard disks and removable disks, magneto-optical disks, and CD-ROM and DVD-ROM disks. The terms "machine storage medium", "device storage medium", and "computer storage medium" mean the same thing and may be used interchangeably in this disclosure. The terms "machine storage medium", "computer storage medium", and "device storage medium" specifically exclude carrier waves, modulated data signals, and other media, at least some of which are included in the term "signal media".
[0091] A "module" refers to logic having boundaries defined by function or subroutine calls, branching points, application program interfaces (APIs), or other techniques that provide partitioning or modularization of certain processing or control functions. Modules are typically combined with other modules via interfaces to perform machine processes. A module may be a packaged functional hardware unit designed for use with other components, and may be a part of a program that typically performs a specific function among multiple related functions. A module may constitute either a software module (e.g., code embodied on a machine-readable medium) or a hardware module. A "hardware module" is a tangible unit that can perform a specific process and may be configured or arranged in a specific physical manner. In various exemplary embodiments, one or more computer systems (e.g., a stand-alone computer system, a client computer system, or a server computer system) or one or more hardware modules (e.g., a processor or group of processors) of a computer system may be configured by software (e.g., an application or application portion) as a hardware module that operates to perform a specific process as described herein. In some embodiments, a hardware module may be implemented mechanically, electronically, or any suitable combination thereof. For example, a hardware module may include dedicated circuitry or logic that is permanently configured to perform a particular operation. For example, a hardware module may be a dedicated processor, such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC). A hardware module may also include programmable logic or circuitry that is temporarily configured by software to perform a particular operation. For example, a hardware module may include software executed by a general purpose processor or other programmable processor.Once configured by such software, a hardware module is no longer a general-purpose processor, since it is a specific machine (or a specific component of a machine) uniquely tailored to perform the functions for which it is configured. It will be appreciated that the decision to implement a hardware module mechanically, with dedicated permanently configured circuitry, or with temporarily configured circuitry (e.g., configured by software) may be made based on cost and time considerations. Thus, the phrase "hardware module" (or "hardware-implemented module") should be understood to encompass a tangible entity that is physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a particular manner or to perform a particular process as described herein. Considering embodiments in which the hardware modules are temporarily configured (e.g., programmed), each of the hardware modules need not be configured or instantiated at any one time. For example, if a hardware module comprises a general-purpose processor configured by software to be a special-purpose processor, the general-purpose processor may be configured at different times as different special-purpose processors (e.g., comprising different hardware modules). Thus, software may, for example, configure a particular processor or processors to configure a particular hardware module at one time and a different hardware module at a different time. Hardware modules may provide information to and receive information from other hardware modules. Thus, the described hardware modules may be considered to be communicatively connected. When multiple hardware modules are simultaneously present, communication may be achieved through signal transmission between two or more of the hardware modules (e.g., via appropriate circuits and buses).In embodiments in which multiple hardware modules are configured or instantiated at different times, communication between such hardware modules may be realized, for example, through the storage and retrieval of information in a memory structure accessible to the multiple hardware modules. For example, one hardware module may perform an operation and store the output of that operation in a communicatively connected memory device. An additional hardware module may then later access the memory device to retrieve and process the stored output. A hardware module may also initiate communication with an input or output device and operate on a resource (e.g., a collection of information). Various operations of the exemplary methods and routines described herein may be performed, at least in part, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the associated operations. Whether temporarily or permanently configured, such processors may constitute processor-implemented modules that operate to perform one or more processes or functions described herein. As used herein, a "processor-implemented module" refers to a hardware module that is implemented using one or more processors. Similarly, the methods described herein may be performed, at least in part, by a processor, with the particular processor or processors being an example of hardware. For example, at least some of the operations of the method may be performed by one or more processors or processor-implemented modules. Furthermore, one or more processors may operate to support execution of the associated operations in a "cloud computing" environment or as a "software as a service" (SaaS). For example, at least some of the operations may be performed by a group of computers (as examples of machines that include a processor), which operations are accessible over a network (e.g., the Internet) and via one or more suitable interfaces (e.g., application program interfaces (APIs)).The execution of a particular process may reside within a single machine, but may also be distributed among processors that are spread across several machines. In some exemplary embodiments, a processor or processor-implemented module may be located in a single geographic location (e.g., in a home environment, an office environment, or a server farm). In other exemplary embodiments, a processor or processor-implemented module may be distributed across several geographic locations.
[0092] A "processor" refers to a circuit or virtual circuit (a physical circuit emulated by logic executing on an actual processor) that manipulates data values in accordance with control signals (e.g., "commands," "opcodes," "machine code," etc.) to generate corresponding output signals that are applied to operate a machine. A processor may be, for example, a central processing unit CPU, a reduced instruction set computing (RISC) processor, a complex instruction set computing CISC processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a radio frequency integrated circuit (RFIC), or any combination thereof. A processor may also be a multi-core processor having two or more independent processors (sometimes referred to as "cores") capable of simultaneously executing instructions.
[0093] "Signal medium" refers to any intangible medium capable of storing, encoding, or carrying instructions for execution by a machine, including digital or analog communication signals or other intangible media for enabling communication of software or data. The term "signal medium" may include forms of modulated data signals, carrier waves, and the like. The term "modulated data signal" means a signal that has one or more characteristics set or changed in such a manner as to encode information in the signal. The terms "transmission medium" and "signal medium" mean the same thing and may be used interchangeably in this disclosure.
Claims
1. 1. A non-contact method for photoplethysmographic monitoring of a target, comprising: illuminating the target with radio frequency energy from the transmitter without contacting the target with the transmitter; detecting, with at least one antenna, the radio frequency energy reflected back from the person; and generating, using at least one processor, a photoplethysmography waveform based on the reflected energy.
2. The method of claim 1 , wherein the at least one processor includes a convolutional encoder-decoder model.
3. 3. The method of claim 2, further comprising training the model using reflected radio frequency data and photoplethysmography sensor data collected substantially simultaneously from one or more targets.
4. 4. The method of claim 3, wherein the training step further comprises estimating a round trip length of irradiated energy to generate a round trip length profile, acquiring a phase profile over a time window of the estimated round trip length profile, and applying band pass filtering to the acquired phase profile and the collected photoplethysmography sensor data.
5. 4. The method of claim 3, further comprising the step of resampling the reflected radio frequency data and the photoplethysmography sensor data at a common frequency prior to training.
6. The method of claim 1 , further comprising the steps of estimating a round trip length profile for the reflected energy, generating a phase profile from the estimated round trip length, and band pass filtering the phase profile.
7. The method of claim 6 , further comprising the step of performing self-attention selection on the phase profile using an attention encoder and an attention projector.
8. The method of claim 7 , wherein the self-attention selection selects radar phase profiles having multipath returns over direct returns.
9. The method of claim 1 , further comprising the step of removing background reflections that are not reflected from the target.
10. The method of claim 1 , further comprising applying a loss function to the detected reflected radio frequency energy to compensate for inversion of the target.
11. 1. A non-contact photoplethysmography detection device, comprising: a radio frequency transmitter configured to irradiate the target with radio frequency energy without contacting the target with the radio frequency transmitter; at least one antenna configured to detect the radio frequency energy reflected back from the target; at least one processor configured to generate a photoplethysmography waveform based on the reflected energy.
12. The apparatus of claim 11 , wherein the at least one processor includes a convolutional encoder-decoder model.
13. 13. The apparatus of claim 12, wherein the convolutional encoder-decoder model is trained using reflected radio frequency data and photoplethysmography sensor data collected substantially simultaneously from one or more targets.
14. 14. The apparatus of claim 13, wherein training further comprises estimating a round trip length of irradiated energy to generate a round trip length profile, acquiring a phase profile over a time window of the estimated round trip length profile, and applying band pass filtering to the acquired phase profile and the collected photoplethysmography sensor data.
15. 14. The apparatus of claim 13, wherein the at least one processor is further configured to resample the reflected radio frequency data and the photoplethysmography sensor data at a common frequency prior to training.
16. 12. The apparatus of claim 11, wherein the at least one processor is further configured to estimate a round trip length profile for the reflected energy, generate a phase profile from the estimated round trip length, and band pass filter the phase profile.
17. 20. The apparatus of claim 16, wherein the at least one processor is further configured to perform self-attention selection on the phase profile using an attention encoder and an attention projector.
18. The apparatus of claim 17 , wherein the self-attention selection selects radar phase profiles having multipath returns over direct returns.
19. The apparatus of claim 11 , wherein the at least one processor is further configured to remove background reflections that are not reflected from the target.
20. The apparatus of claim 11 , wherein the at least one processor is further configured to apply a loss function to the detected reflected radio frequency energy to compensate for inversion of the target.
Citation Information
Cited By
Heart rate measurement device, heart rate measurement method, learning model generation device, trained model, and program
WO2026140417A1
