Method and system for photoplethysmography data preparation in machine learning training

Abstract

A method for training a machine learning (ML) model using photoplethysmography (PPG) data is developed. The method includes receiving PPG training data including respective series of PPG data corresponding to first and second light wavelengths; generating time segments in at least one PPG data set; selecting only a portion of the time segments having high frequency noise below a predetermined threshold for inclusion in training data; removing a rolling average from each time segment in the training data; identifying peaks and troughs in the training data; generating normalized amplitudes of the peaks and troughs in the training data; identifying waves in the normalized amplitudes, each wave being identified based on a spike in the normalized amplitudes; and using a training process to generate a trained ML model from an untrained ML model and the identified waves to classify data related to a biomarker.

Description

[0001] Priority Statement This patent application claims priority to International Patent Application No. PCT / US2024 / 016460, filed on February 20, 2024, which is incorporated herein by reference in its entirety. Technical Field

[0002] This disclosure relates to systems and methods for improving the quality of signal data received from photoplethysmography (PPG) sensors, enabling improvements to machine learning classifiers for physiological data encoded in PPG signals that rely on a training process. Background Technology

[0003] In the health and fitness field, photoplethysmography (PPG) sensors are a common form of non-invasive sensor used to monitor physiological characteristics in humans or other animals. Examples of commercially available PPGs include one or more optical transceivers placed on a user's skin. The transceiver emits light pulses that penetrate at least a portion of the user's skin and blood vessels, and detects the return signal from the pulses, which is then processed to measure biomarkers within the user's body. Specifically, time-series PPG measurements can detect the dilation and constriction of a user's blood vessels, but the light signals detected in the PPG data can include other characteristics that can be used to detect the user's biomarkers. Some PPG sensors use transceivers with multiple wavelengths of light, including green, red, yellow, and near-infrared light, which can penetrate the skin to varying degrees and transmit complementary signals. Common uses of PPG sensors in health and fitness devices include measuring heart rate and blood oxygen saturation (pulse oxygen saturation) biomarkers.

[0004] Beyond the traditional applications of PPG sensors, there is interest in extracting other biomarkers from PPG sensor data. Many of these works involve using machine learning (ML) models to classify biomarkers based on latent features in the PPG signal. However, commercially available PPG sensors are known to generate significant amounts of noise. While signal processing techniques exist to mitigate the impact of noise on common applications such as heart rate and pulse oxygen saturation, noise can introduce artifacts, leading to errors during the training of ML models. If errors are introduced during training, subsequent inference using the trained ML model to classify biomarkers of interest may produce unintended inaccuracies when the ML model incorrectly interprets noise as a feature of interest in the PPG data. Therefore, it would be beneficial to improve systems and methods for processing PPG data to improve the quality of ML model training and actual inference. Summary of the Invention

[0005] In one embodiment, a method for training a machine learning model using photoplethysmography (PPG) data has been developed. The method includes: receiving sensor data comprising at least one PPG dataset using a training system, the at least one PPG dataset further comprising a first series of PPG data corresponding to a first optical wavelength and a second series of PPGs corresponding to a second optical wavelength; generating multiple time segments in the at least one PPG dataset using the training system; selecting only a portion of the multiple time segments having high-frequency noise below a predetermined threshold for inclusion in the training dataset using the training system; removing rolling averages for the first series of PPG data and the second series of PPG data from one or more time segments in the training dataset using the training system; identifying multiple peaks and troughs in each of the first series of PPG data and the second series of PPG data within one or more time segments in the training dataset using the training system; generating multiple normalized amplitudes of the multiple peaks and troughs corresponding to each of the first series of PPG data and the second series of PPG data within one or more time segments in the training dataset using the training system; and generating multiple normalized amplitudes of the multiple peaks and troughs corresponding to each of the first series of PPG data and the second series of PPG data within one or more time segments in the training dataset using the training system. Multiple waves are identified from multiple normalized amplitudes corresponding to multiple peaks and troughs in the data, each wave being identified based on a peak among multiple normalized amplitudes; and a training system is used to generate a trained machine learning model from an untrained machine learning model and multiple waves identified in the training dataset to classify data related to biomarkers.

[0006] In another embodiment, a system for training a machine learning model using photoplethysmography (PPG) data has been developed. The system includes a memory and a processor operatively connected to the memory. The memory is configured to store program instructions; sensor data including at least one PPG dataset, which further includes a first series of PPG data corresponding to a first light wavelength and a second series of PPG data corresponding to a second light wavelength; and data corresponding to a machine learning model. A processor is operatively connected to memory and configured to execute stored program instructions to generate multiple time segments in at least one PPG dataset; select only a portion of the multiple time segments having high-frequency noise below a predetermined threshold for inclusion in a training dataset; remove rolling averages for a first series of PPG data and a second series of PPG data from one or more time segments in the training dataset; identify multiple peaks and troughs in each of the first series of PPG data and the second series of PPG data within one or more time segments in the training dataset; generate multiple normalized amplitudes of the multiple peaks and troughs corresponding to each of the first series of PPG data and the second series of PPG data within one or more time segments in the training dataset; identify multiple waves in the multiple normalized amplitudes of the multiple peaks and troughs corresponding to each of the first series of PPG data and the second series of PPG data within one or more time segments in the training dataset, each wave being identified based on a spike in the multiple normalized amplitudes; and use a training process to generate a trained machine learning model stored in memory with an untrained machine learning model and the multiple waves identified in the training dataset for classifying data related to biomarkers. Attached Figure Description

[0007] To facilitate identification of any particular element or behavior being discussed, one or more of the most effective numbers in the figure references refer to the figure number in which the element was first introduced.

[0008] Figure 1 This is a schematic diagram of a system for training and inference using machine learning (ML) models based on photoplethysmography (PPG) sensor data.

[0009] Figure 2 It is a block diagram of the methods for preparing PPG data for ML model training, training the ML model using the prepared training data, and using the trained ML model.

[0010] Figure 3 Two distinct time segments of the photoplethysmography were depicted, which included waves from two different light wavelengths with low and high levels of high-frequency noise.

[0011] Figure 4 The zero-crossing points in the time segments of the photoplethysmogram were depicted, as well as histograms depicting the distribution of zero-crossing points for low-frequency and high-frequency noise.

[0012] Figure 5 The peaks and valleys in the wave of the photoplethysmography were depicted, and a normalized wave was generated from the photoplethysmography data.

[0013] Figure 6 Peak identification in the normalized wave of photoplethysmography data was described.

[0014] Figure 7 Is using Figure 2 An instance of an autoencoder ML model trained using the described method. Detailed Implementation

[0015] These and other advantages, effects, features, and purposes will be better understood from the following description. Reference is made in the accompanying drawings, which form a part of the specification, and embodiments of the inventive concept are shown in the drawings by way of illustration but not limitation. Throughout the drawings, corresponding reference numerals indicate corresponding parts.

[0016] While the inventive concept is readily adaptable to various modifications and alternatives, exemplary embodiments thereof are illustrated by way of example in the accompanying drawings and described in detail herein. However, it should be understood that the following description of exemplary embodiments is not intended to limit the inventive concept to the specific forms disclosed, but rather is intended to cover all advantages, effects, and features falling within its essence and scope, as defined in the embodiments described herein and the following embodiments. Therefore, reference should be made to the embodiments described herein and the following embodiments in order to clarify the scope of the inventive concept. It should be noted that the embodiments described herein may have advantages, effects, and features useful for solving other problems.

[0017] The apparatuses, systems, and methods will now be described more fully below with reference to the accompanying drawings, which illustrate some, but not all, embodiments of the inventive concept. In fact, these apparatuses, systems, and methods may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided to enable this disclosure to meet applicable legal requirements.

[0018] Similarly, those skilled in the art to which this disclosure pertains will, upon benefiting from the teachings set forth in the foregoing description and the accompanying drawings, contemplate numerous modifications and other embodiments of the apparatuses, systems, and methods described herein. Therefore, it should be understood that these apparatuses, systems, and methods are not limited to the specific embodiments disclosed, and that variations and other embodiments are intended to be included within the scope of the embodiments. Although specific terminology is used herein, it is used in a general and descriptive sense only and not for limiting purposes.

[0019] Unless otherwise defined, all technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. Although similar or equivalent methods and materials may be used in the practice or testing of this method, preferred methods and materials are described herein.

[0020] Furthermore, the use of the indefinite article "a / an" to refer to an element does not preclude the possibility of the existence of more than one element, unless the context explicitly requires that there be exactly one element. Therefore, the indefinite article "a / an" usually means "at least one / a kind." Similarly, the terms "have," "contain," or "include," or any of their arbitrary grammatical variations, are used in a non-exclusive manner. Thus, these terms can refer either to a situation where no further features exist in the entity described in the context besides those introduced by these terms, or to a situation where one or more further features exist. For example, expressions such as "A has B," "A contains B," and "A includes B" can refer to a situation where no other elements exist in A besides B (…). Right now A can refer to the case where A consists solely and exclusively of B, or it can refer to the case where, in addition to B, there are one or more other elements in A, such as element C, element C and D, or even other elements.

[0021] As used herein, the term Individual with Diabetes (PwD) refers to a person diagnosed with one or more forms of diabetes or at risk of being diagnosed with one or more forms of diabetes, including prediabetes, type 1 diabetes, type 2 diabetes, gestational diabetes, and optionally one or more comorbidities associated with diabetes.

[0022] As used herein, the term "biomarker" refers to any quantifiable aspect of the physiology of PwD that is directly identified or calculated from one or more measurements generated by an analytical device, such as a PPG sensor. For PwD, biomarkers of interest may be directly related to monitoring one or more of the following: diabetes in PwD, diabetes-related comorbidities in PwD, or general health and well-being in PwD. Non-limiting examples of biomarkers that can be encoded in PPG data include pulse oxygen saturation, heart rate, respiratory rate, blood pressure, and chemical analytes in blood or other body fluids and tissues.

[0023] As used herein, the term "PPG dataset" refers to at least one time-series of data generated by a photoplethysmography (PPG) sensor placed on the skin of a human or animal user. In the examples described herein, each PPG dataset comprises a series of discrete PPG data points, which may also be interchangeably referred to herein as "sampling points" or "measurements." In some embodiments described herein, the PPG dataset further comprises PPG data from a PPG sensor operating simultaneously at two different light wavelengths. Thus, these PPG datasets include a first series of PPG data and a second series of PPG data for the user, representing measurements performed simultaneously at different light wavelengths. Each series of PPG data is also referred to herein as a photoplethysmography plot.

[0024] Figure 1 depicts a system 104 for training a machine learning model to classify biomarkers in photoplethysmography sensor data received from user-operated PPG sensors. Figure 1 further depicts multiple training users 152 with corresponding training PPG sensors 154, multiple inference users 162 with corresponding inference PPG sensor device systems 164, and multiple healthcare providers (HCPs) 172 with corresponding HCP computing terminals 174.

[0025] exist Figure 1In this embodiment, the training PPG sensor 154 collects training PPG sensor data from the training user 152 during the training of the machine learning model. After the machine learning model is trained, the inference PPG sensor device system 164 collects PPG sensor data from the inference user 162, and the trained machine learning model generates an output indicating biomarkers in the PPG data. In some embodiments, the training PPG sensor 154 has the same hardware configuration as the inference PPG sensor device system 164. In other embodiments, the inference PPG sensor device system 164 is either incorporated into or communicatively coupled to additional hardware elements for inference operations. Therefore, references to the inference PPG sensor device system 164 herein refer to the PPG sensor hardware and the additional hardware and software elements that interact with the PPG sensor, including, for example, smartphones, smartwatches, personal computers, or other electronic devices used by the inference user 162. In the configuration described herein, both the training PPG sensor 154 and the inference PPG sensor device system 164 include a multi-wavelength transceiver that emits and detects reflected or transmitted light (depending on the sensor configuration) having wavelengths within at least two different wavelength ranges. For example, PPG sensors 154 and 164 are configured to emit and detect light in a first wavelength range of approximately 500 nm to 600 nm corresponding to green light and light in a second wavelength range of approximately 750 nm to 1400 nm corresponding to near-infrared light. Other PPG configurations optionally use different selected wavelengths corresponding to other wavelength ranges, which are typically in the visible or infrared spectrum due to the light absorption characteristics of human epidermis and subcutaneous tissue. In most practical embodiments, inference user 162 represents a larger group than training user 152, but it is conceivable that some or all of the training users 152 may also become inference users 162 after the training process is completed.

[0026] exist Figure 1In this configuration, training system 104 includes processor 108, communication interface 112, and memory 120. Processor 108 is formed of one or more digital processing devices, such as a central processing unit (CPU), graphics processing unit (GPU), field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), digital signal processor (DSP), and any other suitable digital logic device. In some embodiments, processor 108 optionally includes one or more neural network accelerators (NNAs) configured to efficiently perform either or both of machine learning model training and machine learning model inference operations, although these accelerators are not necessarily required to implement system 104 and perform the methods described herein. Communication interface 112 is a means of enabling training system 104 to communicate with other computing systems via network 140, including training PPG sensor 154, inference PPG sensor device system 164, and HCP terminal 174. Non-limiting examples of communication interface 112 include wired and wireless network adapters, such as Ethernet and IEEE 802.11 familys of wireless network adapters. Network 140 is typically a local area network (LAN) or wide area network (WAN), such as the Internet. While network 140 enables convenient remote data exchange between training / inference system 104 and PPG sensors 154 / 164, in alternative configurations, network 140 is a physical "shoe network" that allows data storage devices such as USB flash drives or other data storage media to be exchanged so that training / inference system 104 can receive photoplethysmography data from PPG sensors 154 / 164 via direct access to the data storage media.

[0027] In training system 104, memory 120 is a non-transient storage system including one or more data storage devices, comprising at least one of non-volatile or volatile storage devices configured to store program instructions 122, raw PPG training data 124, machine learning (ML) model 126, PPG inference data 132, and ML output data 136. The stored program instructions 122 include instructions for a data transformation process that receives the raw PPG training data 124 and generates training PPG data 128 corresponding to identified waves in the high-quality portion of the PPG data, while eliminating low-quality data. The raw PPG training data 124 optionally includes additional ground-based data about biomarkers of interest to the supervised training process, enabling training system 104 to validate the training progress of ML model 126 from an initial untrained state to a trained state. Ground-based data is typically generated using a separate sensing device (not shown) that operates simultaneously with the training user 152 and the PPG sensor 154. However, after the training process is complete, the inference user 162 operates the inference PPG sensor system 164 and the trained ML model 126 without using additional separate sensing devices.

[0028] The stored program instructions further include instructions for the training process to generate a trained ML model 126. In some embodiments, the trained ML model 126 further incorporates multiple ML models configured in series or parallel to classify information about one or more biomarkers of interest in the PPG inference data. An example of an ML model is an autoencoder model, although other ML models can be trained and used for inference using the same techniques described herein. Specifically, in one configuration, an autoencoder ML model is trained to reproduce a high-quality representation of the PPG signal data to reduce or eliminate signal noise and other artifacts in the raw PPG sensor data, and the autoencoder optionally generates intermediate training output data, inference output data, or both, for a second trained ML model to classify information associated with one or more biomarkers. In a configuration where system 104 also functions as an inference system, memory 120 stores PPG inference data 132, which includes PPG data types similar to the original training data 124, but these data are not part of a predetermined training dataset. Figure 1In this process, inference system 104 receives PPG inference data 132 from PPG sensor device system 164, which is typically located outside the original training pool. During inference operations, inference system 104 applies the PPG inference data 132 to the trained ML model 126 to generate ML output data 136. Figure 1 In the context of this, ML output data 136 includes data for estimating biomarkers related to the interests of one of the inference users 162.

[0029] During operation, training system 104 receives training data in the form of multiple photoplethysmograms from PPG sensor 154 operated by training user 152. Following data cleaning and training operations, described in further detail below, training system 104 trains machine learning (ML) model 126. The trained ML model 126 is then used, optionally as a combined training / inference system 104, or by providing the trained ML model 126 to other computing devices for inference. While training system 104 is depicted as a single device for illustrative purposes, those skilled in the art will recognize that training system 104 can be implemented using a cluster of multiple computing systems communicatively coupled via network 140 or other network connections. As described above, training system 104 optionally acts as inference system 104, but other computing devices include dedicated network-connected inference systems or individual electronic devices, such as PPG sensor device system 164 or other electronic devices operated by inference user 162, which can perform inference operations using the previously trained machine learning model. Therefore, any references to the training system 104 or the inference system 104 herein are to be understood as applicable to either of these hardware and software configurations.

[0030] Figure 2 This is a block diagram of process 200 for processing PPG data to reduce the impact of noise and other low-quality data during ML model training. In the following description, references to process 200 performing a function or action refer to the processor executing stored program instructions to perform a function or action in coordination with other components of the PPG sensing and machine learning system. For illustrative purposes, combined with... Figure 1 The implementation process is described in 200.

[0031] Process 200 begins (block 204) when the training / inference system 104 receives PPG training data from the training PPG sensor 154. Figure 1In this configuration, PPG sensor 154 collects PPG data from training user 152 and transmits the PPG data to training system 104 via network 140. Training system 104 stores photoplethysmography (PPG) images in raw PPG training data 124 in memory 120. Each PPG dataset includes PPG images at a first optical wavelength and a second optical wavelength, which correspond to the time series of measurements obtained from training user 152 operating one of the PPG sensors 154.

[0032] As described above, the raw PPG training data includes useful PPG sensor data as well as noise and other artifacts generated by the training PPG sensor 154 during operation, but this is not useful for training a machine learning model. Process 200 continues as the training system 104 performs a series of operations to reduce the effects of noise and selects time segments that include high-quality PPG signal data (training PPG wave data 128 stored in memory 120), and the training system 104 then uses this data to generate a trained ML model 126. During process 200, the training system generates multiple time segments from the PPG training data and selects only portions of time segments that include a sufficiently low level of high-frequency noise useful for the machine learning training process (block 208). In one embodiment, each time segment lasts four (4) seconds and the training PPG sensor 154 generates 25 samples per second, resulting in 100 PPG data samples in each time segment. In another embodiment, each time segment is 60 seconds long, and the training PPG sensor 154 also generates 25 sampling points per second, resulting in 1500 PPG data sampling points in each time segment. However, alternative configurations can use different time segment lengths and sampling rates. The selection process includes a wavelength correlation process and a zero-crossing measurement operation to identify whether each time segment has a sufficiently low level of high-frequency noise for selection in the training operation. In the wavelength correlation process, the training system 104 identifies the correlation between waves in a first series of PPG data at a first wavelength and waves in a second series of PPG data at a second wavelength for each time segment. The correlation level is inversely proportional to the high-frequency noise level in the first and second series of PPG data, and since the first and second series of PPG data are expected to have a strong positive correlation with acceptable high-frequency noise levels. If the correlation score exceeds a predetermined threshold, the training system 104 stores the time segments for both the first and second optical wavelengths for further processing to identify waves within the PPG dataset, but if the score is too low, the time segments are excluded from the training PPG data 128. Figure 3Two distinct instances of PPG measurements are depicted, with a first PPG time series plot 304 including low levels of high-frequency noise and a second PPG time series plot 316 including high levels of high-frequency noise. As depicted in Figure 304, trace 308 represents PPG data corresponding to a first wavelength of green light (e.g., approximately 500 nm to 600 nm), and the second trace 312 represents PPG data corresponding to a second wavelength of near-infrared light (e.g., approximately 750 nm to 1400 nm). The first and second traces exhibit a high level of positive correlation, indicating low levels of high-frequency noise. During operation, training system 104 selects this time segment for further processing to identify the normalized wavelengths included in the training PPG data 128. Conversely, Figure 316 depicts PPG data traces 320 and 324, corresponding to the first wavelength of green light and the second wavelength of infrared light, respectively. Figure 316 depicts the low level of correlation between traces 320 and 324, indicating the level of high-frequency noise exceeding a predetermined threshold. During operation, the training system 104 does not select this time segment for inclusion in the training PPG data 128.

[0033] In addition to the aforementioned processes, the training system 104 also determines the number of zero-crossing counts for the signal in each time segment to evaluate low-frequency noise, high-frequency noise, or both low-frequency and high-frequency noise. Figure 4 The zero-crossing counting technique used for filtering is described in more detail. Figure 404 depicts the signal across a time segment spanning the zero amplitude threshold at zero-crossing points 408A-408G. The system 104 detects each zero-crossing by identifying consecutive sampling points in the PPG data that transition from negative to positive, positive to negative, or when a single PPG data sampling point has a zero value. Figure 4In the example, the time segment in Figure 404 includes seven (7) zero-crossings at references 408A-408G. To determine the noise level within the time window, training system 104 compares the number of detected zero-crossings with one or both of a minimum threshold and a maximum threshold for zero-crossings. The minimum threshold for zero-crossings represents the minimum number of zero-crossings expected to be identified within the time segment, with values ​​below the minimum threshold representing low-frequency noise in the time segment. The maximum threshold for zero-crossings represents the maximum number of zero-crossings expected to be identified within the time segment, with values ​​above the minimum threshold representing high-frequency noise in the time segment. Based on these thresholds, training system 104 rejects time segments exhibiting excessive low-frequency or high-frequency noise, and time segments included within either or both of the thresholds in the training PPG data 128. To identify the minimum and maximum thresholds, training system 104 refers to stored data corresponding to histograms of zero-crossings observed in a large number of time segments in the training data, such as... Figure 4 Histogram 412 in the image. Figure 4 In one example, training system 104 identifies a minimum threshold with seven (7) zero-crossing points at reference 416 in histogram 412 and a maximum threshold with nine (9) zero-crossing points at reference 420, but the exact values ​​of the minimum and maximum thresholds may differ in other configurations.

[0034] Refer again Figure 2 As training system 104 subtracts the rolling average of the PPG data in each selected time segment to reduce the DC signal bias (block 212), process 200 continues. In the embodiments described herein, the rolling average refers to the average of a window of ten (10) PPG data sampling points in the time-series PPG data surrounding each PPG data point, but larger or smaller rolling average windows may be used. As used herein, the term "average" includes weighted or unweighted arithmetic, geometric, harmonic, and logarithmic averages, or any other technique recognized in the art for averaging time-series data. Training system 100 subtracts the rolling average from each data point to mitigate the effects of the DC bias, such that the PPG data time segments generated between different times and different training users 152 show reduced variance due to the DC bias. Since the DC bias typically does not contain useful information for training the ML model, this process improves consistency between different training datasets. Training system 104 performs the subtraction of the rolling average for both the first and second wavelengths in the PPG data.

[0035] The high-frequency noise filtering and rolling average subtraction processes described above are performed before the remaining processing for generating training data, as illustrated in the illustrative example of Figure 2. However, in alternative configurations, these operations can be performed at a later stage of processing, particularly after PPG signal data normalization. Additionally, the high-frequency filtering and rolling average subtraction processes can be performed in any order or concurrently with a parallel computer processing system.

[0036] When the training system 104 generates normalized amplitude values ​​(block 216) in the training data based on the identified peaks and valleys in the photoplethysmography data of the selected time segment after subtracting the above rolling average, process 200 continues. Figure 5 Examples of standard and normalized PPG data are described in more detail. Figure 5 Figure 504 depicts a time series of PPG data with varying peaks and troughs, indicating changes in the detected optical signal due to the expansion and contraction of blood vessel volume in one of the training users 152. During process 200, the training system 104 divides the PPG data into predetermined time segments, such as the 60-second time segments depicted in Figure 5, but the length of the time segments can be shorter or longer. Due to the periodic expansion and contraction of blood vessels, the PPG data also depicts periodic peaks and troughs. As depicted in Figure 504, even if the peak and trough amplitudes of some signals in the sequence are very similar to each other, the absolute value of the PPG data will change over time, such as the similar peak and trough amplitudes at references 506 and 508, but the absolute value of the PPG data varies between these references. Figure 516 depicts a normalization plot, corresponding to the different PPG data segments depicted in Figure 504, where the trough of each wave is set to a predetermined level, such as... Figure 5 The zero (0) in the PPG data. Even if the absolute values ​​of the waves in the original PPG data are different, the normalized data makes it possible to identify waves with similar amplitudes and frequencies in the PPG data, and makes it possible to identify waves with significantly different amplitudes and frequencies from other waves, such as wave 520, in a certain time segment. The training system 104 performs the above-described normalization on the photoplethysmograms of both the first and second wavelengths in the PPG data. Process 200 uses the normalized PPG data for the remainder of the training process described herein. During a later inference operation, the inference system 104 or another system using the trained ML model 126 also applies the above-described normalization process to PPG inference data, such as PPG inference data 132 depicted in Figure 1.

[0037] Refer again Figure 2Process 200 continues to identify waves in the normalized data based on continuous amplitude spikes (block 220). This process can identify waves in the PPG data corresponding to the aforementioned vasodilation / vasoconstriction cycle, while reducing the possibility of misidentifying signal noise from the corresponding training PPG sensor 154 as physiological signals representing those from the training user 152. (See reference...) Figure 6 Figure 600 illustrates the identification of peaks in the normalized PPG data in more detail. To identify the peaks of each wave, the training system 104 processes the time-series PPG data to identify when the signal level first exceeds a predetermined level in the normalized waveform data (dashed line 604), such as the threshold of 0.4 in Figure 6 (scaled from 0.0 to 1.0), but other threshold levels may be used in other embodiments. The training system 104 further identifies peaks in a series of waveform samples by observing monotonically increasing and subsequently monotonically decreasing sample points, indicating the rising and falling portions of peaks in the waveform. Figure 6 In the example, a series of sampling points monotonically increase to a peak and then monotonically decrease toward a predetermined threshold indicated by line 604 to identify the spikes of waveform peaks 608A-608D. The training system 104 does not identify PPG datasets that do not include spikes matching these criteria as waves to discard noise and erroneous signals from the normalized PPG data. In one configuration, the training system 104 identifies a predetermined minimum number of consecutive waves, such as the four (4) waves in the example of Figure 6, where no false spikes are detected between the detected waves, and the training data 128 includes normalized PPG data for the identified waves. In an alternative configuration, the training system 104 identifies individual waves or a different number of waves in the training data. The above normalization process enables efficient methods to identify waves in the training dataset, which would be more complex and error-prone if the raw PPG data were processed directly. The training system 104 performs the above wave identification for photoplethysmography of both the first and second optical wavelengths in the PPG data.

[0038] Refer again Figure 2 As training system 104 trains ML model 126 (block 224) using waves identified in normalized training PPG data 128, process 200 continues. A non-restricted instance of an ML model is an autoencoder, and Figure 7An example of an autoencoder ML model 700 is depicted for illustrative purposes. The autoencoder 700 includes groups of nodes forming an encoder 704, a latent coding space 712, and a decoder 716. More specifically, the encoder 704 includes an input layer 706, where each input is a one-dimensional array of 100 amplitudes obtained from training PPG data 128 corresponding to at least one identified wave. The coding layer 708 is a "hidden layer" that includes fewer nodes than the input layer 706. Each node in the coding layer 708 receives input data and compresses the input data by applying weights to the combined inputs of the input layer 706, and uses a non-linear output activation function to generate an output value based on the combination of inputs. While Figure 7 An intermediate layer 708 is depicted in encoder 704, but alternative configurations use multiple intermediate layers in the encoder. The latent coding space layer 712 comprises a minimum number of nodes in autoencoder 700 that compress information about the most important features of the input data. Because there are fewer nodes in latent space layer 712 compared to input layer 706, latent space layer 712 stores a lossy representation of the most important features of the original input data, where the training process adjusts the autoencoder to identify the most important features of the training data. After the training process is complete, in some configurations, the compressed encoded data in latent space layer 712 represents the output of the autoencoder. Decoder 716 further includes at least one intermediate layer 718 that receives the outputs from the nodes in latent space layer 712 and generates multiple output values ​​that decode the encoded information from latent space layer 712 into an output with more information, and the final output layer 720 of the decoder includes more nodes that combine the outputs from intermediate layer 718 to generate the final decoded output. Similar to encoder 704, the nodes in intermediate layer 718 and output layer 720 of decoder 716 include weight parameters generated during training. In autoencoder 700, output layer 720 includes 100 output nodes, matching the number of input nodes in input layer 706; however, in alternative configurations, the output layer optionally includes a different number of nodes compared to the input layer.

[0039] During training, the training system 104 uses a training process including a loss function and an optionally selected penalty function to provide sparsity in the nodes within the autoencoder 700, thereby generating specific numerical weight values ​​in each of the inner layers of the encoder 704, the latent space 712, and the decoder 716. For many autoencoders, the training process is unsupervised because the training objective is to reproduce the output of the simulated input within a predetermined tolerance threshold, thus requiring no additional... a prioriWhile live data is used, supervised training techniques can be employed with autoencoders and other types of ML models. Process 200 enables the training system to provide high-quality PPG training data 128 to the ML model 126 during the training process, which in turn improves the quality of the encoded data in the latent space 712 and other layers of the encoder 704 and decoder 716. Therefore, during subsequent inference operations, even if the input PPG inference data 132 includes noise or other signal artifacts, the trained ML model 126 is appropriately configured to identify the most important features in the PPG inference data 132, while reducing the possibility of mistaking noise for the most important features in the PPG inference data 132. During inference operations after the training process is complete, in some configurations, the autoencoder 700 omits the use of the decoder 716 and generates output based on the encoded data in the latent space layer 712. While the autoencoder is one example of an ML model that can be trained using the techniques described herein, those skilled in the art will recognize other ML models, including but not limited to vector machines, random forest classifiers, convolutional neural networks, recurrent neural networks, long short-term memory neural networks, transformer neural networks, Bayesian neural networks, and other forms of shallow and deep artificial neural networks known in the art, suitable for use with process 200. Various forms of ML models can be employed to measure or otherwise characterize various biomarkers of interest based on inference PPG data. In embodiments of the training system 104 configured to train ML model 126, process 200 optionally terminates with the generation of a trained ML model 126.

[0040] Refer again Figure 2 Process 200 optionally continues after the training process is complete, and the trained ML model 126 is used to perform inference operations to generate information about one or more biomarkers of interest (block 228). Figure 1 In this configuration, system 104 acts as inference system 104. During operation, inference system 104 receives PPG data from inference PPG sensor device system 164, which records PPG measurements corresponding to inference user 162. Of course, one or more of the training users 152 may also be inference user 162, but in many practical cases, inference user 162 and inference PPG sensor device system 164 do not play a role in the early training process, but still rely on the trained ML model 126 to process PPG inference data 132.

[0041] During operation, inference system 104 applies PPG inference data 132 to a trained ML model 126 to generate ML output data 136. In one configuration, inference system 104 applies some or all of the processing steps described in reference blocks 208-220 to the inference data in substantially the same manner as those applied to the training data to improve the quality of the inference PPG data provided to the trained ML model 126. ML output data 136 may be, for example, classifier results or other output data generated by the trained ML model to indicate information about one or more biomarkers of interest. Using the autoencoder 700 of FIG7 as an example, in one configuration, inference system 104 applies PPG inference data 132 to the input layer 706 of autoencoder 700. Processor 108 extracts ML output data 136 from nodes in the latent space layer 712, which encodes a compressed representation of PPG inference data 132 for biomarker classification. These inference output data can be compared with other coded representations of biomarkers, for example, via clustering algorithms or as input to another trained machine learning model, to directly measure biomarkers or provide other analytical information about them. In another configuration, a fully automatic encoder 700 generates output PPG waveforms based on PPG inference data, wherein the output PPG waveforms are provided to a second trained ML model for either further training of the second ML model or for further classification using the second ML model, wherein the second trained ML model can be any of the aforementioned ML models. The advantage of this configuration is that the trained autoencoder 700 reproduces the most important features in the PPG inference data 132 while acting as a filter to remove signal noise and other artifacts that may be present in the raw inference data generated by the inference PPG sensor device system 164. In yet another configuration, different ML models can produce direct classifications of biomarkers in the ML output data 136. In some embodiments, biomarker data may optionally be transmitted directly or indirectly to HCP terminal 174 via an online electronic health record service, enabling HCP 172 to review the information as part of providing medical treatment to inference user 162.

[0042] As described above, the training / inference system 104 is capable of performing both the training and inference operations described herein. However, in an alternative configuration, the trained ML model 126 is transferred to one or more additional computing devices to enable inference operations across a wide range of hardware and software configurations. In a first non-limiting alternative configuration, one or more separate networked server computing systems receive a copy of the ML model 126 and perform the inference process in a similar manner to that described above. In a second non-limiting alternative configuration, the inference PPG sensor device system 164 and the associated computing device receive a copy of the ML model 126, enabling individual inference PPG sensor device systems 164 to generate ML output data 136.

[0043] The embodiments described herein relate to improvements in the processing of PPG sensor data and machine learning for classifying data associated with one or more biomarkers; however, those skilled in the art will understand how these techniques are equally applicable to other forms of non-invasive sensing. For example, the techniques described herein are also applicable to other non-invasive sensing technologies, including radio frequency sensors, terahertz imaging sensors, ultrasonic sensors, etc. In particular, the techniques described herein are applicable to multispectral sensing technologies that generate measurement data for two or more frequencies in the electromagnetic band. Furthermore, in addition to the time-domain signals discussed above, the techniques described herein can also be applied to frequency-domain signals.

[0044] This disclosure has been described in conjunction with what are considered to be the most practical and preferred embodiments. However, these embodiments are presented by way of illustration of improvements to the technology described herein, and these improvements are not strictly limited to the disclosed embodiments. Therefore, those skilled in the art will recognize that this disclosure covers all modifications and alternative arrangements as set forth in the following claims and within the spirit and scope of this disclosure.

Claims

1. A method for training a machine learning model using photoplethysmography data, the method comprising: The training system is used to receive sensor data including at least one photoplethysmography (PPG) dataset, the at least one PPG dataset further including a first series of PPG data corresponding to a first optical wavelength and a second series of PPGs corresponding to a second optical wavelength; The training system is used to generate multiple time segments in the at least one PPG dataset; The training system selects only a portion of the high-frequency noise with a value below a predetermined threshold from the plurality of time segments for inclusion in the training dataset; The training system is used to remove the rolling average for the first series of PPG data and the second series of PPG data from one or more time segments in the training dataset; The training system is used to identify multiple peaks and troughs in each of the first series of PPG data and the second series of PPG data within one or more time segments in the training dataset; The training system is used to generate multiple normalized amplitudes of the multiple peaks and troughs corresponding to each of the first series of PPG data and the second series of PPG data within one or more time segments in the training dataset. The training system identifies multiple waves in multiple normalized amplitudes of multiple peaks and troughs corresponding to each of the first series of PPG data and the second series of PPG data within one or more time segments in the training dataset, each wave being identified based on a peak in the multiple normalized amplitudes. as well as Using the training system, a trained machine learning model is generated using a training process with an untrained machine learning model and the multiple waves identified in the training dataset to classify data related to biomarkers.

2. The method of claim 1, wherein selecting a portion of the plurality of time segments for inclusion in the training dataset further comprises: The high-frequency noise is identified based on the correlation between PPG data in the first series of PPG data and the second series of PPG data for each time segment, and based on the number of zero crossings identified in the first series of PPG data and the second series of PPG data for each time segment.

3. The method of claim 2, wherein selecting a portion of the plurality of time segments for inclusion in the training data further comprises: The training system is used to identify the number of zero crossings in each time segment; as well as The training system selects only time segments from the plurality of time segments that have a number of zero-crossings less than a first predetermined threshold to eliminate high-frequency noise and a number of zero-crossings greater than a second predetermined threshold to eliminate low-frequency noise, for inclusion in the training dataset.

4. The method of claim 2, wherein selecting a portion of the plurality of time segments for inclusion in the training data further comprises: The training system is used to identify the number of zero crossings in each time segment; as well as The training system utilizes only time segments among the plurality of time segments that have a number of zero-crossings greater than a first predetermined threshold to eliminate low-frequency noise for inclusion in the training dataset.

5. The method of claim 1, wherein identifying each of the plurality of waves further comprises: Using the training system, each spike is identified based on a series of consecutive sampling points that monotonically rise to a peak value and then monotonically decline from the peak value while remaining above a predetermined threshold among the plurality of normalized amplitudes.

6. The method of claim 1, wherein the first light wavelength is in the range of about 500 nm to 600 nm corresponding to green light, and the second light wavelength is in the range of about 750 nm to 1400 nm corresponding to near-infrared light.

7. The method of claim 1, wherein the autoencoder receives the training dataset to generate input training data for training the machine learning model.

8. The method of claim 1, further comprising: The system utilizes an inference system to receive sensor data including at least one inference PPG dataset, the at least one PPG dataset further including a first series of PPG data corresponding to the first optical wavelength and a second series of PPG data corresponding to the second optical wavelength; as well as The reasoning system is used to perform the reasoning process to classify the data related to the biomarker in the at least one reasoning PPG dataset using the trained machine learning model.

9. The method of claim 8, wherein the inference system is the training system.

10. The method of claim 1, further comprising: The trained machine learning model is provided to an inference system, which is separate from the training system.

11. A system for training a machine learning model using photoplethysmography data, the system comprising: The memory, which is configured to store: Program instructions, Sensor data, comprising at least one photoplethysmography (PPG) dataset, said at least one PPG dataset further comprising a first series of PPG data corresponding to a first optical wavelength and a second series of PPG data corresponding to a second optical wavelength, and Data, which corresponds to machine learning models; as well as A processor, operatively connected to the memory, is configured to execute stored program instructions to: Multiple time segments are generated in the at least one PPG dataset. Only a portion of the multiple time segments containing high-frequency noise below a predetermined threshold is selected for inclusion in the training dataset. Remove the rolling average for the first series of PPG data and the second series of PPG data from one or more time segments in the training dataset. Identify multiple peaks and troughs in each of the first series of PPG data and the second series of PPG data within one or more time segments in the training dataset. Generate multiple normalized amplitudes of the multiple peaks and troughs corresponding to each of the first series of PPG data and the second series of PPG data within one or more time segments in the training dataset. Multiple waves are identified within the multiple normalized amplitudes of the multiple peaks and troughs corresponding to each of the first series of PPG data and the second series of PPG data within one or more time segments of the training dataset. Each wave is identified based on a peak among the multiple normalized amplitudes. The training process is used to generate a trained machine learning model stored in the memory using an untrained machine learning model and the multiple waves identified in the training dataset, in order to classify data related to biomarkers.

12. The system of claim 11, wherein the processor is further configured to: The high-frequency noise is identified based on the correlation between PPG data in the first series of PPG data and the second series of PPG data for each time segment, and based on the number of zero crossings identified in the first series of PPG data and the second series of PPG data for each time segment.

13. The system of claim 12, wherein the processor is further configured to: Identify the number of zero-crossing points in each time segment; and Only time segments with a number of zero-crossings less than a first predetermined threshold are selected from the plurality of time segments to eliminate high-frequency noise, and time segments with a number of zero-crossings greater than a second predetermined threshold are selected to eliminate low-frequency noise, for inclusion in the training dataset.

14. The system of claim 12, wherein the processor is further configured to: Identify the number of zero-crossing points in each time segment; and Only time segments with a number of zero-crossings greater than a first predetermined threshold are selected from the plurality of time segments to eliminate low-frequency noise for inclusion in the training dataset.

15. The system of claim 11, wherein the processor is further configured to: Each spike is identified by a series of consecutive sampling points that monotonically rise to a peak value and then monotonically decline from the peak value while remaining above a predetermined threshold.

16. The system of claim 11, wherein the first light wavelength is in the range of approximately 500 nm to 600 nm corresponding to green light, and the second light wavelength is in the range of approximately 750 nm to 1400 nm corresponding to near-infrared light.

17. The system of claim 11, wherein the autoencoder receives the training dataset to generate input training data for training the machine learning model.

18. The system of claim 11, wherein the memory is further configured to: The storage includes sensor data comprising at least one inference PPG dataset, the at least one PPG dataset further comprising a first series of PPG data corresponding to the first optical wavelength and a second series of PPG data corresponding to the second optical wavelength; and The processor is further configured to execute the stored program instructions to: An inference process is performed to classify the data related to the biomarker in the at least one inference PPG dataset using the trained machine learning model.

19. The system of claim 11, wherein the system is further configured to: The trained machine learning model is provided to a separate inference system configured to use the trained machine learning model to perform an inference process to classify the data related to the biomarker in at least one inference PPG dataset using the trained machine learning model.

Images