Heart rate prediction and model training method and system, wearable device and medium

Abstract

The invention relates to a heart rate prediction and model training method and system, wearable equipment and a medium. The training method comprises the following steps: acquiring a training sample set and a corresponding label set; wherein each training sample of the training sample set comprises a photoelectric volume pulse wave sequence and a corresponding acceleration sequence, and the label set is a real heart rate; preprocessing each training sample to obtain a corresponding photoelectric volume pulse wave frequency spectrum sequence and an acceleration frequency spectrum sequence; all the photoelectric volume pulse wave frequency spectrum sequences and the acceleration frequency spectrum sequences are input into a heart rate prediction model to be trained, and corresponding predicted heart rates are obtained; and calculating the difference degree between the predicted heart rate and the corresponding real heart rate, and updating parameters of the heart rate prediction model based on the difference degree to obtain a trained heart rate prediction model. According to the method, the problems of high signal processing cost and high calculation complexity caused by the use of a complex adaptive filtering process in the prior art are solved, and the heart rate prediction accuracy can be remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of signal processing technology, and in particular to a heart rate prediction and model training method, system, wearable device and medium. Background Art

[0002] With people's increasing attention to medical health, especially driven by digital health, the development of medical wearable devices is in full swing. Heart rate monitoring has become a "standard feature" of wearable devices, and real-time heart rate monitoring technology has been widely used. Wearable reflective photoplethysmography (PPG) sensors can be integrated into watches or straps to provide instantaneous heart rate with minimal inconvenience to users. However, the sensor is sensitive to motion artifacts, which often leads to inaccurate heart rate estimates.

[0003] In order to solve the noise and artifact problems in PPG signals, the industry usually adopts a combination of signal preprocessing and adaptive filtering to measure heart rate through signal denoising, dynamic filtering and heart rate extraction. However, due to the complex adaptive filtering calculation, this method increases the power consumption of the device when worn for a long time. Therefore, it is necessary to provide a heart rate prediction and model training method, system, wearable device and medium. Summary of the invention

[0004] In view of the shortcomings of the prior art described above, the purpose of the present invention is to provide a heart rate prediction and model training method, system, wearable device and medium, which improves the problems of high cost and computational complexity when the prior art uses a combination of signal preprocessing and adaptive filtering to predict the user's heart rate.

[0005] To achieve the above-mentioned purpose and other related purposes, the present invention provides a training method for a heart rate prediction model, the training method comprising: obtaining a training sample set and a corresponding label set; wherein each training sample in the training sample set comprises a photoplethysmogram sequence and a corresponding acceleration sequence, and the label set is a real heart rate; preprocessing each training sample to obtain a corresponding photoplethysmogram spectrum sequence and an acceleration spectrum sequence; inputting all photoplethysmogram spectrum sequences and acceleration spectrum sequences into a heart rate prediction model to be trained to obtain a corresponding predicted heart rate; calculating the difference between the predicted heart rate and the corresponding real heart rate, and updating the parameters of the heart rate prediction model based on the difference to obtain a trained heart rate prediction model.

[0006] In one embodiment of the present invention, the training samples are obtained by preprocessing the initial photoplethysmogram sequence and the initial acceleration sequence. The process of obtaining the training samples includes: obtaining the photoplethysmogram within a preset sampling time and the acceleration within the preset time to form an initial photoplethysmogram sequence and an initial acceleration sequence; based on a preset bandwidth, bandpass filtering the initial photoplethysmogram sequence and the initial acceleration sequence respectively to obtain a filtered photoplethysmogram sequence and a filtered acceleration sequence; based on a sliding window of a preset size, sliding sampling is performed on the filtered photoplethysmogram sequence and the filtered acceleration sequence respectively to obtain multiple photoplethysmogram sequences and multiple acceleration sequences, and each photoplethysmogram sequence and the corresponding acceleration sequence is used as a training sample.

[0007] In one embodiment of the present invention, the photoplethysmogram is a multi-channel signal, and the preprocessing of each training sample to obtain a corresponding photoplethysmogram spectrum sequence and acceleration spectrum sequence includes: integrating the photoplethysmogram sequences of each channel to obtain a comprehensive photoplethysmogram sequence;

[0008] The comprehensive photoplethysmogram sequence and the acceleration sequence are respectively subjected to fast Fourier transformation to obtain a photoplethysmogram spectrum sequence and an acceleration spectrum sequence accordingly.

[0009] In one embodiment of the present invention, the step of integrating the photoplethysmogram sequences of each channel to obtain a comprehensive photoplethysmogram sequence includes: averaging the photoplethysmogram sequences of each channel to obtain a comprehensive photoplethysmogram sequence.

[0010] In one embodiment of the present invention, the fast Fourier transform is performed on the integrated photoplethysmogram sequence and the acceleration sequence respectively to obtain a photoplethysmogram spectrum sequence and an acceleration spectrum sequence, including: fast Fourier transform is performed on the integrated photoplethysmogram sequence and the acceleration sequence respectively to obtain a first photoplethysmogram spectrum sequence and a first acceleration spectrum sequence; the first photoplethysmogram spectrum sequence and the first acceleration spectrum sequence are intercepted according to a preset spectrum interval to generate a second photoplethysmogram spectrum sequence and a second acceleration spectrum sequence; the second photoplethysmogram spectrum sequence and the second acceleration spectrum sequence are standardized to obtain a photoplethysmogram spectrum sequence and an acceleration spectrum sequence.

[0011] In an embodiment of the present invention, when the photoplethysmogram is a multi-channel signal, the step of acquiring a training sample set further includes: setting at least part of the photoplethysmogram of at least one channel to zero to form a new training sample.

[0012] In one embodiment of the present invention, the difference is obtained by weighting the smoothed L1 loss of the predicted heart rate and the corresponding real heart rate with the weight of the corresponding real heart rate, and the weight generation process of the real heart rate is: according to the number of real heart rates in the label set, all real heart rates are divided into a preset number of heart rate groups based on the equal frequency binning method; for each heart rate group: Gaussian convolution is performed on the number of real heart rates included in the heart rate group to obtain the number of smoothed real heart rates; according to the number of smoothed real heart rates, the weight of each real heart rate in the corresponding heart rate group is determined.

[0013] In one embodiment of the present invention, a heart rate prediction method is also provided, the prediction method comprising: obtaining a photoplethysmogram sequence and a corresponding acceleration sequence to be analyzed; preprocessing the photoplethysmogram sequence and the corresponding acceleration sequence respectively to obtain a corresponding photoplethysmogram spectrum sequence and an acceleration spectrum sequence; inputting the photoplethysmogram spectrum sequence and the acceleration spectrum sequence into a heart rate prediction model to obtain a predicted heart rate.

[0014] In one embodiment of the present invention, the step of inputting the photoplethysmogram spectrum sequence and the acceleration spectrum sequence into a heart rate prediction model to obtain a predicted heart rate comprises: inputting the photoplethysmogram spectrum sequence and the acceleration spectrum sequence into the heart rate prediction model to obtain an initial predicted heart rate; and performing exponentially weighted movement on the initial predicted heart rate based on a pre-stored historical heart rate to obtain a final predicted heart rate.

[0015] In one embodiment of the present invention, a heart rate prediction model training system is also provided, the system comprising: a data acquisition module, used to acquire a training sample set and a corresponding label set; wherein each training sample in the training sample set comprises a photoplethysmogram sequence and a corresponding acceleration sequence, and the label set is a real heart rate; a preprocessing module, used to preprocess each training sample to obtain a corresponding photoplethysmogram spectrum sequence and an acceleration spectrum sequence; a heart rate prediction module, used to input all photoplethysmogram spectrum sequences and acceleration spectrum sequences into a heart rate prediction model to be trained to obtain a corresponding predicted heart rate; a parameter updating module, used to calculate the difference between the predicted heart rate and the corresponding real heart rate, and update the parameters of the heart rate prediction model based on the difference to obtain a trained heart rate prediction model.

[0016] In one embodiment of the present invention, a wearable device is also provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the wearable device implements any of the heart rate prediction methods described above.

[0017] In one embodiment of the present invention, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor of a computer, the computer executes any of the above-mentioned heart rate prediction model training methods or heart rate prediction methods.

[0018] As described above, a heart rate prediction and model training method, system, wearable device and medium of the present invention have the following beneficial effects: the original photoelectric volume pulse wave sequence and acceleration sequence are respectively converted from time domain signals into spectral feature sequences, so as to more clearly display information related to heart rate changes on specific frequency components. And this spectral sequence is input as a training sample into the heart rate prediction model to be trained, and by learning the physiological signal of the photoelectric volume pulse wave and the relevant motion information of the acceleration, a more accurate heart rate prediction result can be obtained by multimodally fusing these two types of information. In addition, the present invention uses the end-to-end learning ability of the model to directly learn the main frequency characteristics of the signal from the input frequency domain characteristics, and predict the corresponding heart rate, which improves the use of complex adaptive filtering processes in the prior art, resulting in high signal processing costs and computational complexity, and can significantly improve the accuracy of heart rate prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic flow chart of a method for training a heart rate prediction model provided by an embodiment of the present invention;

[0020] Figure 2 A flow chart of a heart rate prediction method provided by an embodiment of the present invention

[0021] Figure 3 Shown is a structural block diagram of a training system for a heart rate prediction model provided by an embodiment of the present invention;

[0022] Figure 4 Shown is a structural schematic diagram of a wearable device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0024] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0025] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0026] The inventors found that in order to solve the noise and artifact problems existing in PPG signals, the industry usually adopts a combination of signal preprocessing and adaptive filtering, that is, firstly, the PPG signal is preliminarily processed, such as denoising, baseline correction, etc., to reduce the impact of environmental interference and instrument errors on heart rate measurement. Then, according to the characteristics of the PPG signal and the heart rate range, an adaptive filter is designed, and the filter parameters are dynamically adjusted to effectively filter out noise and artifacts that are not related to the heart rate, such as motion artifacts and respiratory fluctuations. Finally, in the filtered PPG signal, a specific algorithm (such as peak detection, frequency domain analysis, etc.) is used to extract the heart rate information to achieve accurate measurement of the heart rate. However, this method has the following disadvantages: (1) Power consumption problem: The adaptive filtering algorithm usually requires a high computational complexity, resulting in increased device power consumption, which is a considerable challenge for wristband devices that need to be worn for a long time. (2) Performance bottleneck: Although adaptive filtering can effectively filter out some noise, its performance may be limited in the face of complex and changeable noise environments, resulting in a decrease in the accuracy of heart rate measurement. (3) Cost considerations: Implementing adaptive filtering algorithms requires higher-level processor and memory resources, which increases the manufacturing cost of the device and limits its application in the low-end market. Although the combination of signal preprocessing and adaptive filtering solves the problem of PPG and heart rate artifacts to a certain extent, it has obvious shortcomings in terms of power consumption, performance and cost.

[0027] In order to improve the above problems, the present invention provides a training method for a heart rate prediction model, which converts the original photoplethysmogram sequence and acceleration sequence from time domain signals into spectral feature sequences respectively, so as to more clearly display information related to heart rate changes on specific frequency components. And this spectral sequence is input as a training sample into the heart rate prediction model to be trained. By learning the physiological signals of the photoplethysmogram and the relevant motion information of the acceleration, a more accurate heart rate prediction result can be obtained by multimodally fusing these two types of information. In addition, the present invention uses the end-to-end learning ability of the model to directly learn the main frequency characteristics of the signal from the input frequency domain characteristics, and predicts the corresponding heart rate, which improves the use of complex adaptive filtering processes in the prior art, resulting in high signal processing costs and computational complexity, and can significantly improve the accuracy of heart rate prediction.

[0028] See also Figure 1 ,The training method of the heart rate prediction model includes the following steps:

[0029] S11. Obtain a training sample set and a corresponding label set; wherein each training sample in the training sample set includes a photoplethysmogram sequence and a corresponding acceleration sequence, and the label set is a real heart rate.

[0030] The acceleration in the present invention is a multi-axis acceleration, including but not limited to three-axis acceleration, six-axis acceleration and nine-axis acceleration. For the convenience of description, the following content of the present invention is described by taking three-axis acceleration as an example, but it should be understood that other multi-axis accelerations are also applicable to the present invention. To ensure the diversity and accuracy of the model data, the present invention uses two sets of experimental equipment: one set is a multi-channel PPG sensor with a preset sampling rate (such as 25HZ) and a three-axis accelerometer (Acceleromete, ACC), which is used to collect physiological signals at the wrist within a preset time length, and obtain a photoelectric volume pulse wave sequence and an acceleration sequence within the preset time length; the other set is a professional heart rate belt, which is used to measure the user's real heart rate as the source of the heart rate label. All collected data are synchronized by time alignment. The data of the present invention covers a variety of sports and daily life scenes, including but not limited to sitting, daily activities, going up and down stairs, outdoor walking, outdoor jogging, treadmill variable speed running and cycling, etc., to ensure the collection of heart rate data under different exercise intensities and environments. It should be noted that the training samples correspond to the real heart rate one by one, and each photoplethysmogram sequence within a preset time and the acceleration sequence within the preset time correspond to a real heart rate, which is the heart rate value at the end of the preset time. For example, when the preset time is 8 seconds, the label corresponding to the photoplethysmogram sequence within the 8 seconds and the acceleration sequence within the 8 seconds is the real heart rate at the 8th second.

[0031] In one embodiment of the present invention, the training samples are obtained by preprocessing the initial photoplethysmogram sequence and the initial acceleration sequence. The process of obtaining the training samples includes:

[0032] Acquire the photoplethysmogram within a preset sampling time and the acceleration within the preset time to form an initial photoplethysmogram sequence and an initial acceleration sequence;

[0033] Based on a preset bandwidth, bandpass filtering is performed on the initial photoplethysmogram sequence and the initial acceleration sequence to obtain a filtered photoplethysmogram sequence and a filtered acceleration sequence;

[0034] Based on a sliding window of a preset size, the filtered photoplethysmogram sequence and the filtered acceleration sequence are respectively subjected to sliding sampling to obtain a plurality of photoplethysmogram sequences and a plurality of acceleration sequences, and each photoplethysmogram sequence and the corresponding acceleration sequence are used as a training sample.

[0035] The photoelectric volume pulse wave signal and the acceleration signal are continuously collected from the wearable device within the preset sampling time to form an initial photoelectric volume pulse wave sequence and an initial acceleration sequence. As an example, the photoelectric volume pulse wave data and the acceleration data are continuously collected according to the preset sampling frequency (such as 25HZ). When the cumulative collection time reaches the preset sampling time, the sampling is terminated, and the photoelectric volume pulse wave signal and the acceleration signal obtained within the sampling time are the initial photoelectric volume pulse wave sequence and the initial acceleration sequence. In order to retain the main frequency related to the heart rate and reduce irrelevant noise interference, the present invention also needs to perform bandpass filtering on the initial photoelectric volume pulse wave sequence and the initial acceleration sequence according to the preset bandpass range (such as 0-4Hz), so that the filtered data can cover the common heart rate range, and obtain the filtered photoelectric volume pulse wave sequence and the filtered acceleration sequence. The filtered sequence is divided according to the sliding window to form a plurality of training samples. Specifically, the length of the sliding window is preset to be the same as the number of sampling points corresponding to the preset duration, so that the sliding window intercepts a data segment of a fixed length (such as 8 seconds) each time to form a training sample, and translates the window backward according to the preset step length (such as 1 second), and intercepts the data of the same length again as the next training sample. Thus, the photoelectric volume pulse wave sequence and acceleration sequence in each training sample correspond to the signal in the same time interval. Therefore, the photoelectric volume pulse wave sequence and acceleration sequence in the same training sample represent the signal in the same time interval. It should be noted that the step length of the sliding window can be adaptively set based on actual needs. In this embodiment, the length of the sliding window is 8 seconds and the step length is 1 second, so that the model extracts features from the photoelectric volume pulse wave sequence and acceleration sequence of the current preset duration (such as 8 seconds), and outputs the heart rate value at the end of the duration. As the sliding window slides, the model predicts the heart rate value of the next duration in a postponed manner to maintain the continuity and real-time nature of the prediction, thereby realizing segmented processing of the signal and real-time heart rate prediction.

[0036] S12. Preprocess each training sample to obtain corresponding photoplethysmographic spectrum features and acceleration spectrum features.

[0037] Specifically, in one embodiment of the present invention, the photoplethysmogram is a multi-channel signal, and the preprocessing of each training sample to obtain a corresponding photoplethysmogram spectrum sequence and acceleration spectrum sequence includes:

[0038] The photoplethysmogram sequences of each channel are integrated to obtain a comprehensive photoplethysmogram sequence;

[0039] The comprehensive photoplethysmogram sequence and the acceleration sequence are respectively subjected to fast Fourier transformation to obtain a photoplethysmogram spectrum sequence and an acceleration spectrum sequence accordingly.

[0040] For each training sample, the training sample includes a photoelectric volume pulse wave signal and an acceleration signal within a preset time length. In one embodiment, in order to make the final predicted heart rate more accurate, a multi-channel photoelectric volume pulse wave is used so as to capture different photoelectric volume pulse wave information at the same time point. Therefore, for a multi-channel photoelectric volume pulse wave signal, the photoelectric volume pulse waves of each channel need to be integrated into a single-channel comprehensive photoelectric volume pulse wave sequence. It should be noted that the comprehensive photoelectric volume pulse wave can be obtained by weighting the photoelectric volume pulse waves of each channel, or by calculating the average value of the photoelectric volume pulse waves of each channel. As an example, in one embodiment of the present invention, the photoelectric volume pulse wave sequence of each channel is integrated to obtain a comprehensive photoelectric volume pulse wave sequence, including: averaging the photoelectric volume pulse wave sequence of each channel to obtain a comprehensive photoelectric volume pulse wave sequence. By averaging the values ​​of each channel at the same time point, a comprehensive photoelectric volume pulse wave is obtained, so that while retaining the main information characteristics of the photoelectric volume pulse wave, the influence of random noise and artifacts can be further reduced, thereby improving the accuracy and stability of heart rate detection. For the integrated photoplethysmogram sequence, a fast Fourier transform is performed on it to obtain a spectrum sequence of the photoplethysmogram, wherein each spectrum feature in the spectrum sequence corresponds to the corresponding integrated photoplethysmogram. The acceleration in the present invention is a multi-axis acceleration, wherein the acceleration signal can be expressed as the decomposed components in each axis direction or as the magnitude of the overall acceleration after synthesis, which is not limited here. When the acceleration signal is expressed as the decomposed components of each axis, for the acceleration sequence, a fast Fourier transform is performed on the acceleration sequence of each axis to obtain the corresponding acceleration spectrum sequence.

[0041] In one embodiment of the present invention, the fast Fourier transform is performed on the integrated photoplethysmogram sequence and the acceleration sequence respectively to obtain a photoplethysmogram spectrum sequence and an acceleration spectrum sequence, including:

[0042] Performing fast Fourier transform on the comprehensive photoplethysmogram sequence and the acceleration sequence respectively, and obtaining a first spectrum sequence of photoplethysmogram and a first spectrum sequence of acceleration respectively;

[0043] The first spectrum sequence of the photoplethysmogram and the first spectrum sequence of acceleration are intercepted according to a preset spectrum interval to generate a second spectrum sequence of the photoplethysmogram and a second spectrum sequence of acceleration;

[0044] The second spectrum sequence of the photoplethysmogram and the second spectrum sequence of acceleration are standardized to obtain a corresponding photoplethysmogram spectrum sequence and an acceleration spectrum sequence.

[0045] After the above process, the comprehensive photoplethysmogram sequence and the acceleration sequence of each axis are sequences of length N. The comprehensive photoplethysmogram sequence of length N and each acceleration sequence are subjected to a 1024-point fast Fourier transform, and the time domain data is converted into frequency domain data to obtain the first spectrum sequence of the comprehensive photoplethysmogram and the first spectrum sequence of acceleration of length N. In order to remove irrelevant frequency information so that the model pays more attention to the frequency band related to the heart rate, the present invention intercepts the complete spectrum sequence, retains only the corresponding frequency points in the preset spectrum interval (such as 0.6Hz to 3.72Hz, corresponding to the heart rate range of 36bpm to 223bpm), and obtains the corresponding comprehensive photoplethysmogram second spectrum sequence and acceleration second spectrum sequence. Among them, 128 frequency domain data points are retained in the comprehensive photoplethysmogram second spectrum sequence and the acceleration second spectrum sequence. Considering that the spectrum amplitudes of different data will be greatly different, in order to make the model easier to train and compare and fuse between different spectrum features, the spectrum features can be standardized to unify the dimensions. Among them, the standardization processing method includes but is not limited to Z-score standardization or Min-Max normalization. In one embodiment of the present invention, the standardization is Z-score standardization, and the mean and standard deviation of the intercepted PPG spectrum features and ACC spectrum features are calculated separately, and they are converted into a data distribution form with zero mean and unit variance. The final N ch The time-frequency spectrum constitutes the input of the deep learning model, and each segment of data is represented as a 4×128 matrix, including the comprehensive PPG and acceleration spectrum features of the three axes. The label of each segment is the real heart rate at the end of the preset time provided by the heart rate belt.

[0046] Furthermore, although the present invention can collect multi-channel photoelectric volume pulse wave signals for analysis to improve the accuracy of heart rate prediction, it is considered that in actual user applications, due to problems such as device wearing, light interference or poor skin contact, there may be a situation where one or more channel signals are lost, resulting in inaccurate model prediction results. To improve this problem, in one embodiment of the present invention, when the photoelectric volume pulse wave is a multi-channel signal, the acquisition of the training sample set also includes: setting at least part of the photoelectric volume pulse wave of at least one channel to zero to form a new training sample. During the training process, a single channel is randomly selected to be zeroed or multiple channels are simultaneously zeroed to improve the diversity of samples. In order to further increase the diversity of data, only part of the data of the selected channel signal can be set to zero, such as setting the data of channel 1 from the 50th to the 100th point to zero, and retaining the signals at other time points. The data generated after each zeroing constitutes a new training sample together with the original acceleration sequence, and the model is trained together with the original non-zeroed sample as training data. In this way, the stability of heart rate monitoring can be ensured when some channels are abnormal.

[0047] S13. Input all the photoplethysmographic spectrum features and acceleration spectrum features into the heart rate prediction model to be trained to obtain the corresponding predicted heart rate.

[0048] For each training sample: the corresponding photoelectric volume pulse wave spectrum sequence and acceleration spectrum sequence are input into the heart rate prediction model. It should be noted that the heart rate prediction model can be any model that can achieve accurate prediction of heart rate, including but not limited to LSTM, CNN, Transformer architecture models, hybrid models, etc. In one embodiment of the present invention, the heart rate prediction model is a CNN-LSTM architecture, so that a more robust and accurate heart rate prediction can be obtained in complex and changeable physiological signals. The PPG and ACC signals are respectively input into their respective CNN branches. The CNN module uses multi-layer one-dimensional convolution to extract local time series features, capture the heart rate main frequency information in the PPG signal and the motion artifact features in the ACC signal, and reduce the dimension and prevent overfitting through pooling and Dropout layers. The extracted features are then fused into a unified time series feature representation, and feature enhancement and synthesis are performed through an attention mechanism or direct splicing. The fused time series features are input into a double-layer LSTM module, which captures the long-term dependencies and dynamic changes in the signal through a gating mechanism, and outputs a hidden state containing global information. The hidden state of the LSTM is passed to the fully connected layer for nonlinear mapping to predict the heart rate value.

[0049] S4. Calculate the difference between the predicted heart rate and the corresponding true heart rate, and update the parameters of the heart rate prediction model based on the difference to obtain a trained heart rate prediction model.

[0050] For each training sample, the loss between the predicted heart rate and the corresponding true heart rate is calculated. The heart rate prediction model continuously optimizes parameters through the loss function and utilizes the complementary characteristics of PPG and ACC signals to effectively improve the robustness and accuracy of heart rate prediction in motion artifacts and complex environments.

[0051] In one embodiment of the present invention, the difference is obtained by weighting the smoothed L1 loss of the predicted heart rate and the corresponding real heart rate with the weight of the corresponding real heart rate. The weight generation process of the real heart rate is:

[0052] According to the number of real heart rates in the label set, all real heart rates are divided into a preset number of heart rate groups based on an equal frequency binning method;

[0053] For each heart rate group: performing Gaussian convolution on the number of real heart rates included in the heart rate group to obtain the number of smoothed real heart rates;

[0054] The weight of each real heart rate in the corresponding heart rate group is determined according to the number of smoothed real heart rates.

[0055] In order to solve the problem of uneven collection caused by low heart rate data in daily scenarios and large individual heart rate differences, the present invention additionally designs a weight coefficient to guide the network to evenly focus on data in different heart rate ranges in the data set. After randomly dividing the data set into a training set and a test set, the n real heart rates in the label set are sorted in ascending order, and all the n real heart rates are divided into a preset number (such as m) of groups based on the equal frequency binning method. The interval range of the real heart rate of the kth group is (i k ,j k ],j k ≥i k For all k. For each grouping interval (i k ,j k ], count the number of true heart rates falling within this interval N k , where the heart rate quantity vector is assumed to be N = {N1, N2, ..., N m}, represents the number of true heart rates contained in each group. Gaussian convolution is performed on the true heart rate number of each group as shown in formula (1) to obtain the smoothed true heart rate number:

[0056]

[0057] Among them, E k is the real heart rate number after smoothing of the kth group, N k is the true heart rate number of the kth group before smoothing, G is the Gaussian kernel, is the convolution operation. For the real heart rate in each group, the corresponding weight is calculated as shown in formula (2):

[0058]

[0059] Where K is a constant, usually taken as m (number of intervals) to ensure the normalization of weights, and γ is a preset adjustment parameter used to control the dynamic range of weights. k Assign to all the real heart rates in the group to ensure that all the real heart rates in the same group have equal weights. When calculating the difference between the predicted heart rate and the corresponding real heart rate, it can be calculated according to formula (3):

[0060] L l =SmoothL1(x)×W k (3)

[0061] Among them, L l is the weighted loss value of the lth training sample, is the difference between the true heart rate and the predicted heart rate of the training sample, SmoothL1(x) is the smooth L1 loss function, W kis the weighting coefficient corresponding to the heart rate group to which the training sample belongs. The present invention performs Gaussian smoothing on each binned data according to the heart rate distribution to generate a weight coefficient. The weight coefficient is added to the loss function of the network during training, so that the model pays appropriate attention to the data of different heart rate segments during training, reduces the offset, and thus improves the stability and robustness of the model. Through continuous iterative optimization, the weighted loss between the predicted value and the true heart rate is minimized, so that the model can gradually converge to a lower and stable loss value. Finally, by observing the performance changes of the model during the training process, the best model parameters are selected as the final training results.

[0062] See also Figure 2 In one embodiment of the present invention, a heart rate prediction method is also provided, the prediction method comprising:

[0063] S21, obtaining a photoplethysmogram sequence and a corresponding acceleration sequence to be analyzed;

[0064] S22, preprocessing the photoplethysmogram and the acceleration respectively to obtain corresponding photoplethysmogram spectrum characteristics and acceleration spectrum characteristics;

[0065] S23, inputting the photoplethysmography spectrum characteristics and acceleration spectrum characteristics into a heart rate prediction model to obtain a predicted heart rate.

[0066] Photoplethysmography and multi-axis acceleration data are continuously collected from wearable devices within a preset time period to obtain the initial PPG sequence and ACC sequence corresponding to the period. These original sequences are preprocessed, and the time domain signals are converted into frequency domain features through bandpass filtering and fast Fourier transform, and the specific frequency range related to heart rate is intercepted. The intercepted PPG and ACC spectrum features are input into the heart rate prediction model. After feature fusion and time series modeling, the model outputs the corresponding predicted heart rate value to achieve accurate evaluation of the heart rate state in the target period.

[0067] In one embodiment of the present invention, the step of inputting the photoplethysmogram spectrum sequence and acceleration spectrum features into a heart rate prediction model to obtain a predicted heart rate includes:

[0068] Inputting the photoplethysmogram spectrum sequence and acceleration spectrum sequence into a heart rate prediction model to obtain an initial predicted heart rate;

[0069] Based on the pre-stored historical heart rate, the initial predicted heart rate is exponentially weighted shifted to obtain a final predicted heart rate.

[0070] After the photoelectric volume pulse wave spectrum sequence and the acceleration spectrum sequence are input into the heart rate prediction model, an initial predicted heart rate is obtained. Although the initial predicted heart rate can already reflect the judgment of the heart rate prediction model on the current signal segment, considering that in actual applications, the predicted heart rate data may have certain anomalies, resulting in deviations between the predicted results and the actual situation. To improve this situation, the present invention pre-stores a preset number of historical predicted heart rates, and calculates new predicted values ​​by assigning exponentially decreasing weights to past predicted values ​​and current predicted values, and using weighted historical values ​​and current observed values. The weight decreases over time, and a higher weight is given to the latest data. It can respond to the latest trend changes more quickly and can adapt to rapidly changing heart rates. In addition, a historical backtracking adjustment method can also be used to smooth the predicted heart rate results to further improve the continuity and accuracy of the heart rate data. The historical backtracking adjustment step detects the smoothed heart rate data through a sliding window to identify sudden anomalies or significant instantaneous jumps therein. When an abnormal point is detected, the point can be interpolated and replaced using the adjacent smooth values ​​before and after, thereby removing the impact of the sudden abnormality and ensuring the overall smoothness and rationality of the heart rate curve.

[0071] See also Figure 3 , the training system 100 of the heart rate prediction model includes: a data acquisition module 110, a preprocessing module 120, a heart rate prediction module 130 and a parameter updating module. The above-mentioned data acquisition module 110 is used to obtain a training sample set and a corresponding label set; wherein each training sample of the training sample set includes a photoelectric volume pulse wave sequence and a corresponding acceleration sequence, and the label set is a real heart rate. The preprocessing module 120 is used to preprocess each training sample to obtain a corresponding photoelectric volume pulse wave spectrum sequence and an acceleration spectrum sequence. The heart rate prediction module 130 is used to input all photoelectric volume pulse wave spectrum sequences and acceleration spectrum sequences into the heart rate prediction model to be trained to obtain the corresponding predicted heart rate. The parameter updating module 140 is used to calculate the difference between the predicted heart rate and the corresponding real heart rate, and update the parameters of the heart rate prediction model based on the difference to obtain a trained heart rate prediction model.

[0072] For the specific limitations of the training system for the heart rate prediction model, please refer to the limitations of the training method for the heart rate prediction model in the above text, which will not be repeated here. Each module in the above-mentioned heart rate prediction model training system can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware format, or can be stored in the memory in the computer device in software format, so that the processor can call the operations corresponding to the above modules.

[0073] It should be noted that, in order to highlight the innovative part of the present invention, the present embodiment does not introduce modules that are not closely related to solving the technical problem proposed by the present invention, but this does not mean that there are no other modules in the present embodiment.

[0074] See also Figure 4 The wearable device 1 may include a memory 12, a processor 13 and a bus, and may also include a computer program stored in the memory 12 and executable on the processor 13, such as a heart rate prediction program.

[0075] Among them, the memory 12 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 12 can be an internal storage unit of the wearable device 1, such as a mobile hard disk of the wearable device 1. In other embodiments, the memory 12 can also be an external storage device of the wearable device 1, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the wearable device 1. Further, the memory 12 can also include both the internal storage unit of the wearable device 1 and the external storage device. The memory 12 can not only be used to store application software and various types of data installed on the wearable device 1, such as the code for heart rate prediction, etc., but also can be used to temporarily store data that has been output or is to be output.

[0076] In some embodiments, the processor 13 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The processor 13 is the control core (Control Unit) of the wearable device 1, and uses various interfaces and lines to connect the various components of the entire wearable device 1, and executes or executes programs or modules (such as heart rate prediction programs, etc.) stored in the memory 12, and calls the data stored in the memory 12 to execute various functions of the wearable device 1 and process data.

[0077] The processor 13 executes the operating system and various installed applications of the wearable device 1. The processor 13 executes the applications to implement the steps in the above-mentioned heart rate prediction method.

[0078] Exemplarily, the computer program may be divided into one or more modules, which are stored in the memory 12 and executed by the processor 13 to complete the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the wearable device 1. For example, the computer program may be divided into a data acquisition module 110, a preprocessing module 120, a heart rate prediction module 130, and a parameter update module.

[0079] The above-mentioned integrated unit implemented in the form of a software function module can be stored in a computer-readable storage medium, and the computer-readable storage medium can be non-volatile or volatile. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a computer device, or a network device, etc.) or a processor to perform part of the functions of the training method of the heart rate prediction model described in each embodiment of the present application.

[0080] In summary, the present invention discloses a method, system, wearable device and medium for training a heart rate prediction and model. The multi-channel PPG sensor can collect multiple sets of data at the same time. By comparison and calibration, the errors caused by factors such as skin color, hair, sweat or ambient light are significantly reduced, thereby providing more accurate heart rate readings. Even if the data of some channels are interfered with or abnormal, the remaining channels can continue to provide effective heart rate information to ensure the continuity and stability of heart rate monitoring. The three-channel ACC accelerometer can accurately monitor the user's motion state, distinguish static heart rate from exercise heart rate, avoid the interference of motion artifacts on heart rate measurement, and make the heart rate data more accurate and reliable. By adding a weight coefficient to the loss function, it is ensured that data in different heart rate ranges are balanced and the model is prevented from being biased towards certain specific heart rate intervals. The method of frequency domain signal processing can not only improve the efficiency and noise suppression ability of signal processing, but also optimize the learning effect of neural networks, improve the robustness of the system, and facilitate the realization of real-time heart rate monitoring. The CNN-LSTM architecture combines the feature extraction capability of CNN and the time series modeling capability of LSTM, which can capture complex time series dependencies and improve the accuracy of heart rate prediction. The present invention focuses on an end-to-end prediction model for heart rate monitoring using a wristband device. The model integrates data from a multi-channel PPG sensor and a three-channel accelerometer, and uses a deep learning network to predict heart rate in real time. By introducing frequency domain analysis and bandpass filtering technology, key information within the heart rate range is effectively retained, and a specific loss function for heart rate data is innovatively designed. The weighted coefficient of the loss function is dynamically adjusted in combination with a heuristic method, which can significantly improve the accuracy of heart rate prediction. The wristband device of the present invention is not only portable and comfortable, but also avoids the risk of leakage of sensitive information such as personal health data, and can also adapt to heart rate monitoring in complex sports scenes. Therefore, the present invention effectively overcomes the various shortcomings of the prior art and has a high industrial utilization value.

[0081] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A training method for a heart rate prediction model, characterized in that: The training method comprises: Acquire a training sample set and a corresponding label set; wherein each training sample in the training sample set includes a photoplethysmogram sequence and a corresponding acceleration sequence, and the label set is a real heart rate; Preprocess each training sample to obtain the corresponding photoplethysmographic spectrum sequence and acceleration spectrum sequence; Input all photoplethysmographic spectrum sequences and acceleration spectrum sequences into the heart rate prediction model to be trained to obtain the corresponding predicted heart rate; The difference between the predicted heart rate and the corresponding true heart rate is calculated, and the parameters of the heart rate prediction model are updated based on the difference to obtain a trained heart rate prediction model.

2. The training method of the heart rate prediction model according to claim 1, characterized in that: The training samples are obtained by preprocessing the initial photoplethysmogram sequence and the initial acceleration sequence. The process of obtaining the training samples includes: Acquire the photoplethysmogram within a preset sampling time and the acceleration within the preset time to form an initial photoplethysmogram sequence and an initial acceleration sequence; Based on a preset bandwidth, bandpass filtering is performed on the initial photoplethysmogram sequence and the initial acceleration sequence to obtain a filtered photoplethysmogram sequence and a filtered acceleration sequence; Based on a sliding window of a preset size, the filtered photoplethysmogram sequence and the filtered acceleration sequence are respectively subjected to sliding sampling to obtain a plurality of photoplethysmogram sequences and a plurality of acceleration sequences, and each photoplethysmogram sequence and the corresponding acceleration sequence are used as a training sample.

3. The training method of the heart rate prediction model according to claim 1, characterized in that: The photoplethysmogram is a multi-channel signal, and the preprocessing of each training sample to obtain the corresponding photoplethysmogram spectrum sequence and acceleration spectrum sequence includes: The photoplethysmogram sequences of each channel are integrated to obtain a comprehensive photoplethysmogram sequence; The comprehensive photoplethysmogram sequence and the acceleration sequence are respectively subjected to fast Fourier transformation to obtain a photoplethysmogram spectrum sequence and an acceleration spectrum sequence accordingly.

4. The training method of the heart rate prediction model according to claim 3, characterized in that: The synthesizing the photoplethysmogram sequences of each channel to obtain a comprehensive photoplethysmogram sequence includes: averaging the photoplethysmogram sequences of each channel to obtain a comprehensive photoplethysmogram sequence.

5. The training method of the heart rate prediction model according to claim 3, characterized in that: The method of performing fast Fourier transform on the comprehensive photoplethysmogram sequence and the acceleration sequence to obtain a photoplethysmogram spectrum sequence and an acceleration spectrum sequence respectively comprises: Performing fast Fourier transform on the comprehensive photoplethysmogram sequence and the acceleration sequence respectively, and obtaining a first spectrum sequence of photoplethysmogram and a first spectrum sequence of acceleration respectively; The first spectrum sequence of the photoplethysmogram and the first spectrum sequence of acceleration are intercepted according to a preset spectrum interval to generate a second spectrum sequence of the photoplethysmogram and a second spectrum sequence of acceleration; The second spectrum sequence of the photoplethysmogram and the second spectrum sequence of acceleration are standardized to obtain a corresponding photoplethysmogram spectrum sequence and an acceleration spectrum sequence.

6. The training method of the heart rate prediction model according to claim 3, characterized in that: When the photoplethysmogram is a multi-channel signal, the acquiring of the training sample set further includes: setting at least part of the photoplethysmogram of at least one channel to zero to form a new training sample.

7. The training method of the heart rate prediction model according to claim 1, characterized in that: The difference is obtained by weighting the smoothed L1 loss of the predicted heart rate and the corresponding real heart rate with the weight of the corresponding real heart rate. The weight generation process of the real heart rate is: According to the number of real heart rates in the label set, all real heart rates are divided into a preset number of heart rate groups based on an equal frequency binning method; For each heart rate group: performing Gaussian convolution on the number of real heart rates included in the heart rate group to obtain the number of smoothed real heart rates; The weight of each real heart rate in the corresponding heart rate group is determined according to the number of smoothed real heart rates.

8. A heart rate prediction method, characterized in that: The prediction method comprises: Obtaining a photoplethysmogram sequence and a corresponding acceleration sequence to be analyzed; Preprocessing the photoplethysmogram sequence and the corresponding acceleration sequence respectively to obtain a corresponding photoplethysmogram spectrum sequence and an acceleration spectrum sequence; The photoplethysmogram spectrum sequence and the acceleration spectrum sequence are input into a heart rate prediction model to obtain a predicted heart rate.

9. The heart rate prediction method according to claim 8, characterized in that: The step of inputting the photoplethysmogram spectrum sequence and the acceleration spectrum sequence into a heart rate prediction model to obtain a predicted heart rate comprises: Inputting the photoplethysmogram spectrum sequence and acceleration spectrum sequence into a heart rate prediction model to obtain an initial predicted heart rate; Based on the pre-stored historical heart rate, the initial predicted heart rate is exponentially weighted shifted to obtain a final predicted heart rate.

10. A heart rate prediction model training system, characterized in that: The system comprises: A data acquisition module, used to acquire a training sample set and a corresponding label set; wherein each training sample in the training sample set includes a photoplethysmogram sequence and a corresponding acceleration sequence, and the label set is a real heart rate; A preprocessing module is used to preprocess each training sample to obtain a corresponding photoplethysmogram spectrum sequence and acceleration spectrum sequence; A heart rate prediction module is used to input all photoplethysmographic spectrum sequences and acceleration spectrum sequences into the heart rate prediction model to be trained to obtain the corresponding predicted heart rate; The parameter updating module is used to calculate the difference between the predicted heart rate and the corresponding real heart rate, and update the parameters of the heart rate prediction model based on the difference to obtain a trained heart rate prediction model.

11. A wearable device, characterized in that: The wearable device comprises: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enables the wearable device to implement the heart rate prediction method as described in any one of claims 7-8.

12. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is caused to execute the training method of the heart rate prediction model described in any one of claims 1 to 6 or the heart rate prediction method described in any one of claims 7-8.

Images