A multi-sensor fusion heart rate monitoring method

Abstract

The application discloses a kind of multi-sensor fusion heart rate monitoring methods, it is related to medical health monitoring field, the method comprises: step 1, acquisition original mixed body motion signal, separate out normal BCG signal and abnormal BCG signal from it;Step 2, training heart rate sign extraction model, extracts heart rate sign;Step 3, heart rate feature is used to train classification network model, and the heart rate judgment result is output;Step 4, new heart rate data is input into heart rate sign extraction model, feature representation is extracted, and feature representation is input into the classification network model trained again, obtains the judgment result of new heart rate data.The technical scheme provided in the application improves the accuracy and efficiency of detection and analysis by using artificial intelligence technology and deep learning algorithm, applies the self-supervised method of contrast learning, and learns to distinguish the feature representation of the positive abnormal BCG signal by comparing them, reduces the dependence on manually labeled data, reduces the labor cost and error.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical health monitoring, and in particular to a multi-sensor fusion heart rate monitoring method. BACKGROUND

[0002] By utilizing artificial intelligence technology and deep learning algorithms, the heart rate of infants, the elderly and patients can be monitored, potential heart problems or other health abnormalities can be checked, and suggestions for heart and physical health conditions can be given to provide personalized health suggestions and intervention measures for users. For example, the user's abnormal heart rate can be detected in time, and the user can be reminded to seek medical treatment in time.

[0003] Therefore, the person skilled in the art is committed to developing a multi-sensor fusion heart rate monitoring method which models on the basis of the collected body motion signals to obtain a heart rate judgment result. SUMMARY

[0004] In view of the above defects of the prior art, the technical problem to be solved by the present application is how to model and analyze on the basis of the collected original body motion signals to obtain a heart rate judgment result.

[0005] To achieve the above-mentioned purpose, the present application provides a multi-sensor fusion heart rate monitoring method, comprising the following steps:

[0006] Step 1, collecting original mixed body motion signals, and separating normal BCG signals and abnormal BCG signals therefrom;

[0007] Step 2, training a heart rate sign extraction model with the signals collected in step 1 to extract heart rate signs;

[0008] Step 3, using the heart rate features to train a classification network model to output a heart rate judgment result;

[0009] Step 4, inputting new heart rate data into the heart rate sign extraction model to extract feature representations, and inputting the feature representations into the trained classification network model to obtain a judgment result of the new heart rate data.

[0010] Further, step 2 comprises the following sub-steps:

[0011] Step 2.1, randomly selecting N signals from the original mixed body motion signals of step 1, and separating the normal BCG signals therefrom as positive sample data sets and the abnormal BCG signals as negative sample data sets;

[0012] Step 2.2, sending the original mixed body motion signals, the positive sample data sets and the negative sample data sets into corresponding encoders to capture time-frequency features of the BCG signals, and representing the features as , +、 The encoders share parameters.

[0013] Step 2.3: Represent the features using a Project network. , +、 The heart rate signs are obtained by mapping to another space, normalizing, and calculating the contrast loss function.

[0014] Furthermore, in step 2.2, the feature representation + is obtained by using the time-frequency features obtained from the original hybrid dynamic signal and the positive sample dataset as inputs to the Transformer, and fusing the features through a multi-head attention mechanism.

[0015] Furthermore, the encoder in step 2.2 employs a convolutional neural network, a recurrent neural network, LSTM, ResNet, or UNet.

[0016] Furthermore, the Project network in step 2.3 is a CNN.

[0017] Furthermore, the normalization in step 2.3 specifically involves normalizing the mapping to a hypersphere.

[0018] Furthermore, the contrastive loss function in step 2.3 is the InfoNCE Loss, and its specific calculation formula is as follows:

[0019]

[0020] in, The contrastive loss function is represented by the numerator, which represents the similarity of positive samples. After adjusting for temperature parameters, the denominator of the exponent value represents the similarity between the current sample and all samples. Sum of the exponential values ​​after temperature parameter adjustment It is the total number of samples, including positive samples and outlier samples; It is a temperature parameter used to adjust the distribution of similarity.

[0021] Furthermore, the similarity of the positive samples The calculation function is cosine similarity, specifically:

[0022] .

[0023] Furthermore, the similarity between the current sample and all samples The calculation function of the cosine similarity is as follows:

[0024] .

[0025] Further, the classification network model in step 3 is obtained by removing the Project network in the heart rate feature extraction model, fixing the Encoder network, and training using a cross-entropy loss function.

[0026] The multi-sensor fusion heart rate monitoring method provided by the application has at least the following technical effects:

[0027] The technical solution provided by the application improves the accuracy and efficiency of detection and analysis by using artificial intelligence technology and deep learning algorithms, applying a self-supervised method of contrast learning, learning to distinguish the feature representation of normal and abnormal BCG signals, reducing the dependence on manually labeled data, and reducing the labor cost and errors.

[0028] The concept, specific structure and technical effects of the application will be further described below with reference to the accompanying drawings, so as to fully understand the purpose, features and effects of the application. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 is a flowchart of a heart rate monitoring method according to a preferred embodiment of the application. DETAILED DESCRIPTION

[0030] The following describes the preferred embodiments of the application with reference to the accompanying drawings, so as to make the technical content of the application clearer and easier to understand. The application can be embodied in many different forms, and the protection scope of the application is not limited to the embodiments mentioned herein.

[0031] Example 1

[0032] As shown in Figure 1 , a flowchart of a multi-sensor fusion heart rate monitoring method according to an embodiment of the application. The method adopts a two-stage training method, which specifically includes the following steps:

[0033] Step 1, collect the original mixed body motion signal, and separate the normal BCG signal and the abnormal BCG signal therefrom;

[0034] Step 2, train the heart rate feature extraction model using the signals collected in step 1 to extract the heart rate feature;

[0035] Step 3, use the heart rate feature to train the classification network model to output the heart rate judgment result;

[0036] Step 4, input the new heart rate data into the heart rate feature extraction model, extract the feature representation, and input the feature representation into the trained classification network model to obtain the judgment result of the new heart rate data.

[0037] In particular, ballistocardiogram (BCG) is a non-invasive, non-contact and convenient cardiovascular function monitoring method. In step 1, the BCG component usually appears as a vibration with a slightly higher frequency than the respiratory component, with a frequency range of 0.5 to 10 Hz. In medicine, it is generally considered that the heart rate is normal at 40 to 150 per minute, i.e. the frequency is 0.67 to 2.5 Hz. The original mixed body motion signal is collected in the following manner: first, a band-pass filter with a band-pass filter of 0.5 Hz-3 Hz is designed to preliminarily extract the BCG signal. Then, the first derivative or the second derivative of the preliminarily extracted BCG signal is calculated to help identify the slope change points, which are usually the candidates for peak values. A threshold is set to identify possible peak values. Check above the set threshold, connect the rising and falling parts, mark the local maximum as the peak value, compare the current sample point with the previous and subsequent sample points to determine whether it is a peak value, set the minimum distance between peak values to avoid detecting repeated peak values, and remove false peak values. Take the largest point in the peak value as the J point in the BCG signal.

[0038] The beating of the heart is caused by the depolarization and repolarization of myocardial cell tissue. At the normal pacemaker of the heart, the sinoatrial node, the generated impulse is propagated to the atrium via the atrial conduction system, stimulates the atrial muscle cells, and then conducts the impulse to the atrioventricular node. Then, the atrioventricular node stimulates the entire ventricular muscle through the left and right bundle branches, completing a cardiac cycle. This process can be observed and recorded by electrocardiogram in electrophysiology. A standard BCG heartbeat mainly includes H, I, J, K, L, M, and N waves, wherein the H-L interval signal forms a wave group like the letter "w", representing the process of cardiac contraction, and L, M, and N represent the process of cardiac diastole. The time interval between adjacent J peaks is called "J-J interval", and the reciprocal of the J-J interval is calculated, i.e. the instantaneous heart rate based on the BCG signal is obtained. By positioning the J peak in the BCG signal peak group, not only the heart rate information can be calculated, but also the basis for other sleep index calculation and subsequent analysis work can be laid.

[0039] Embodiment 2

[0040] On the basis of embodiment 1, step 2 includes the following sub-steps:

[0041] Step 2.1, randomly select N signals from the original mixed PPG signal in step 1 to form a Batch, and for any signal in the Batch, the normal BCG signal separated from it is taken as the positive sample dataset and the abnormal BCG signal is taken as the negative sample dataset;

[0042] Step 2.2, the original mixed PPG signal, the positive sample dataset and the negative sample dataset are all sent into the corresponding encoder to capture the time-frequency features of the BCG signal, and the feature representations are denoted as 、 +、 wherein the parameters among the encoders are shared;

[0043] Step 2.3, through a Project network, the feature representations 、 +、 are mapped into another space (usually with smaller dimensions), and then normalized (standardized mapping to a hypersphere) and the contrastive loss function is calculated to obtain the heart rate sign.

[0044] In particular, in step 2.2, the feature representation + is to take the time-frequency features obtained from the original mixed PPG signal and the positive sample dataset as the input of the Transformer, and obtain it through the multi-head attention mechanism. The Transformer multi-head attention can help the model learn the correlation and interaction between different features, so as to help the model pay attention to the key features in the signal and extract the feature representation more conducive to the classification task.

[0045] In particular, the encoder in step 2.2 adopts a convolutional neural network, a recurrent neural network, an LSTM, a Resnet or a Unet.

[0046] Embodiment 3

[0047] On the basis of embodiment 2, the Project network in step 2.3 is a CNN.

[0048] In particular, the contrastive loss function in step 2.3 is InfoNCE Loss, and the specific calculation formula is:

[0049]

[0050] wherein, represents the contrastive loss function, the numerator represents the similarity of the positive sample the exponential value after temperature parameter adjustment, the purpose is to make the positive sample pair have a higher contribution in the loss function; the denominator part is the similarity of the current sample to all samples The sum of the exponential values after temperature parameter adjustment is equivalent to considering the comparison between the current sample and all samples. N is the total number of samples, including normal samples and abnormal samples. is a temperature parameter used to adjust the distribution of similarity, usually a small positive number. The similarity calculation function takes the dot product after L2 regularization of the representation vector or the cosine similarity between the representation vectors.

[0051] In particular, the similarity of the normal sample The calculation function of the cosine similarity is as follows:

[0052] .

[0053] In particular, the similarity of the current sample and all samples The calculation function of the cosine similarity is as follows:

[0054] .

[0055] Embodiment 4

[0056] On the basis of Embodiment 2 or 3, the classification network model in step 3 removes the Project network in the heart rate feature extraction model, fixes the Encoder network, and uses the cross-entropy loss function for training, for subsequent heart rate classification tasks.

[0057] The use process of the multi-sensor fusion heart rate monitoring method provided by the embodiments of the application is as follows:

[0058] 1. Input the new heart rate data sample to be detected into the trained model to extract the feature representation.

[0059] 2. Input the feature representation of the new sample into the trained classification network model for classification to obtain the normal or abnormal heart rate judgment result.

[0060] This abnormal heart rate detection method based on contrast learning can effectively utilize the features of normal heart rate signals and enhance the learning ability of the model for abnormal patterns. Through parameter sharing, the generalization of the model can be improved and the risk of overfitting can be reduced.

[0061] While heart rate is being detected, traditional heart rate variability (HRV) analysis is performed to further analyze heart rate health, including: calculating the time intervals between consecutive heart rate events (J-J intervals) and performing HRV analysis to assess the activity of the autonomic nervous system. Heart rate variability refers to the changes in the rhythm of the heart rate over time and is a method of analyzing the differences between successive sinus heart rates. HRV is a reflection of the activity of the autonomic nervous system and a quantitative assessment of cardiac sympathetic and vagal tone and its balance, thereby determining its condition and prevention of cardiovascular disease. The greater the value of HRV, the better the recovery and rest, and the healthier it is. Studies have shown that heart rate variability can also be applied to sleep staging, coronary heart disease, hypertension, and heart failure detection, especially for continuous monitoring of the prognosis of cardiovascular disease, by understanding the changes in the heart to prevent the occurrence of further deterioration.

[0062] HRV parameters are obtained through time domain analysis and nonlinear analysis.

[0063] Among them, the time domain analysis is the simple statistical measure of calculating the J-J interval, including:

[0064] Mean HR: the average value of all J-J intervals.

[0065] SDNN: the overall standard deviation of J-J intervals, reflecting the overall HRV.

[0066] Mean NN50: the average number of consecutive J-J interval differences greater than 50 milliseconds.

[0067] SDNN Index: the standard deviation of the number of NN50, reflecting short-term HRV.

[0068] TINN: the triangular distribution index of the J-J interval difference.

[0069] Frequency domain analysis: through fast Fourier transform (FFT) or other frequency domain analysis methods, calculate the power spectral density (PSD). Determine the following frequency domain indicators:

[0070] Total Power: the power of the entire frequency spectrum.

[0071] LF Power: the power in the range of 0.04-0.15 Hz, reflecting the combined effects of sympathetic and parasympathetic nerves.

[0072] HF Power: the power in the range of 0.15-0.4 Hz, mainly reflecting parasympathetic nerve activity.

[0073] LF / HF ratio: assesses the balance between sympathetic and parasympathetic nervous activity.

[0074] Nonlinear analysis includes:

[0075] Calculate nonlinear statistical measures, such as: Poincare Plot of heart rate.

[0076] Approximate entropy (ApEn) or sample entropy (SampEn) of heart rate, to assess the complexity and predictability of heart rate.

[0077] According to the calculated HRV parameters, the activity and balance state of the autonomic nervous system are evaluated, and the graphs of HRV parameters, such as time series graph, histogram, power spectrum graph, etc. are drawn to visually display the HRV characteristics, and a user feedback mechanism is established, including: designing a user feedback mechanism. When an abnormal heart rate is detected, the user is prompted through visual or auditory cues, and is advised to undergo further medical examination.

[0078] The above describes the preferred embodiments of the present application in detail. It should be understood that those skilled in the art can make many modifications and changes without creative labor based on the concept of the present application. Therefore, any technical solution obtained by logical analysis, reasoning or limited experiment based on the existing technology according to the concept of the present application shall be within the protection scope determined by the claims.

Claims

1. A multi-sensor fusion method for heart rate monitoring, characterized in that, The method includes the following steps: Step 1: Collect the raw mixed body motion signal and separate the normal BCG signal and abnormal BCG signal from it; Step 2: Use the signals collected in Step 1 to train the heart rate and vital signs extraction model and extract heart rate and vital signs; Step 3: Use the heart rate signs to train the classification network model and output the heart rate judgment result; Step 4: Input the new heart rate data into the heart rate sign extraction model to extract feature representations, and then input the feature representations into the trained classification network model to obtain the judgment result of the new heart rate data; Step 2 includes the following sub-steps: Step 2.1: Randomly select N signals from the original mixed motion signals in Step 1, and use the separated normal BCG signals as the positive sample dataset and the separated abnormal BCG signals as the negative sample dataset. Step 2.2: Input the original hybrid motion signal, the positive sample dataset, and the negative sample dataset into the corresponding encoders to capture the time-frequency features of the BCG signal, and denote the feature representations as follows: , +、 The encoders share parameters. Step 2.3: Represent the features using a Project network. , +、 The heart rate signs are obtained by mapping to another space, normalizing, and calculating the contrast loss function. The normalization in step 2.3 specifically involves mapping to a hypersphere through standardization. The contrast loss function mentioned in step 2.3 is InfoNCE Loss, and the specific calculation formula is as follows: in, The contrastive loss function is represented by the numerator, which represents the similarity of positive samples. After adjusting for temperature parameters, the denominator of the exponent value represents the similarity between the current sample and all samples. Sum of the exponential values ​​after temperature parameter adjustment It is the total number of samples, including positive samples and outlier samples; It is a temperature parameter used to adjust the distribution of similarity; The similarity of the positive samples The calculation function is cosine similarity, specifically: 。 2. The multi-sensor fusion heart rate monitoring method as described in claim 1, characterized in that, In step 2.2, feature representation + is obtained by using the time-frequency features obtained from the original hybrid dynamic signal and the positive sample dataset as inputs to the Transformer, and fusing the features through a multi-head attention mechanism.

3. The multi-sensor fusion heart rate monitoring method as described in claim 1, characterized in that, The encoder in step 2.2 uses a convolutional neural network, a recurrent neural network, LSTM, ResNet, or UNet.

4. The multi-sensor fusion heart rate monitoring method as described in claim 1, characterized in that, The Project network in step 2.3 is a CNN.

5. The multi-sensor fusion heart rate monitoring method as described in claim 4, characterized in that, The similarity between the current sample and all samples The calculation function is cosine similarity, specifically: 。 6. The multi-sensor fusion heart rate monitoring method as described in claim 1, characterized in that, The classification network model in step 3 is obtained by first removing the Project network from the heart rate and vital signs extraction model, then fixing the Encoder network, and training it using the cross-entropy loss function.

Images