Deep learning-based heart rate detection methods and wearable devices

Abstract

This application provides a deep learning-based heart rate detection method and wearable device, relating to the field of communication technology. This scheme uses PPG and ACC sensor signals as input, and heart rate and motion scene information as model output to train a deep attention network. The trained model is then used for heart rate detection. Since motion artifact noise distribution varies across different scenarios, the deep attention network learning mechanism can fit the nonlinear relationship between scene, noise, and heart rate. This allows for adaptive denoising and feature extraction in different scenarios, achieving signal denoising, signal fusion, and complex scene recognition in complex multi-scene environments. This eliminates the interference of motion artifact noise on the PPG signal, improving the accuracy of heart rate detection. Therefore, this scheme solves the problem of inaccurate heart rate detection results caused by motion artifact interference with the PPG signal.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a heart rate detection method and wearable device based on deep learning. Background Technology

[0002] Heart rate is an important indicator of health. Traditional measurement methods are based on electrocardiogram (ECG) heart rate analysis, which requires specialized equipment and knowledge and cannot meet users' daily measurement needs. In recent years, with the development and maturity of wearable devices (such as smart wristband devices), wearable device-based heart rate detection solutions have become mainstream.

[0003] Wearable devices can use photoplethysmography (PPG) signals to calculate heart rate. A PPG sensor includes a light-emitting diode (LED) and a receiver. The LED continuously projects light onto the skin, which is absorbed by the blood flow through the skin tissue. Simultaneously, the receiver receives the reflected light signal. Because the intensity of the reflected light is related to blood flow velocity, and blood flow velocity is affected by the periodic heart rhythm, the PPG signal can reflect heart rate information.

[0004] However, because PPG signals are susceptible to motion artifacts, heart rate detection results based on PPG signals are not accurate enough during exercise. Summary of the Invention

[0005] This application provides a heart rate detection method and wearable device based on deep learning, which solves the problem that the heart rate value detected by the traditional heart rate algorithm based on PPG signal is not accurate enough in sports scenarios.

[0006] To achieve the above objectives, this application adopts the following technical solution:

[0007] Firstly, this application provides a heart rate detection method based on deep learning, the method comprising:

[0008] When the user wears the wearable device, in response to the heart rate detection command, it acquires the photoplethysmography (PPG) signal and the acceleration (ACC) signal.

[0009] Based on the effective ACC spectrum data corresponding to the ACC signal, the effective PPG spectrum data corresponding to the PPG signal, and the target prediction model, heart rate and scene information are obtained.

[0010] The heart rate and scene information are displayed on the screen of the wearable device;

[0011] The target prediction model is a model trained using ACC sample data and PPG sample data as inputs, and heart rate labels and sports scene labels as target variables, through a deep attention mechanism Attention network. The target prediction model has scene recognition and heart rate prediction functions.

[0012] This solution uses PPG and ACC sensor signals as input and heart rate and motion scene information as model output to train a deep attention network. The trained model is then used for heart rate detection. Since motion artifact noise distribution varies across different scenarios, the deep attention network's learning mechanism can fit the non-linear relationship between scene, noise, and heart rate. This allows for adaptive denoising and feature extraction in different scenarios, achieving signal denoising, signal fusion, and complex scene recognition in complex multi-scene environments. This eliminates the interference of motion artifact noise on the PPG signal, improving the accuracy of heart rate detection. Therefore, this solution addresses the problem of inaccurate heart rate detection results caused by motion artifact interference with the PPG signal.

[0013] Among some possible implementations of the first aspect, the method further includes:

[0014] Upon obtaining heart rate and scene information, the heart rate and scene information are recorded in a tag cache;

[0015] The spectral peak data corresponding to the heart rate and scene information are recorded in the data cache;

[0016] The spectral peak data corresponding to the heart rate and scene information includes the PPG spectral peak position and amplitude, as well as the ACC spectral peak position and amplitude.

[0017] This solution allows for initial model execution during heart rate detection, storing the heart rate and scene information obtained from the model, along with the peak positions and amplitude values ​​of the corresponding PPG and ACC frequency domain data. Later, during heart rate detection, this stored data can be directly retrieved when certain conditions are met, saving computational resources, reducing latency, and enabling faster heart rate value acquisition.

[0018] In some possible implementations of the first aspect, obtaining heart rate and scene information based on the effective ACC spectrum data corresponding to the ACC signal, the effective PPG spectrum data corresponding to the PPG signal, and the target prediction model includes:

[0019] Based on the effective spectrum data of PPG and the effective spectrum data of ACC, obtain the spectrum data of the first N peak points;

[0020] If the data buffer does not have any of the peak spectrum data of the first N peaks in advance, then the heart rate and scene information are obtained based on the effective ACC spectrum data corresponding to the ACC signal, the effective PPG spectrum data corresponding to the PPG signal, and the target prediction model.

[0021] If any of the peak spectrum data from the first N peak spectrum data has been pre-stored in the data cache, then the heart rate and scene information corresponding to any of the peak spectrum data are read from the tag cache.

[0022] This method allows for the determination of the positions and amplitudes of the three highest peaks in both the PPG and ACC frequency domains. It then checks if the current data has appeared in the data buffer; if not, it is entered into the model. If it has appeared, the corresponding heart rate and scene information stored in the tag buffer are read. This saves computational resources, reduces latency, and allows for faster acquisition of heart rate values.

[0023] In some possible implementations of the first aspect, obtaining heart rate and scene information based on the effective ACC spectrum data corresponding to the ACC signal, the effective PPG spectrum data corresponding to the PPG signal, and the target prediction model includes:

[0024] The effective ACC spectrum data and the effective PPG spectrum data are input into the first prediction model to obtain the heart rate and scene information;

[0025] The target prediction model is the first prediction model, which is a model trained by a deep attention network with ACC sample data and PPG sample data as inputs and heart rate label and sports scene label as target variables. The first prediction model has scene recognition and heart rate prediction functions.

[0026] The above-described solution provides a heart rate detection method: taking PPG and ACC frequency domain data as input, and simultaneously outputting motion scene information and heart rate information. Thus, in practical implementation, the wearable device can simultaneously output current scene information and heart rate value when detecting heart rate. This application's solution takes into account user behavior or exercise state, and can support the detection of heart rate values ​​when the user is in different behavioral states (e.g., resting state, and various exercise states). This application's solution can solve the problem of various noise interferences to PPG signals in different scenarios, achieving effective heart rate detection in various noise environments.

[0027] In some possible implementations of the first aspect, obtaining heart rate and scene information based on the effective ACC spectrum data corresponding to the ACC signal, the effective PPG spectrum data corresponding to the PPG signal, and the target prediction model includes:

[0028] The effective ACC spectrum data is input into the second prediction model to obtain scene information; wherein, the second prediction model is a model trained by a deep neural network with ACC sample data as input and motion scene labels as target variables, and the second prediction model has scene recognition function;

[0029] The effective spectrum data of PPG and the obtained scene information are input into the third prediction model to obtain heart rate information. The third prediction model is a model trained by a deep Attention network with PPG sample data, ACC sample data and motion scene labels as inputs and heart rate labels as target variables. The third prediction model has heart rate detection function.

[0030] The target prediction model includes the second prediction model and the third prediction model.

[0031] The above-mentioned solution provides another heart rate detection method: it separates the motion scene information and heart rate information recognition functions. First, a deep neural network model is used to recognize the scene information, and then the recognition result and sensor frequency domain data are used as inputs for deep attention to output the heart rate value. This application's solution can solve the problem of various noise interferences on PPG signals in different scenarios, and achieve effective heart rate detection in different noise scenarios.

[0032] In some possible implementations of the first aspect, after acquiring the PPG signal and the ACC signal, the method further includes:

[0033] The ACC signal and the PPG signal are respectively subjected to a first preprocessing to obtain ACC effective spectrum data and PPG effective spectrum data;

[0034] The first preprocessing includes Fast Fourier Transform (FFT) and filtering.

[0035] The above method allows for FFT transformation and filtering of the acquired signal to obtain the effective spectrum. The heart rate value can then be detected using the effective spectrum. Since the effective spectrum removes interference signals, this improves the accuracy of heart rate detection.

[0036] In some possible implementations of the first aspect, acquiring the PPG signal and the ACC signal includes:

[0037] The PPG signal is acquired using a PPG sensor, and the ACC signal is acquired using an accelerometer.

[0038] The PPG signal can be used to detect heart rate, and the ACC signal can be used to identify the motion scene. This application's solution combines the use of PPG and ACC signals for heart rate detection, which can solve the problem of various noise interferences to the PPG signal in different scenarios, achieving effective heart rate detection under different noise conditions.

[0039] Among some possible implementations of the first aspect, the method further includes:

[0040] The heart rate and scene information are sent to a terminal device, which is an electronic device that is wirelessly connected to the wearable device.

[0041] The heart rate and scene information are displayed on the screen of the terminal device.

[0042] This solution allows users to view heart rate and scene information not only through wearable devices, but also through terminal devices connected to the wearable devices, making it easier for users to view their heart rate values ​​and improving the user experience.

[0043] Secondly, this application provides a method for training a model for detecting heart rate, comprising:

[0044] Obtain a multi-scene sample set, which is a collection of data samples obtained based on the detection of multiple motion scenes;

[0045] ACC sample data, PPG sample data, and heart rate labels were extracted from the multi-scenario sample set.

[0046] Using the ACC sample data and the PPG sample data as inputs, and the heart rate label and the motion scene label as target variables, the system is trained using a deep attention mechanism Attention network.

[0047] A first prediction model is obtained, which has scene recognition and heart rate prediction functions.

[0048] The model training method provided in this application uses PPG and ACC frequency domain data as input and trains the model with motion scenarios and heart rate information as targets. The resulting model has scene recognition and heart rate prediction functions. Since the distribution of motion artifact noise varies in different scenarios, a deep attention network learning mechanism can be used to fit the nonlinear relationship between scene, noise, and heart rate. This allows for adaptive denoising and feature extraction in different scenarios, achieving signal denoising, signal fusion, and complex scene recognition in complex scenarios. This eliminates the interference of motion artifact noise on the PPG signal, thereby improving the accuracy of heart rate detection. Therefore, this solution can solve the problem of inaccurate heart rate detection results caused by motion artifact interference in the PPG signal.

[0049] In some possible implementations of the second aspect, the step of using the ACC sample data and the PPG sample data as input, and the heart rate label and the motion scene label as target variables, and training through a deep Attention network includes:

[0050] The model parameters are adjusted by cross-validation, enabling the model to learn and predict heart rate information under different exercise scenarios.

[0051] The proposed solution involves continuously cross-validating and adjusting model parameters during the training of a deep Attention network model until the training objective is achieved. This ensures that the trained model possesses excellent heart rate prediction capabilities, thereby improving the accuracy of heart rate detection.

[0052] In some possible implementations of the second aspect, obtaining the multi-scenario sample set includes:

[0053] Connect wearable devices and heart rate monitors to the data acquisition module;

[0054] When a user wearing the wearable device and the heart rate belt device performs the first exercise, the data acquisition module acquires the first heart rate detection data of the wearable device and the second heart rate detection data of the heart rate belt device;

[0055] When the user performs the second exercise, the data acquisition module acquires the third heart rate detection data of the wearable device and the fourth heart rate detection data of the heart rate belt device;

[0056] Wherein, the first motion and the second motion are the motions indicated by the motion scene label, and the multi-scene sample set includes the first heart rate detection data, the second heart rate detection data, the third heart rate detection data, and the fourth heart rate detection data.

[0057] For example, various exercise scenarios can be defined to collect data. These include walking, running, swimming, hiking, rope skipping, playing soccer, and playing basketball. It should be noted that these exercise scenarios can also include resting scenarios.

[0058] In this process, when the subject performs exercise in a specific scenario, the heart rate belt device collects heart rate label data in that specific scenario, and this heart rate label data can be used as the target value for model training; the wearable device (such as a wristband device) collects PPG signal and ACC signal in that specific scenario as model parameters to be trained.

[0059] In some possible implementations of the second aspect, the extraction of ACC sample data and PPG sample data from the multi-scenario sample set includes:

[0060] Extract the ACC sample data and the PPG sample data from the detection data of the wearable device;

[0061] The wearable device's detection data includes the first heart rate detection data and the third heart rate detection data.

[0062] In some possible implementations of the second aspect, the extraction of heart rate tags from the multi-scenario sample set includes:

[0063] Extract the heart rate tag from the detection data of the heart rate belt device;

[0064] The heart rate monitoring device's detection data includes the second heart rate detection data and the fourth heart rate detection data.

[0065] According to the proposed solution, when training the model, the heart rate label detected by the heart rate belt device can be used as the target variable, so that the trained model has the function of heart rate prediction.

[0066] In some possible implementations of the second aspect, after extracting ACC sample data, PPG sample data, and heart rate tags from the multi-scenario sample set, the method further includes:

[0067] The ACC sample data and the PPG sample data are subjected to Fast Fourier Transform (FFT) and filtering to obtain filtered ACC sample data and PPG sample data.

[0068] The process involves using the ACC sample data and the PPG sample data as input, and the heart rate label and motion scene label as target variables, and training a deep Attention network, including:

[0069] Using filtered ACC sample data and PPG sample data as input, and the heart rate label and motion scene label as target variables, the deep Attention network is trained.

[0070] In some possible implementations of the second aspect, the filtering process is used to filter out noise data outside the [0.7Hz, 4Hz] range in the spectrum.

[0071] In this way, using the effective spectral data after filtering out noise data for model training helps to eliminate the interference of motion artifact noise on the PPG signal.

[0072] In some possible implementations of the second aspect, after obtaining the first prediction model, the method further includes:

[0073] The first prediction model is quantized using preset quantization parameters to obtain the quantized first prediction model.

[0074] In this embodiment, the trained model can be quantized using 8-bit or 16-bit parameters to store its network structure information and quantized parameter data. Since model quantization reduces the storage space of wearable devices and accelerates integer-type computation, it can reduce the storage space of wearable device components and improve computation speed.

[0075] Thirdly, this application provides a method for training a model for detecting heart rate, comprising:

[0076] Obtain a multi-scene sample set, which is a collection of data samples obtained based on the detection of multiple motion scenes;

[0077] ACC sample data, PPG sample data, and heart rate labels were extracted from the multi-scenario sample set.

[0078] Using the ACC sample data, the PPG sample data, and the motion scene label as inputs, and the heart rate label as the target variable, the system is trained using a deep attention mechanism Attention network.

[0079] A third prediction model is obtained, which has a heart rate prediction function.

[0080] The model training method provided in this application uses PPG and ACC frequency domain data and scene information as input, and trains the model with heart rate information as the target. The resulting model has heart rate prediction capabilities. Since motion artifact noise distribution varies across different scenarios, a deep attention network learning mechanism can be used to fit the nonlinear relationship between scene, noise, and heart rate. This allows for adaptive denoising and feature extraction in different scenarios, achieving signal denoising, signal fusion, and complex scene recognition in complex scenarios. This eliminates the interference of motion artifact noise on the PPG signal, thereby improving the accuracy of heart rate detection. Therefore, this solution addresses the problem of inaccurate heart rate detection results caused by motion artifact interference with the PPG signal.

[0081] In some possible implementations of the third aspect, the method further includes:

[0082] Using the ACC sample data and the PPG sample data as input, and the motion scene label as the target variable, a second prediction model is obtained by training through a deep neural network. The second prediction model has scene recognition function.

[0083] In some possible implementations of the third aspect, the step of using the ACC sample data, the PPG sample data, and the motion scene label as input, and the heart rate label as the target variable, and training through a deep Attention network includes:

[0084] The parameters of the Attention network model are adjusted by cross-validation, enabling the Attention network model to learn and predict heart rate information under different sports scenarios.

[0085] The proposed solution involves continuously cross-validating and adjusting model parameters during the training of a deep Attention network model until the training objective is achieved. This ensures that the trained model possesses excellent heart rate prediction capabilities, thereby improving the accuracy of heart rate detection.

[0086] In some possible implementations of the third aspect, obtaining the multi-scenario sample set includes:

[0087] Connect wearable devices and heart rate monitors to the data acquisition module;

[0088] When a user wearing the wearable device and the heart rate belt device performs the first exercise, the data acquisition module acquires the first heart rate detection data of the wearable device and the second heart rate detection data of the heart rate belt device;

[0089] When the user performs the second exercise, the data acquisition module acquires the third heart rate detection data of the wearable device and the fourth heart rate detection data of the heart rate belt device;

[0090] Wherein, the first motion and the second motion are the motions indicated by the motion scene label, and the multi-scene sample set includes the first heart rate detection data, the second heart rate detection data, the third heart rate detection data, and the fourth heart rate detection data.

[0091] In some possible implementations of the third aspect, the extraction of ACC sample data and PPG sample data from the multi-scenario sample set includes:

[0092] Extract the ACC sample data and the PPG sample data from the detection data of the wearable device;

[0093] The wearable device's detection data includes the first heart rate detection data and the third heart rate detection data.

[0094] In some possible implementations of the third aspect, the extraction of heart rate tags from the multi-scenario sample set includes:

[0095] Extract the heart rate tag from the detection data of the heart rate belt device;

[0096] The heart rate monitoring device's detection data includes the second heart rate detection data and the fourth heart rate detection data.

[0097] According to the proposed solution, when training the model, the heart rate label detected by the heart rate belt device can be used as the target variable, so that the trained model has the function of heart rate prediction.

[0098] In some possible implementations of the third aspect, after extracting ACC sample data, PPG sample data, and heart rate labels from the multi-scenario sample set, the method further includes:

[0099] The ACC sample data and the PPG sample data are subjected to Fast Fourier Transform (FFT) and filtering to obtain filtered ACC sample data and PPG sample data.

[0100] The process involves using the ACC sample data, the PPG sample data, and the motion scene label as inputs, with the heart rate label as the target variable, and training a deep Attention network, including:

[0101] The deep Attention network is trained using filtered ACC sample data and PPG sample data, as well as motion scene labels, with the heart rate label as the target variable.

[0102] In some possible implementations of the third aspect, the filtering process is used to filter out noise data outside [0.7Hz, 4Hz] in the spectral data corresponding to the ACC sample data and the PPG sample data.

[0103] In this way, using the effective spectral data after filtering out noise data for model training helps to eliminate the interference of motion artifact noise on the PPG signal.

[0104] In some possible implementations of the third aspect, after obtaining the third prediction model, the method further includes:

[0105] The third prediction model is quantized using preset quantization parameters to obtain the quantized third prediction model.

[0106] Because the storage space of wearable devices is reduced and the computation speed of integer types is accelerated after model quantization, the storage space of wearable device components can be reduced and the computation speed can be improved.

[0107] Fourthly, this application provides a heart rate detection device based on deep learning, which includes units for performing the method described in the first aspect above. This device can correspond to performing the method described in the first aspect above; for a detailed description of the units within the device, please refer to the description in the first aspect above, and for the sake of brevity, will not be repeated here.

[0108] The methods described in the first, second, and / or third aspects above can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions. For example, a processing module or unit, a display module or unit, etc.

[0109] Fifthly, this application provides an electronic device including a processor coupled to a memory for storing computer programs or instructions, and the processor for executing the computer programs or instructions stored in the memory, such that the method in the first aspect is executed.

[0110] For example, a processor is used to execute computer programs or instructions stored in memory, causing the electronic device to perform the method in the first aspect.

[0111] In a sixth aspect, this application provides a computer-readable storage medium having a computer program (also referred to as instructions or code) stored thereon for implementing the methods of the first, second and / or third aspects.

[0112] For example, when the computer program is executed by the computer, it enables the computer to perform the methods in the first aspect.

[0113] In a seventh aspect, this application provides a chip including a processor. The processor is configured to read and execute a computer program stored in a memory to perform the methods of the first aspect, the second aspect, and / or the third aspect, and any possible implementation thereof.

[0114] Optionally, the chip further includes a memory, which is connected to the processor via a circuit or wire.

[0115] Eighthly, this application provides a chip system including a processor. The processor is configured to read and execute a computer program stored in a memory to perform the methods of the first, second, and / or third aspects and any possible implementation thereof.

[0116] Optionally, the chip system further includes a memory, which is connected to the processor via a circuit or wire.

[0117] Ninthly, this application provides a computer program product comprising a computer program (also referred to as instructions or code), which, when executed by a computer, causes the computer to implement the methods of the first aspect, the second aspect, and / or the third aspect.

[0118] It is understood that the beneficial effects of aspects four through nine above can be found in the relevant descriptions in aspects one, two and / or three above, and will not be repeated here. Attached Figure Description

[0119] Figure 1 A schematic diagram of the system architecture for the application of the deep learning-based heart rate detection method provided in the embodiments of this application;

[0120] Figure 2 A flowchart illustrating a deep learning-based heart rate detection method provided in this application embodiment;

[0121] Figure 3 This application provides a schematic diagram of the offline training process of a deep learning-based heart rate detection method in an embodiment of the present application.

[0122] Figure 4 A schematic diagram of the filtered waveform in the deep learning-based heart rate detection method provided in this application embodiment;

[0123] Figure 5 This application provides a schematic diagram of the online prediction process of a deep learning-based heart rate detection method, as illustrated in an embodiment of the present application.

[0124] Figure 6 A flowchart illustrating another deep learning-based heart rate detection method provided in this application embodiment;

[0125] Figure 7 A schematic diagram of the offline training process of a model in another deep learning-based heart rate detection method provided in this application embodiment;

[0126] Figure 8 A schematic diagram of the online prediction process of another deep learning-based heart rate detection method provided in this application embodiment;

[0127] Figure 9 A schematic diagram of the interface for the application of the deep learning-based heart rate detection method provided in the embodiments of this application;

[0128] Figure 10 A schematic diagram of the structure of a deep learning-based heart rate detection device provided in an embodiment of this application;

[0129] Figure 11 This is a schematic diagram of the structure of a wearable device provided in an embodiment of this application. Detailed Implementation

[0130] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0131] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0132] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. For example, "first preprocessing" and "second preprocessing," etc., are used to distinguish different preprocessing processes, not to describe a specific order of preprocessing.

[0133] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0134] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0135] PPG is a non-invasive detection technology that uses optical principles to convert human biosignals into electrical signals. Specifically, LED light is shone onto the skin, and a photosensitive sensor receives the light that is absorbed by the skin tissue and then reflected or transmitted back. The electrical signal obtained by the photosensitive sensor is converted into a digital signal, which is the PPG signal.

[0136] Due to its advantages such as being non-invasive, simple, and portable, it is widely used for health monitoring of human physiological parameters such as heart rate, blood oxygen, and stress. Among these, heart rate, as one of the parameters measuring the heart's pumping ability, has significant medical importance for accurate detection.

[0137] Ideally, PPG signals, like ECG signals, can accurately detect heart rate values ​​at various times. However, noise from complex scenarios such as hardware, temperature, and motion distorts the true PPG signal, significantly limiting the accuracy of heart rate calculation. Motion artifacts are the most significant contributor, causing missing or distorted peaks and troughs in the PPG time-domain signal, reducing the accuracy of time-domain counting methods. In the frequency domain, this manifests as abnormal PPG spectral peaks, making it impossible to accurately pinpoint the dominant frequency of the PPG signal at the current moment, thus hindering accurate results. Therefore, adaptively eliminating complex noise in PPG signals to improve heart rate detection accuracy is a crucial issue.

[0138] Figure 1 The diagram illustrates the system architecture involved in various exemplary embodiments of this application. For example... Figure 1 As shown, the system architecture includes a wearable device 1, which can detect human heart rate values ​​using PPG technology. Optionally, the wearable device can display the heart rate values. Optionally, the system architecture may also include a terminal device 2 wirelessly connected to the wearable device 1, which can receive and display the heart rate values ​​sent by the smart wearable device 1 for the user to view.

[0139] Wearable device 1 can be a smartwatch, smart bracelet, or other wearable electronic device that supports heart rate detection. For ease of explanation, the following description of wearable device 1 will use a smart bracelet as an example.

[0140] However, traditional heart rate algorithms are relatively accurate in detecting heart rate at rest, but not accurate enough during exercise. For example... Figure 1 As shown, when the smart bracelet 1 performs heart rate detection, it displays the message: "Please remain still." Due to motion noise interference, the heart rate value detected during exercise may be inaccurate. This is because during exercise, factors such as displacement between the smart bracelet and the skin, shaking of the smart bracelet, and pressure on blood vessels can distort the heart rate detection signal, making the heart rate value detected using traditional heart rate algorithms insufficiently accurate.

[0141] Current traditional technologies involve extensive artificial noise preprocessing and rule formulation for signals, requiring substantial domain knowledge and achieving good results only under specific parameters and scenarios, thus lacking an end-to-end learning model. Current technologies cannot simultaneously find an adaptive solution for noise reduction and signal compensation under various complex motion artifacts and noise conditions.

[0142] In other words, traditional signal processing methods are severely affected by motion artifacts, resulting in low heart rate accuracy.

[0143] Therefore, this embodiment of the application uses PPG and ACC sensor signals as input and heart rate and motion scene information as model output to train a deep Attention network, and then uses the trained model for heart rate detection. Since motion artifact noise distribution varies in different scenarios, the deep Attention network learning mechanism can fit the nonlinear relationship between scene, noise, and heart rate. This allows for adaptive denoising and feature extraction in different scenarios, achieving signal denoising, signal fusion, and complex scene recognition in complex scenarios, thereby eliminating the interference of motion artifact noise on the PPG signal and improving the accuracy of heart rate detection. Therefore, this solution can solve the problem of inaccurate heart rate detection results caused by motion artifact interference with the PPG signal.

[0144] This application's solution takes into account user behavior and exercise status, and can support the detection of heart rate values ​​when the user is in different exercise scenarios (such as various exercise states, including the resting state). Furthermore, it can detect the current scenario and the heart rate value within the current scenario. Through this application's solution, continuous, real-time, and high-precision heart rate monitoring can be achieved even in the presence of motion noise interference.

[0145] First Embodiment

[0146] The following is combined with Figure 2 This application provides a detailed description of the deep learning-based heart rate detection method provided in the first embodiment.

[0147] Figure 2 This is a schematic flowchart of a heart rate detection method provided in an embodiment of this application. (Refer to...) Figure 2 As shown, the method includes the following steps S101-S104.

[0148] S101, when the user wears the wearable device, responds to the heart rate detection command and acquires the PPG signal and ACC signal.

[0149] In this embodiment of the application, when a user wears a wearable device and the wearable device is turned on, the wearable device can be used to detect the user's heart rate at various times. Heart rate refers to the number of heartbeats per minute in a normal person at rest, also called resting heart rate, which is generally 60-100 beats per minute (bpm).

[0150] Wearable devices can be equipped with PPG sensors and accelerometers. The PPG sensor can detect PPG signals, and the accelerometer can detect acceleration (ACC) signals. The ACC signal can reflect the user's motion data, and the PPG signal can reflect the user's heart rate data.

[0151] For example, when a preset heart rate detection trigger condition is met, the wearable device will be triggered to perform heart rate detection. For instance, the wearable device will receive a heart rate detection command and, in response to the heart rate detection command, trigger the PPG sensor to detect or acquire the PPG signal and trigger the accelerometer to detect or acquire the ACC signal, thereby enabling the wearable device to acquire the PPG signal and the ACC signal.

[0152] Optionally, in this embodiment, the preset heart rate detection trigger condition can be any of the following: the wearable device receives an operation from the user to trigger the heart rate detection function; the wearable device detects the heart rate in real time; or the wearable device periodically detects the heart rate and begins the heart rate detection for the Nth cycle. The specific condition can be determined according to actual usage needs, and this embodiment does not impose any limitations.

[0153] S102, perform the first preprocessing on the ACC signal and PPG signal respectively to obtain the effective spectrum data of ACC and the effective spectrum data of PPG.

[0154] The first preprocessing step mentioned above may include a fast Fourier transform (FFT) and filtering. The filtering step may be a bandpass filter.

[0155] In this embodiment, after acquiring the ACC signal, the wearable device can perform an FFT transform on the ACC signal to obtain the corresponding ACC spectrum. Then, the ACC spectrum is bandpass filtered to remove noise data, resulting in the effective ACC spectrum.

[0156] Similarly, after acquiring the PPG signal, the wearable device can perform an FFT transform on the PPG signal to obtain the corresponding PPG spectrum. Then, the PPG spectrum is bandpass filtered to remove noise data, resulting in the effective PPG spectrum.

[0157] S103. Based on the effective frequency band of ACC, the effective frequency band of PPG, and the first prediction model, heart rate and scene information are obtained. The first prediction model is a model trained by a deep attention network with ACC sample data and PPG sample data as input and heart rate label and motion scene label as target variables. The first prediction model has the ability to recognize scenes and detect heart rate.

[0158] Among them, the Attention network mentioned above adopts an attention mechanism, which actively and consciously focuses on a certain object with a predetermined target task, and only selects some key information inputs for processing, so as to improve the efficiency of the neural network.

[0159] In this embodiment, the wearable device can use ACC effective spectrum data and PPG effective spectrum data as input parameters of the first prediction model, and execute an online prediction process using the first prediction model. Through this first prediction model, the wearable device can identify whether the user's current exercise scenario is resting or active, and can accurately identify the specific type of exercise, such as walking, swimming, running, mountain climbing, etc.; and can accurately detect the user's heart rate value in the current exercise scenario.

[0160] For ease of explanation, the motion scene identified by the wearable device is described as scene information, and the heart rate value detected by the wearable device is described as heart rate information. Thus, after the online prediction process using the first prediction model is completed, the first prediction model can simultaneously output both scene information and heart rate information.

[0161] It should be noted that the first prediction model provided in this application embodiment is a scene recognition and heart rate detection model trained using a deep Attention network based on heart rate information and motion scene information. In other words, the solution in this application combines motion scene information and heart rate information for model training, which can reduce motion noise and avoid the impact of motion artifacts on the accuracy of heart rate detection. Therefore, the heart rate value output by the first prediction model is more accurate.

[0162] The offline training process of the first prediction model will be described in detail below, and will not be repeated here.

[0163] S104 displays heart rate and scene information.

[0164] In one possible implementation, the wearable device can display heart rate and scene information.

[0165] In one possible implementation, when the wearable device maintains a wireless connection with the terminal device, and the wearable device acquires heart rate and scene information, it can send the heart rate and scene information to the terminal device for display by the user.

[0166] For example, if a fitness app is installed on a device, the device can display heart rate and scene information on the app's interface.

[0167] Of course, this embodiment may also display only heart rate information without displaying scene information. The specific settings can be configured according to actual usage requirements, and this embodiment does not impose any limitations.

[0168] It should be noted that the embodiments of this application do not limit the display style of heart rate information and scene information, such as font size, font color, etc.

[0169] This application employs a deep attention learning mechanism, capable of recognizing various motion scenarios and adaptively fitting different scenarios and noise levels without requiring extensive data preprocessing or the formulation of manually derived parameters. The adaptive deep attention network enables real-time heart rate monitoring, resolving the issue of various noise interferences affecting PPG signals in different scenarios, thus achieving effective heart rate detection under diverse noise conditions.

[0170] The following is combined with Figure 3 and Figure 4 The offline training process and online prediction process of the first prediction model in the first embodiment are described respectively.

[0171] Figure 3 This describes the offline training process of the deep Attention network for the first prediction model in the first embodiment. For example... Figure 3 As shown, the offline training process of the deep Attention network includes steps S201-S208.

[0172] S201, Obtain a multi-scenario sample set.

[0173] First, various sports scenarios that need to be collected can be defined. For example, walking, running, swimming, mountain climbing, rope skipping, playing football, playing basketball, etc. It should be understood that this is an exemplary list, and the various sports scenarios that need to be collected in this application embodiment are not limited to these. The specific scenarios can be determined according to actual usage needs, and this application embodiment does not limit them.

[0174] It should be noted that the "movement scene" here can include the "resting scene".

[0175] Then, connect both the wristband and heart rate monitor to the Android application package (APK) for data collection via Bluetooth.

[0176] Then, when the subjects wear the wristband and heart rate monitor and engage in exercise in a specific scenario, data is collected for that scenario.

[0177] For example, suppose the sports scene label data is set to running. When the subject runs, data in the running scene can be collected.

[0178] For example, suppose the motion scene label data is set to walking. When the subject walks, data can be collected in the walking scene.

[0179] For example, suppose the sports scene label data is set to jump rope. When the subject jumps rope, data in the jump rope scene can be collected.

[0180] For example, suppose the motion scene label data is set to resting, then data can be collected in a resting scene when the subject is not moving.

[0181] Specifically, when the subject engages in exercise in a specific scenario, the heart rate belt device collects heart rate label data in that specific scenario, which can be used as the target value for model training; the wristband device collects PPG and ACC signals in that specific scenario, which are used as model parameters to be trained.

[0182] Then, after the collection time and specified collection requirements are met, data collection is stopped.

[0183] For example, when the wearable device detects that the user is in motion, it can use a sampling frequency of 100Hz, that is, 100 points per second. Alternatively, when the wearable device detects that the user is at rest, it can use a sampling frequency of 25Hz, that is, 25 points per second.

[0184] Optionally, the sampling time window duration can be 8 seconds, 10 seconds, 12 seconds, or any other duration that meets the requirements. The specific duration can be set according to actual usage needs, and this application embodiment does not impose any limitations.

[0185] For example, with a sampling frequency of 25Hz and a time window duration of 12 seconds, if 25 points are sampled every second, a total of 300 points can be sampled in 12 seconds.

[0186] S202 organizes all collected sample data to obtain multi-channel PPG signals, ACC three-channel sensor signals, heart rate label data, and motion scene label data, forming an offline dataset.

[0187] A comparison of the proposed solution with traditional algorithms reveals that traditional algorithms require manual extraction of numerous empirical features to train the model, while the proposed solution enables end-to-end learning during model training, adaptively extracting data in different scenarios, thus making model training more intelligent.

[0188] S203 performs FFT transformation on the PPG signal and ACC signal respectively to obtain the spectrum of the PPG signal and the spectrum of the ACC signal.

[0189] S204 performs bandpass filtering on the spectrum of the PPG signal and the spectrum of the ACC signal respectively to obtain the effective spectrum data of PPG and the effective spectrum data of ACC.

[0190] For example, assuming [0.7Hz, 4Hz] is a commonly used frequency range for heart rate detection, when filtering the spectra of the PPG signal and the ACC signal respectively, noise data outside [0.7Hz, 4Hz] can be filtered out, thus determining the frequency range useful for heart rate detection. The spectral data within this frequency range is called the effective spectral data.

[0191] Assuming a typical human heart rate of 60 bpm, the heart rate range corresponding to the commonly used frequency range [0.7 Hz, 4 Hz] can be [42 bpm, 240 bpm], which can be considered a reference range for heart rate values.

[0192] In this application, the execution order of bandpass filtering and FFT transformation is not limited. For example, FFT transformation can be performed first and then bandpass filtering can be performed; of course, bandpass filtering can also be performed first and then FFT transformation can be performed.

[0193] For example, still assuming the sampling frequency is 25Hz and the time window is 12 seconds, then 25 points are sampled every 1 second, and a total of 300 points can be sampled in 12 seconds; after filtering, [0, 127] sampling points can be obtained.

[0194] Accordingly, after FFT transformation, 128 effective frequency points can be obtained from the 300 sampled points.

[0195] It should be noted that the embodiments of this application do not limit the type of filter used, and the specific filter can be determined according to actual usage requirements.

[0196] For ease of explanation, the acquired PPG and ACC signals will be collectively referred to as sensor time-domain data, and the data obtained after FFT transformation and filtering of the PPG and ACC signals will be collectively referred to as sensor frequency-domain data.

[0197] For example, Figure 4 The waveform diagram of the sensor frequency domain data after filtering is shown. The vertical axis represents the FFT amplitude value and the horizontal axis represents the number of samples. Assuming the sampling frequency is 25Hz, the corresponding horizontal axis resolution of the FFT is 25 / 256 = 0.097Hz.

[0198] like Figure 4As shown in (a), in the original spectrogram, the FFT horizontal axis points range is [0, 127]. Since the FFT horizontal axis unit interval resolution is 0.097Hz, the frequency range corresponding to the FFT horizontal axis point range [0, 127] is [0Hz, 127*0.097Hz], approximately [0Hz, 12Hz]. In other words, the original spectrogram includes spectral data in the frequency range [0Hz, 12Hz], which contains noise data.

[0199] The original spectrogram is filtered to remove noise data, such as... Figure 4 As shown in (b), in the filtered spectrum, the FFT abscissa points range from [5, 35]. Since the FFT abscissa unit interval resolution is 0.097Hz, the frequency range corresponding to the FFT abscissa point range [5, 35] is [5*0.097Hz, 35*0.097Hz], approximately [0.5Hz, 3.5Hz]. The processed spectrum includes spectral data with a frequency range of [0.7Hz, 3.5Hz], and the resulting frequency domain data is considered valid spectral data.

[0200] In this way, using the effective spectral data after filtering out noise data for model training helps to eliminate the interference of motion artifact noise on the PPG signal.

[0201] S205 uses the extracted sensor frequency domain data as model input and heart rate and motion scene information as model output to train a deep Attention network.

[0202] Specifically, PPG effective spectrum data and ACC effective spectrum data can be used as inputs to the deep Attention network, while heart rate label data and motion scene label data can be used as target values ​​for offline training of the deep Attention network.

[0203] In this application embodiment, since the distribution of motion artifact noise is different in different scenarios, the solution of this application utilizes the end-to-end training method of deep Attention network to simultaneously obtain the ability to recognize the scene and the ability to detect heart rate, and to fit the nonlinear relationship between scene, noise and heart rate to a greater extent.

[0204] S206, determine whether the training objective has been achieved.

[0205] If the training objective is not achieved, proceed to step S207; if the training objective is achieved, proceed to step S208.

[0206] S207 uses cross-validation to adjust the parameters used to train the model in S205, continuously optimizing the model.

[0207] Model optimization can be achieved by adjusting the weights and biases of the network (e.g., convolutional neurons).

[0208] In this embodiment, the parameters are continuously adjusted and the model is optimized until the training objective is achieved, even if the training objective is not reached. This enables the model to learn the label information of complex scenes and the adaptive weight parameters of the Attention network, thereby achieving the effects of signal denoising, signal fusion, and complex scene recognition in complex multi-scene scenarios.

[0209] Comparing the proposed solution with traditional algorithms reveals that the adaptive filtering, principal component analysis, and signal decomposition methods used in traditional heart rate prediction algorithms are only applicable to specific noisy scenarios. As model algorithms with fixed parameters, they cannot adaptively predict heart rate values ​​in complex scenarios. In contrast, the proposed solution utilizes the Attention mechanism to adaptively fit the characteristics of scene signals and noise, adaptively adjust network weight parameters, and specifically address motion artifacts in various scenarios, thereby improving generalization ability.

[0210] S208, quantization model, output model file.

[0211] In this embodiment, a deep attention network is used for offline training to obtain a deep attention model. The output task of this model is heart rate value and scene information. Therefore, this deep attention model can also be called a scene recognition and heart rate detection model.

[0212] In this embodiment, the trained model can be quantized using 8-bit or 16-bit parameters to store its network structure information and quantized parameter data. Since model quantization reduces the storage space of wearable devices and accelerates integer-type computation, it can reduce the storage space of wearable device components and improve computation speed.

[0213] This application employs a deep Attention network to learn through an attention mechanism, performing different noise reduction operations based on scene information. The resulting deep Attention network model is a dynamic, adaptive model capable of end-to-end learning. It can adaptively perform data denoising and feature extraction in different scenarios, achieving signal denoising, signal fusion, and complex scene recognition in complex multi-scene environments. Therefore, using the trained deep Attention network model for heart rate detection can improve the accuracy of heart rate detection.

[0214] The above combination Figure 3This section details the process of offline training using a deep Attention network to obtain scene recognition and heart rate detection models. The following section will further elaborate on this process. Figure 5 Describe in detail the process of online prediction using the trained scene recognition and heart rate detection models. For example... Figure 5 As shown, the online prediction process includes the following steps S301-S313.

[0215] S301, Model initialization.

[0216] The wearable device activates the heart rate detection module and loads the pre-saved structure and parameters of the deep attention model to build the forward inference process.

[0217] S302, real-time acquisition of PPG and ACC signals.

[0218] Wearable devices collect PPG and ACC signals synchronously and in real time.

[0219] S303, determine whether the data collection completion conditions are met.

[0220] Optionally, when the amount of collected data reaches a preset amount, the collection can be considered to be complete.

[0221] Optionally, when the data collection time reaches the preset time, the data collection can be considered to be completed.

[0222] For example, when the acquisition of PPG and ACC signals begins, a timer is started. The timer duration can be set to a preset duration. Once the timer expires, it is considered that the data acquisition has been completed and the acquisition completion condition has been met.

[0223] After the acquisition completion conditions are met, proceed to the next step S305; otherwise, execute step S304 to continue acquiring PPG and ACC signals until the acquisition completion conditions are met.

[0224] S305 performs FFT transformation on the acquired PPG signal and ACC signal respectively to obtain the spectrum of the PPG signal and the spectrum of the ACC signal.

[0225] S306 performs bandpass filtering on the spectrum of the PPG signal and the spectrum of the ACC signal respectively to obtain the effective spectrum data of PPG and the effective spectrum data of ACC.

[0226] S307 determines the frequency domain location and amplitude of the top N highest peaks based on the effective spectrum data of PPG and ACC.

[0227] For example, when N is 3, the wearable device can determine the frequency domain position and amplitude of the top 3 highest peaks of the PPG effective spectrum data, as well as the frequency domain position and amplitude of the top 3 highest peaks of the ACC effective spectrum data.

[0228] In the frequency domain positions and amplitudes of the top N highest peaks in the sensor's frequency domain data, the frequency domain positions and amplitudes of the PPG and ACC spectral peaks can be represented in array form, for example, [F p A p F a A a ], hereinafter referred to as the peak array, where F p and A p F represents the frequency domain location and amplitude of the PPG spectral peak, respectively. a and A a These represent the frequency domain position and amplitude of the ACC spectrum peak, respectively.

[0229] The deep heart rate detection model trained through the embodiments of this application can reduce the computational power consumption of wearable devices by combining the FFT three-peak data recording and comparison mechanism.

[0230] S308 stores the frequency domain position and amplitude of the first N highest peaks of the sensor frequency domain data in the data buffer.

[0231] In this embodiment of the application, the wearable device may be equipped with a data buffer and a label buffer.

[0232] The data buffer can be a first-in, first-out buffer of a certain length, used to store a certain number of peak values. The tag buffer can be used to store heart rate values ​​obtained from heart rate detection using a model.

[0233] S309, determine whether the current peak array has appeared in the data cache.

[0234] On the one hand, if the current peak array has appeared in the data cache, it means that the heart rate value has been predicted in advance using the model. In other words, the current peak array has a corresponding heart rate value, and the corresponding heart rate value is stored in the label cache. Then, continue to execute the following step S310.

[0235] On the other hand, if the current peak array has not appeared in the data cache, it means that the current peak array does not have a corresponding heart rate value, and the model needs to be used to predict the heart rate. Therefore, continue to execute the following step S311, that is, use the deep Attention network model to perform forward inference to calculate the heart rate value.

[0236] In some possible implementations, the heart rate monitoring is first performed for a preset duration (e.g., 12 seconds), the heart rate monitoring data is stored in a data cache, and the corresponding heart rate monitoring results are stored in a tag cache. Then, in the subsequent heart rate monitoring process, the judgment action of S309 above can be performed to compare the currently acquired heart rate monitoring data with the heart rate monitoring data stored in the data cache and determine whether the currently acquired heart rate monitoring data has appeared in the data cache.

[0237] By using the above approach, which utilizes a data cache and performs historical review, redundant calculations can be avoided, thus reducing the computing power consumption of wearable devices.

[0238] S310, when it is determined in S309 that the current peak array has appeared in the data buffer, read the corresponding heart rate and scene information stored in the tag buffer.

[0239] After executing S310, continue with the following step S313.

[0240] S311, when it is determined in S309 that the current peak array has not appeared in the data buffer, the effective spectrum data of PPG and ACC are used as the input of the deep Attention model. The deep Attention model performs forward inference and calculates the corresponding heart rate and scene information.

[0241] The deep heart rate detection model trained through the embodiments of this application can accelerate the calculation speed in the real-time prediction stage by using a simplified quantization model.

[0242] S312 stores heart rate and scene information in the tag cache.

[0243] The purpose of storing heart rate values ​​and scene information in the tag cache is as follows: When the peak array appears in the data cache, it means that the heart rate value has already been predicted using a model, and the tag cache already stores the corresponding heart rate and scene information. Therefore, the corresponding heart rate value and scene information stored in the tag cache can be read directly. In this case, there is no need to use a scene recognition model to identify the current scene, nor is it necessary to use a heart rate detection model to detect the heart rate value. This avoids redundant calculations and reduces the computing power consumption of the wearable device.

[0244] S313 transmits heart rate and scene information to the wearable device's user interface (UI) for display.

[0245] In this embodiment, the wearable device can perform monitoring operations such as heart rate alerts and transmit scene information to other components of the wearable device. This allows the power consumption and intelligent dimming algorithm of the wearable device to be adjusted according to actual usage needs.

[0246] Comparing the proposed solution with traditional algorithms reveals that traditional heart rate detection algorithms do not simultaneously perform heart rate detection and scene recognition. The proposed solution argues that there is a strong correlation between scene and heart rate, and ignoring these relationships will lead to low model recognition capabilities. Therefore, the proposed solution employs a deep attention learning mechanism, which can recognize various motion scenes and adaptively fit scenes and noise accordingly, without requiring excessive data preprocessing or the formulation of manual experience parameters.

[0247] The first embodiment described the deep learning-based heart rate detection method, along with its corresponding deep Attention model training and online prediction processes. The second embodiment, providing a deep learning-based heart rate detection method, will now be described, along with its corresponding model training and online prediction processes.

[0248] Second Embodiment

[0249] The following is combined with Figure 6 This application provides a detailed description of the deep learning-based heart rate detection method provided in the second embodiment.

[0250] Figure 6 This is a flowchart illustrating a heart rate detection method provided in the second embodiment of this application. (Refer to...) Figure 6 As shown, the method includes the following steps S401-S405.

[0251] S401, when the user wears the wearable device, responds to the heart rate detection command and acquires the PPG signal and ACC signal.

[0252] S402 performs the first preprocessing on the ACC signal and PPG signal respectively to obtain the effective spectrum data of ACC and the effective spectrum data of PPG.

[0253] For a detailed description of S401 and S402, please refer to the detailed description of S101 and S102 in the first embodiment above, which will not be repeated here.

[0254] S403, input the effective ACC spectrum data into the second prediction model to obtain scene information; wherein, the second prediction model is a model trained by a deep neural network with ACC sample data as input and motion scene labels as target variables, and the second prediction model has scene recognition function.

[0255] In this embodiment, the wearable device can use ACC effective spectrum data as input parameters for the second prediction model and execute an online prediction process using the second prediction model. Through this second prediction model (also known as a scene recognition model), the wearable device can identify whether the user's current activity scene is resting or active, and can accurately identify the specific type of activity, such as walking, swimming, running, mountain climbing, etc.

[0256] Similarly, for ease of explanation, the motion scene identified by the wearable device will still be described here as scene information.

[0257] S404. Input the effective PPG spectrum data and the obtained scene information into the third prediction model to obtain heart rate information. The third prediction model is a model trained by a deep Attention network with PPG sample data, ACC sample data and motion scene labels as inputs and heart rate labels as the target variable. The third prediction model has heart rate detection function.

[0258] In this embodiment, after the wearable device identifies the exercise scene, the identified scene information and PPG effective spectrum data can be used as input parameters for the third prediction model, and the online prediction process can be executed using the third prediction model. Through this third prediction model (also known as the heart rate detection model), the wearable device can accurately detect the user's heart rate value in the current exercise scene.

[0259] For ease of explanation, the heart rate value detected by the wearable device will still be described as heart rate information. Thus, after executing the online prediction process using the second and third prediction models respectively, scene information and heart rate information can be output.

[0260] It should be noted that the third prediction model provided in this application embodiment is a scene recognition and heart rate detection model trained using a deep Attention network based on heart rate information and motion scene information. In other words, the solution in this application combines motion scene information and heart rate information for model training, which can reduce motion noise and avoid the impact of motion artifacts on the accuracy of heart rate detection. Therefore, the heart rate value output by this third prediction model is more accurate.

[0261] The offline training processes for the second prediction model (i.e., the scene recognition model) and the third prediction model (i.e., the heart rate detection model) will be described in detail below, and will not be repeated here.

[0262] S405 displays heart rate and scene information.

[0263] For a detailed description of S405, please refer to the detailed description of S104 in the first embodiment above, which will not be repeated here.

[0264] The following is combined with Figure 7 and Figure 8 The offline training process and online prediction process of the model in the second embodiment are described separately.

[0265] Figure 7 This describes the offline training process for the scene recognition model and heart rate detection model in the second embodiment. For example... Figure 7 As shown, the offline training process for the model includes steps S501-S512. Specifically, a deep neural network is used for offline training to obtain the scene recognition model, and an Attention network is used for offline training to obtain the heart rate detection model.

[0266] S501, acquire sample data from multiple scenarios.

[0267] Among them, multi-scenario sample data is also called multi-scenario sample set.

[0268] The S502 organizes all collected sample data to obtain multi-channel PPG signals, ACC three-channel sensor signals, heart rate label data, and scene label data, forming an offline dataset.

[0269] S503 performs FFT transformation on the PPG signal and ACC signal respectively to obtain the spectrum of the PPG signal and the spectrum of the ACC signal.

[0270] S504 filters the spectrum of the PPG signal and the spectrum of the ACC signal, filtering out noise data outside [0.7Hz, 4Hz], and obtains the effective spectrum data of PPG and ACC.

[0271] In addition, motion scene label data and heart rate label data were obtained from multi-scene sample data.

[0272] For a detailed description of S501-S504, please refer to the detailed description of S201-S204 in the first embodiment above, which will not be repeated here.

[0273] S505 uses PPG effective spectrum data and ACC effective spectrum data (3) as input to the deep neural network, and motion label data (1) as the target value, i.e. motion scene information as the output, to train the deep neural network so that the deep neural network learns how to recognize multi-scene information.

[0274] The embodiments of this application employ a deep attention learning mechanism, which can recognize various motion scenes and perform corresponding adaptive fitting of scenes and noise without requiring excessive data preprocessing or the formulation of manual experience parameters.

[0275] S506 determines whether the training objective has been achieved.

[0276] If the training objective is not achieved, proceed to step S507; if the training objective is achieved, proceed to step S508.

[0277] S507 uses cross-validation to adjust the parameters used to train the model in S505, continuously optimizing the model.

[0278] In this embodiment, if the training objective is not achieved, the model parameters can be adjusted through cross-validation to continuously optimize the model until the training objective is achieved. This allows the deep neural network to learn how to recognize information from multiple scenes, achieving the effects of signal denoising, signal fusion, and complex scene recognition in complex multi-scene environments.

[0279] S508, quantization model, output model file.

[0280] In this embodiment, a deep neural network is used for offline training to obtain a deep neural network model. The output task of this model is motion scene information, so this deep neural network model can also be called a scene recognition model.

[0281] This involves quantizing the trained model using 8-bit or 16-bit parameters to store its network structure information and quantized parameter data. This operation can reduce the storage space required for wearable devices and accelerate computation.

[0282] The above describes the scene recognition model trained through steps S505-S508. The following describes the heart rate detection model trained through steps S509-S512.

[0283] S509 uses motion scene label data (1), PPG effective spectrum data and ACC effective spectrum data (2) as input to the deep Attention network, and heart rate label data (3) as the training target value, i.e. heart rate information as output, to guide the Attention network to learn to use nonlinear adaptive learning on motion scene data and sensor frequency domain data to train the Attention network's heart rate detection capability.

[0284] In this application embodiment, since the distribution of motion artifact noise varies in different scenarios, the solution of this application utilizes the end-to-end training method of deep Attention network to fit the nonlinear relationship between scene, noise and heart rate to a greater extent, which can improve the accuracy of heart rate detection in different scenarios.

[0285] S510 determines whether the training objective has been achieved.

[0286] If the training objective is not achieved, proceed to step S511; if the training objective is achieved, proceed to step S512.

[0287] S511 uses cross-validation to adjust the parameters used to train the model in S510, continuously optimizing the model.

[0288] Model optimization can be achieved by adjusting the weights and biases of the network (e.g., convolutional neurons).

[0289] In this embodiment of the application, if the training objective is not achieved, the model parameters can be adjusted through cross-validation to continuously optimize the model until the training objective is achieved, so that the deep Attention network can learn to detect heart rate values ​​under different sports scenarios.

[0290] The proposed solution utilizes the Attention mechanism to adaptively fit the characteristics and noise of scene signals, adaptively adjust network weight parameters, and specifically address motion artifacts in various scenes, thereby improving generalization ability.

[0291] S512, quantization model, output model file.

[0292] In this embodiment, a deep attention network is used for offline training to obtain a deep attention model. The output task of this model is the heart rate value, so this deep attention model can also be called a heart rate detection model.

[0293] This involves quantizing the trained model using 8-bit or 16-bit parameters to store its network structure information and quantized parameter data. After model quantization, the storage space of wearable devices is reduced, while the computation speed for integer types is increased. Therefore, this operation achieves the effects of reducing the storage space of wearable device components and accelerating computation.

[0294] The offline training process of the model in the first embodiment will be compared with that in the second embodiment below. The differences between the two are as follows:

[0295] In the first embodiment, the sensor frequency domain signal is used as the input of the Attention network, and the motion scene information and heart rate information are used as the output. The trained Attention network model has scene recognition function and heart rate detection function.

[0296] In the second embodiment, sensor frequency domain data and motion scene label data are used as inputs to the Attention network, and heart rate information is used as output. The trained Attention network model has heart rate detection function.

[0297] The above combination Figure 7 This document details the process of obtaining a scene recognition model through offline training using a deep neural network, and a heart rate detection model through offline training using a deep attention network. The following section will further elaborate on this process. Figure 8 Describe in detail the process of online prediction using a trained deep neural network and an attention network. For example... Figure 8 As shown, the online prediction process includes the following steps S601-S614.

[0298] S601, Model initialization.

[0299] The wearable device activates the heart rate detection module and loads the pre-saved structure and parameters of the deep neural network model and deep attention model to build the forward inference process.

[0300] S602, real-time acquisition of PPG and ACC signals.

[0301] S603, determine whether the data collection completion conditions are met.

[0302] The wearable device collects PPG and ACC signals synchronously and in real time. After the collection completion condition is met, it proceeds to the next step S605; otherwise, it executes step S604 to continue collecting PPG and ACC signals until the collection completion condition is met.

[0303] S605 performs FFT transformation on the acquired PPG signal and ACC signal respectively to obtain the spectrum of the PPG signal and the spectrum of the ACC signal.

[0304] S606 performs bandpass filtering on the PPG frequency domain signal and the ACC frequency domain signal respectively to obtain the effective spectrum data of PPG and the effective spectrum data of ACC.

[0305] In this application, the execution order of bandpass filtering and FFT transformation is not limited. For example, FFT transformation can be performed first and then bandpass filtering can be performed; of course, bandpass filtering can also be performed first and then FFT transformation can be performed.

[0306] S607, determine the frequency domain location and amplitude of the N highest peaks in the sensor's frequency domain data.

[0307] The sensor frequency domain data includes PPG effective spectrum data and ACC effective spectrum data.

[0308] Where N can be a positive integer. For example, when N is 3, the wearable device can determine the frequency domain position and amplitude of the first 3 highest peaks of the PPG effective spectrum data, as well as the frequency domain position and amplitude of the first 3 highest peaks of the ACC effective spectrum data.

[0309] In the frequency domain positions and amplitudes of the top N highest peaks in the sensor's frequency domain data, the frequency domain positions and amplitudes of the PPG and ACC spectral peaks can be represented in array form, for example, [F p A p F a A a ], hereinafter referred to as the peak array.

[0310] The deep heart rate detection model trained through the embodiments of this application can reduce the computational power consumption of wearable devices by combining the FFT first N peak data recording and comparison mechanism.

[0311] S608 stores the frequency domain position and amplitude of the N highest peaks of the sensor frequency domain data in the data buffer.

[0312] In this embodiment of the application, the wearable device may be equipped with a data cache and a tag cache.

[0313] The data buffer can be a first-in, first-out buffer of a certain length, used to store a certain number of peak values. The tag buffer can be used to store heart rate values ​​obtained from heart rate detection using a model.

[0314] S609, determine whether the current peak array has appeared in the data cache.

[0315] On the one hand, if the current peak array has appeared in the data cache, it means that the heart rate value has been predicted in advance using the model. In other words, the current peak array has a corresponding heart rate value, and the corresponding heart rate value is stored in the label cache. Then, continue to execute the following step S610.

[0316] On the other hand, if the current peak array has not appeared in the data cache, it means that the current peak array does not have a corresponding heart rate value, and the model needs to be used to predict the heart rate. Therefore, continue to execute the following step S611, that is, use the deep Attention network model to perform forward inference to calculate the heart rate value.

[0317] S611, when it is determined in S609 that the current peak array has not appeared in the data buffer, the effective spectrum data of PPG and ACC are used as the input of the neural network model, the deep neural network model is used to identify the current motion scene information, and the motion scene information is passed to the deep Attention model.

[0318] S612 takes the identified motion scene information, PPG and ACC effective spectrum data as input to the deep Attention model, and the deep Attention model performs forward inference to calculate the heart rate value in the current scene.

[0319] The deep heart rate detection model trained through the embodiments of this application can accelerate the calculation speed in the real-time prediction stage by using a simplified quantization model.

[0320] S613 stores heart rate values ​​and scene information in the tag cache.

[0321] The purpose of storing heart rate values ​​and scene information in the tag cache is as follows: when the peak array appears in the data cache, it means that the heart rate value has been predicted in advance using the model, and the corresponding heart rate value and scene information have been stored in the tag cache. Therefore, the corresponding heart rate value and scene information stored in the tag cache can be read directly. In this case, there is no need to use the scene recognition model to identify the current scene, nor is there a need to use the heart rate detection model to detect the heart rate value.

[0322] The S614 transmits heart rate values ​​and scene information to the wearable device's UI for display.

[0323] S610, when it is determined in S609 that the current peak array has appeared in the data buffer, read the corresponding heart rate value and scene information stored in the tag buffer.

[0324] By using the above approach, which utilizes a data cache and performs historical review, redundant calculations can be avoided, thus reducing the computing power consumption of wearable devices.

[0325] After executing S610, continue with step S614, which transmits the heart rate value and scene information to the UI interface of the wearable device for display.

[0326] In this embodiment, the wearable device can perform monitoring operations such as heart rate alerts and transmit scene information to other components of the wearable device. This allows the power consumption and intelligent dimming algorithm of the wearable device to be adjusted according to actual usage needs.

[0327] The online prediction process of the model in the second embodiment will be compared with that in the first embodiment. The differences between the two are as follows:

[0328] In the first embodiment, PPG and ACC frequency domain data are used as inputs to the deep Attention model. The deep Attention model outputs motion scene information and heart rate information simultaneously. The model has scene recognition and heart rate detection functions.

[0329] In the second embodiment, the scene recognition function and the heart rate detection function are separated. First, a deep neural network model is used to recognize scene information. Then, the scene recognition result and sensor frequency domain data are used as inputs to the deep attention model, and the deep attention model outputs the heart rate value.

[0330] Figure 9 This is a schematic diagram of the interface for the application of the deep learning-based heart rate detection method provided in the embodiments of this application.

[0331] like Figure 9 As shown in (a), when the wearable device 11 detects that the user wearing the wearable device 11 is in a resting state, the wearable device 11 performs heart rate detection and displays the detection result: 70 beats / min, resting. Therefore, the heart rate detection result of this application can include heart rate and current scene information. When the wearable device 11 maintains a wireless connection with the terminal device 12, the wearable device 11 can send the heart rate detection result to the terminal device 12 for display by the user on the terminal device 12.

[0332] like Figure 9 As shown in (b), when the wearable device 11 detects that the user is walking, the wearable device 11 performs heart rate detection and displays the result: 85 beats / min, walking. Therefore, the heart rate detection result of this application can include heart rate and current scene information. When the wearable device 11 maintains a wireless connection with the terminal device 12, the wearable device 11 can send the heart rate detection result to the terminal device 12 for display by the user.

[0333] like Figure 9 As shown in (c), when the wearable device 11 detects that the user is running, the wearable device 11 performs heart rate detection and displays the result: 120 beats / min, running. Therefore, the heart rate detection result of this application can include heart rate and current scene information. When the wearable device 11 maintains a wireless connection with the terminal device 12, the wearable device 11 can send the heart rate detection result to the terminal device 12 for display by the user on the terminal device 12.

[0334] In summary, the proposed solution achieves the following technical effects:

[0335] On the one hand, deep attention network models learn through attention mechanisms and perform different noise removal operations based on scene information. They are dynamic and adaptive models that can learn end-to-end and can adaptively achieve data denoising and feature extraction in different scenarios.

[0336] On the other hand, deep attention learning mechanisms can recognize various motion scenarios and adaptively fit the scene and noise accordingly, without requiring excessive data preprocessing or the formulation of manual experience parameters.

[0337] On the other hand, the deep heart rate detection model proposed in this application can accelerate the calculation speed in the real-time prediction stage by using a simplified quantization model, while the combination of FFT first three peak data recording and comparison mechanism can reduce the computing power consumption of wearable devices.

[0338] It should be noted that the solutions provided in this application can also be applied to other scenarios such as blood pressure estimation and blood oxygen concentration calculation. The specific implementation methods for blood pressure estimation or blood oxygen concentration calculation are similar to the implementation methods for heart rate detection using a deep attention network model in the first and second embodiments described above.

[0339] It should also be noted that in the embodiments of this application, "greater than" can be replaced with "greater than or equal to", "less than or equal to" can be replaced with "less than", or "greater than or equal to" can be replaced with "greater than", and "less than" can be replaced with "less than or equal to".

[0340] The various embodiments described herein can be independent solutions or combinations thereof based on their inherent logic, and all such solutions fall within the protection scope of this application.

[0341] It is understood that the methods and operations implemented by wearable devices in the above-described method embodiments can also be implemented by components (such as chips or circuits) that can be used in wearable devices.

[0342] The method embodiments provided in this application have been described above. The apparatus embodiments provided in this application will be described below. It should be understood that the description of the apparatus embodiments corresponds to the description of the method embodiments. Therefore, any content not described in detail can be referred to the method embodiments above. For the sake of brevity, it will not be repeated here.

[0343] The foregoing mainly describes the solutions provided by the embodiments of this application from the perspective of method steps. It is understood that, in order to achieve the above functions, the wearable device implementing this method includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of protection of this application.

[0344] This application embodiment can divide wearable devices into functional modules based on the above method examples. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware or as software functional modules. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, other feasible division methods may exist. The following description uses the division of functional modules according to each function as an example.

[0345] Figure 10 This is a schematic block diagram of a deep learning-based heart rate detection device 800 provided in an embodiment of this application. The device 800 can be used to perform the actions performed by the wearable device in the method embodiments described above. The device 800 includes an acquisition unit 810, a processing unit 820, and a display unit 830.

[0346] The acquisition unit 810 is used to acquire the photoplethysmography (PPG) signal and acceleration (ACC) signal in response to a heart rate detection command when the user wears the wearable device.

[0347] Processing unit 820 is used to obtain heart rate and scene information based on the ACC effective spectrum data corresponding to the ACC signal, the PPG effective spectrum data corresponding to the PPG signal, and the target prediction model.

[0348] Display unit 830 is used to display the heart rate and scene information on the screen of the wearable device;

[0349] The target prediction model is a model trained using ACC sample data and PPG sample data as inputs, and heart rate labels and sports scene labels as target variables, through a deep attention mechanism Attention network. The target prediction model has scene recognition and heart rate prediction functions.

[0350] This solution uses PPG and ACC sensor signals as input and heart rate and motion scene information as model output to train a deep attention network. The trained model is then used for heart rate detection. Since motion artifact noise distribution varies across different scenarios, the deep attention network's learning mechanism can fit the non-linear relationship between scene, noise, and heart rate. This allows for adaptive denoising and feature extraction in different scenarios, achieving signal denoising, signal fusion, and complex scene recognition in complex multi-scene environments. This eliminates the interference of motion artifact noise on the PPG signal, improving the accuracy of heart rate detection. Therefore, this solution addresses the problem of inaccurate heart rate detection results caused by motion artifact interference with the PPG signal.

[0351] In some possible implementations, the device 800 further includes a storage unit 840. Exemplarily, the storage unit 840 may include a tag cache and a data cache. Specifically, the storage unit 840 may be used for:

[0352] Upon obtaining heart rate and scene information, the heart rate and scene information are recorded in a tag cache;

[0353] The spectral peak data corresponding to the heart rate and scene information are recorded in the data cache;

[0354] The spectral peak data corresponding to the heart rate and scene information includes the PPG spectral peak position and amplitude, as well as the ACC spectral peak position and amplitude.

[0355] This solution allows for initial model execution during heart rate detection, storing the heart rate and scene information obtained from the model, along with the peak positions and amplitude values ​​of the corresponding PPG and ACC frequency domain data. Later, during heart rate detection, this stored data can be directly retrieved when certain conditions are met, saving computational resources, reducing latency, and enabling faster heart rate value acquisition.

[0356] In some possible implementations, the acquisition unit 810 is specifically used for:

[0357] Based on the effective spectrum data of PPG and the effective spectrum data of ACC, obtain the spectrum data of the first N peak points;

[0358] If the data buffer does not have any of the peak spectrum data of the first N peaks in advance, then the heart rate and scene information are obtained based on the effective ACC spectrum data corresponding to the ACC signal, the effective PPG spectrum data corresponding to the PPG signal, and the target prediction model.

[0359] If any of the peak spectrum data from the first N peak spectrum data has been pre-stored in the data cache, then the heart rate and scene information corresponding to any of the peak spectrum data are read from the tag cache.

[0360] This method allows for the determination of the positions and amplitudes of the three highest peaks in both the PPG and ACC frequency domains. It then checks if the current data has appeared in the data buffer; if not, it is entered into the model. If it has appeared, the corresponding heart rate and scene information stored in the tag buffer are read. This saves computational resources, reduces latency, and allows for faster acquisition of heart rate values.

[0361] In some possible implementations, the processing unit 820 is specifically used for:

[0362] The effective ACC spectrum data and the effective PPG spectrum data are input into the first prediction model to obtain the heart rate and scene information;

[0363] The target prediction model is the first prediction model, which is a model trained by a deep attention network with ACC sample data and PPG sample data as inputs and heart rate label and sports scene label as target variables. The first prediction model has scene recognition and heart rate prediction functions.

[0364] The above-described solution provides a heart rate detection method: taking PPG and ACC frequency domain data as input, and simultaneously outputting motion scene information and heart rate information. Thus, in practical implementation, the wearable device can simultaneously output current scene information and heart rate value when detecting heart rate. This application's solution takes into account user behavior or exercise state, and can support the detection of heart rate values ​​when the user is in different behavioral states (e.g., resting state, and various exercise states). This application's solution can solve the problem of various noise interferences to PPG signals in different scenarios, achieving effective heart rate detection in various noise environments.

[0365] In some other possible implementations, the processing unit 820 is specifically used for:

[0366] The effective ACC spectrum data is input into the second prediction model to obtain scene information; wherein, the second prediction model is a model trained by a deep neural network with ACC sample data as input and motion scene labels as target variables, and the second prediction model has scene recognition function;

[0367] The effective spectrum data of PPG and the obtained scene information are input into the third prediction model to obtain heart rate information. The third prediction model is a model trained by a deep Attention network with PPG sample data, ACC sample data and motion scene labels as inputs and heart rate labels as target variables. The third prediction model has heart rate detection function.

[0368] The target prediction model includes the second prediction model and the third prediction model.

[0369] The above-mentioned solution provides another heart rate detection method: it separates the motion scene information and heart rate information recognition functions. First, a deep neural network model is used to recognize the scene information, and then the recognition result and sensor frequency domain data are used as inputs for deep attention to output the heart rate value. This application's solution can solve the problem of various noise interferences on PPG signals in different scenarios, and achieve effective heart rate detection in different noise scenarios.

[0370] In some possible implementations, the processing unit 820 is also used for:

[0371] The ACC signal and the PPG signal are respectively subjected to a first preprocessing to obtain ACC effective spectrum data and PPG effective spectrum data;

[0372] The first preprocessing includes Fast Fourier Transform (FFT) and filtering.

[0373] The above method allows for FFT transformation and filtering of the acquired signal to obtain the effective spectrum. The heart rate value can then be detected using the effective spectrum. Since the effective spectrum removes interference signals, this improves the accuracy of heart rate detection.

[0374] In some possible implementations, the acquisition unit 810 is specifically used to acquire the PPG signal via a PPG sensor and the ACC signal via an accelerometer.

[0375] The PPG signal can be used to detect heart rate, and the ACC signal can be used to identify the motion scene. This application's solution combines the use of PPG and ACC signals for heart rate detection, which can solve the problem of various noise interferences to the PPG signal in different scenarios, achieving effective heart rate detection under different noise conditions.

[0376] In some possible implementations, the device 800 may further include a transceiver unit 850 for transmitting the heart rate and scene information to a terminal device, the terminal device being an electronic device wirelessly connected to the wearable device, so that the heart rate and scene information are displayed on the screen of the terminal device.

[0377] This solution allows users to view heart rate and scene information not only through wearable devices, but also through terminal devices connected to the wearable devices, making it easier for users to view their heart rate values ​​and improving the user experience.

[0378] The apparatus 800 according to the embodiments of this application can correspond to the execution of the method described in the embodiments of this application, and the above and other operations and / or functions of the units in the apparatus 800 are respectively for implementing the corresponding process of the method, which will not be described again here for the sake of brevity.

[0379] Figure 11 This is a schematic structural diagram of a wearable device 900 provided in an embodiment of this application. The wearable device 900 includes: a processor 910, a memory 920, a communication interface 930, and a bus 940.

[0380] The processor 910 can be connected to the memory 920. The memory 920 can be used to store program code and data. Therefore, the memory 920 can be a storage unit inside the processor 910, an external storage unit independent of the processor 910, or a component that includes both the storage unit inside the processor 910 and the external storage unit independent of the processor 910.

[0381] Optionally, the wearable device 900 may also include a bus 940. The memory 920 and communication interface 930 can be connected to the processor 910 via the bus 940. The bus 940 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. This bus 940 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 11 The symbol is represented by only one line, but this does not mean that there is only one bus or one type of bus.

[0382] It should be understood that in the embodiments of this application, the processor 910 may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. Alternatively, the processor 910 may employ one or more integrated circuits to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0383] The memory 920 may include read-only memory and random access memory, and provides instructions and data to the processor 910. A portion of the processor 910 may also include non-volatile random access memory. For example, the processor 910 may also store device type information.

[0384] When the wearable device 900 is running, the processor 910 executes computer execution instructions in the memory 920 to perform the operation steps of the above method.

[0385] It should be understood that the wearable device 900 according to the embodiments of this application can correspond to the device 800 in the embodiments of this application, the processor 910 in the wearable device 900 can correspond to the processing unit 820 in the device 800, the memory 920 in the wearable device 900 can correspond to the storage unit 840 in the device 800, and the communication interface 930 in the wearable device 900 can correspond to the transceiver unit 850 in the device 800. The above and other operations and / or functions of each unit in the wearable device 900 are used to implement the corresponding processes of the above method, and for the sake of brevity, they will not be described in detail here.

[0386] Optionally, in some embodiments, this application also provides a computer-readable medium storing program code that, when run on a computer, causes the computer to perform the methods described in the above aspects.

[0387] Optionally, in some embodiments, this application also provides a computer program product comprising: computer program code, which, when run on a computer, causes the computer to perform the methods described above.

[0388] In this embodiment, the wearable device includes a hardware layer, an operating system layer running on top of the hardware layer, and an application layer running on top of the operating system layer. The hardware layer may include hardware such as a central processing unit (CPU), a memory management unit (MMU), and main memory. The operating system layer can be any one or more computer operating systems that implement business processing through processes, such as Linux, Unix, Android, iOS, or Windows. The application layer may include applications such as browsers, address books, word processing software, and instant messaging software.

[0389] This application does not impose any particular limitation on the specific structure of the execution subject of the method provided in this application embodiment. As long as it is possible to communicate according to the method provided in this application embodiment by running a program that records the code of the method provided in this application embodiment. For example, the execution subject of the method provided in this application embodiment can be a wearable device, or a functional module in a wearable device that can call and execute a program.

[0390] Various aspects or features of this application can be implemented as methods, apparatus, or articles of manufacture using standard programming and / or engineering techniques. As used herein, the term "article of manufacture" can encompass a computer program accessible from any computer-readable device, carrier, or medium. For example, computer-readable media can include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks, or magnetic tapes), optical discs (e.g., compact discs (CDs), digital versatile discs (DVDs), etc.), smart cards, and flash memory devices (e.g., erasable programmable read-only memory (EPROMs), cards, sticks, or key drives, etc.).

[0391] The various storage media described herein may represent one or more devices and / or other machine-readable media used for storing information. The term "machine-readable media" may include, but is not limited to, wireless channels and various other media capable of storing, containing and / or carrying instructions and / or data.

[0392] It should be understood that the processor mentioned in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0393] It should also be understood that the memory mentioned in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM). For example, RAM can be used as an external cache. By way of example and not limitation, RAM can include a variety of forms, such as: static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0394] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, the memory (storage module) can be integrated into the processor.

[0395] It should also be noted that the memory described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0396] Those skilled in the art will recognize that the units and steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of protection of this application.

[0397] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0398] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of apparatuses or units may be electrical, mechanical, or other forms.

[0399] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0400] In addition, the functional units in the various embodiments of this application can be integrated into one unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0401] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium and includes several instructions that cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0402] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0403] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A heart rate detection method based on deep learning, characterized in that, include: When the user wears the wearable device, in response to the heart rate detection command, it acquires the photoplethysmography (PPG) signal and the acceleration (ACC) signal. The effective spectrum data of the ACC signal and the effective spectrum data of the PPG signal are input into the first prediction model to obtain heart rate and scene information. The heart rate and scene information are displayed on the screen of the wearable device; The first prediction model takes ACC sample data and PPG sample data obtained from multiple motion scene detections as input, and heart rate labels and motion scene labels as target variables. It is trained by a deep attention mechanism Attention network. The model uses the attention mechanism to adaptively fit scene signals and noise, adaptively adjusts network weight parameters, and specifically processes motion artifacts in various scenes. This allows the model to learn the label information of complex scenes and the adaptive weight parameters of the deep attention mechanism network. The first prediction model has scene recognition and heart rate prediction functions.

2. The method according to claim 1, characterized in that, The method further includes: Upon obtaining heart rate and scene information, the heart rate and scene information are recorded in a tag cache; The spectral peak data corresponding to the heart rate and scene information are recorded in the data cache; The spectral peak data corresponding to the heart rate and scene information includes the PPG spectral peak position and amplitude, as well as the ACC spectral peak position and amplitude.

3. The method according to claim 2, characterized in that, The step of inputting the effective ACC spectrum data corresponding to the ACC signal and the effective PPG spectrum data corresponding to the PPG signal into the first prediction model to obtain heart rate and scene information includes: Based on the effective spectrum data of PPG and the effective spectrum data of ACC, obtain the spectrum data of the first N peak points; If the data buffer does not have any of the peak spectrum data of the first N peaks in advance, then the effective ACC spectrum data corresponding to the ACC signal and the effective PPG spectrum data corresponding to the PPG signal are input into the first prediction model to obtain heart rate and scene information. If any of the peak spectrum data from the first N peak spectrum data has been pre-stored in the data cache, then the heart rate and scene information corresponding to any of the peak spectrum data are read from the tag cache.

4. The method according to any one of claims 1 to 3, characterized in that, After acquiring the PPG and ACC signals, the method further includes: The ACC signal and the PPG signal are respectively subjected to a first preprocessing to obtain ACC effective spectrum data and PPG effective spectrum data; The first preprocessing includes Fast Fourier Transform (FFT) and filtering.

5. The method according to any one of claims 1 to 3, characterized in that, Acquire PPG and ACC signals, including: The PPG signal is acquired using a PPG sensor, and the ACC signal is acquired using an accelerometer.

6. The method according to any one of claims 1 to 3, characterized in that, The method further includes: The heart rate and scene information are sent to a terminal device, which is an electronic device that is wirelessly connected to the wearable device. The heart rate and scene information are displayed on the screen of the terminal device.

7. A method for training a model for detecting heart rate, characterized in that, include: Obtain a multi-scene sample set, which is a collection of data samples obtained based on the detection of multiple motion scenes; ACC sample data, PPG sample data, heart rate labels, and sports scene labels are extracted from the multi-scenario sample set. Using the ACC sample data and PPG sample data as input, and the heart rate label and the motion scene label as target variables, the model is trained through a deep attention mechanism Attention network. The attention mechanism is used to adaptively fit the scene signal and noise, adaptively adjust the network weight parameters, and specifically process motion artifacts in each scene, so that the model learns the label information of complex scenes and the adaptive weight parameters of the deep attention mechanism network. A first prediction model is obtained, which has scene recognition and heart rate prediction functions.

8. The method according to claim 7, characterized in that, The process of training a deep Attention network, using the ACC sample data and PPG sample data as inputs and the heart rate label and motion scene label as target variables, includes: The model parameters are adjusted by cross-validation, enabling the model to learn and predict heart rate information under different exercise scenarios.

9. The method according to claim 7, characterized in that, The acquisition of multi-scenario sample sets includes: Connect wearable devices and heart rate monitors to the data acquisition module; When a user wearing the wearable device and the heart rate belt device performs the first exercise, the data acquisition module acquires the first heart rate detection data of the wearable device and the second heart rate detection data of the heart rate belt device; When the user performs the second exercise, the data acquisition module acquires the third heart rate detection data of the wearable device and the fourth heart rate detection data of the heart rate belt device; Wherein, the first motion and the second motion are the motions indicated by the motion scene label, and the multi-scene sample set includes the first heart rate detection data, the second heart rate detection data, the third heart rate detection data, and the fourth heart rate detection data.

10. The method according to claim 9, characterized in that, The extraction of ACC sample data and PPG sample data from the multi-scenario sample set includes: Extract the ACC sample data and the PPG sample data from the detection data of the wearable device; The wearable device's detection data includes the first heart rate detection data and the third heart rate detection data.

11. The method according to claim 9, characterized in that, The step of extracting heart rate tags from the multi-scenario sample set includes: Extract the heart rate tag from the detection data of the heart rate belt device; The heart rate monitoring device's detection data includes the second heart rate detection data and the fourth heart rate detection data.

12. The method according to any one of claims 7 to 11, characterized in that, After extracting ACC sample data, PPG sample data, and heart rate labels from the multi-scenario sample set, the method further includes: The ACC sample data and the PPG sample data are subjected to Fast Fourier Transform (FFT) and filtering to obtain filtered ACC sample data and PPG sample data. The process involves using the ACC sample data and the PPG sample data as input, and the heart rate label and motion scene label as target variables, and training a deep Attention network, including: Using filtered ACC sample data and PPG sample data as input, and the heart rate label and motion scene label as target variables, the deep Attention network is trained.

13. The method according to claim 12, characterized in that, The filtering process is used to filter out noise data outside the [0.7Hz, 4Hz] range in the spectrum.

14. The method according to any one of claims 7 to 11, characterized in that, After obtaining the first prediction model, the method further includes: The first prediction model is quantized using preset quantization parameters to obtain the quantized first prediction model.

15. A wearable device, characterized in that, The device includes a processor coupled to a memory, the processor being configured to execute a computer program stored in the memory to enable the wearable device to perform the method as described in any one of claims 1 to 14.

16. A chip, characterized in that, The chip is coupled to a memory, and the chip is used to read and execute a computer program stored in the memory to implement the method as described in any one of claims 1 to 14.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a wearable device, causes the wearable device to perform the method as described in any one of claims 1 to 14.

Images