Heart rate detection method and electronic device

Abstract

The present application provides a heart rate detection method and electronic device, which relate to the field of communication technology. Through this solution, when it is detected that a user wearing a smart wearable device is performing a first exercise, a PPG signal is collected by a PPG sensor in the smart wearable device; based on the PPG signal and the first deep sequence neural network model, the first heart rate data is obtained; based on the PPG signal and the first frequency tracking algorithm model, the second heart rate data is obtained; based on the first heart rate data and the second heart rate data, data fusion is performed to obtain the user's target heart rate value during the first exercise. Since the frequency tracking algorithm can quickly track changes in heart rate, it can compensate for scenarios where the deep sequence neural network cannot track in time when the heart rate suddenly changes. Therefore, the present application solution can improve the accuracy of heart rate prediction by fusing heart rate data obtained by different models.

Description

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on November 1, 2021, with application number 202111284126.2 and application name “Heart rate prediction method and electronic device based on AI dual algorithm engine decision-making”, the entire contents of which are incorporated by reference in this application. Technical Field

[0002] The present application relates to the field of communication technology, and in particular to a heart rate detection method and electronic equipment. Background Art

[0003] Heart rate is an important indicator of health status. The traditional measurement method is based on electrocardiograph (ECG) heart rate analysis, which requires professional equipment and knowledge and cannot meet the daily measurement needs of users. In recent years, with the development and maturity of smart wearable devices (such as smart wristband devices), heart rate detection solutions based on smart wearable devices have become mainstream.

[0004] At present, smart wearable devices can use photoplethysmography (PPG) signals to calculate heart rate. The PPG sensor contains a light emitting diode (LED) lamp and a photosensor. The LED lamp continuously projects light onto the skin, and the light is absorbed by the blood flow through the skin tissue. At the same time, the photosensor receives the reflected light signal. Since the intensity of the reflected light is related to the blood flow velocity, and the blood flow velocity is affected by the periodic heart rhythm, the PPG signal can reflect the heart rate information.

[0005] However, when collecting PPG heart rate signals, a large amount of noise is introduced due to factors such as ambient light, baseline drift, and motion artifacts, causing signal distortion and resulting in low heart rate detection accuracy. Summary of the invention

[0006] In response to the above technical problems, the present application provides a heart rate detection method and electronic device to solve the problem of low heart rate detection accuracy caused by signal distortion due to a large amount of noise introduced by factors such as ambient light, baseline drift, motion artifacts, etc.

[0007] In a first aspect, the present application provides a heart rate detection method, the method comprising:

[0008] Determining that a user wearing a smart wearable device performs a first exercise, wherein a photoplethysmography (PPG) sensor is provided in the smart wearable device, wherein the PPG sensor is used to collect a PPG signal;

[0009] Based on the PPG signal and the first deep sequence neural network model, obtain first heart rate data;

[0010] Obtaining second heart rate data based on the PPG signal and the first frequency tracking algorithm model;

[0011] Data fusion is performed based on the first heart rate data and the second heart rate data to obtain a target heart rate value of the user during the first exercise.

[0012] Through the solution of this application, when it is detected that a user wearing a smart wearable device is performing a first exercise, a PPG signal is collected through the PPG sensor in the smart wearable device; based on the PPG signal and the first deep sequence neural network model, the first heart rate data is obtained; based on the PPG signal and the first frequency tracking algorithm model, the second heart rate data is obtained; based on the first heart rate data and the second heart rate data, data fusion is performed to obtain the user's target heart rate value during the first exercise. Since the frequency tracking algorithm can quickly track changes in heart rate, it can compensate for scenarios where the deep sequence neural network cannot track in time when the heart rate suddenly changes. Therefore, the solution of this application can improve the accuracy of heart rate prediction by fusing heart rate data obtained by different models.

[0013] In some possible implementations, the step of fusing the first heart rate data and the second heart rate data to obtain a target heart rate value of the user during the first exercise includes:

[0014] The target heart rate value is obtained by performing a weighted summation of the first heart rate data and the second heart rate data using the following equation:

[0015] Y=w1x1+w2x2,

[0016] Among them, x1 represents the first heart rate data, w1 represents the weight value corresponding to the first heart rate data, x2 represents the second heart rate data, and w2 represents the weight value corresponding to the second heart rate data.

[0017] In some possible implementations, before performing weighted summation on the first heart rate data and the second heart rate data, the method further includes:

[0018] Obtaining first sample data of the first deep sequence neural network and the first frequency tracking algorithm model respectively in different motion scenes;

[0019] Based on the following equation, the Bayesian algorithm is used to perform deep learning on the first sample data:

[0020] Y=w1x1+w2x2

[0021] The values ​​of w1 and w2 in different scenarios are obtained through training.

[0022] In some possible implementations, w1+w2=1.

[0023] In some possible implementations, before performing data fusion based on the first heart rate data and the second heart rate data, the method further includes:

[0024] Determining whether the first motion is a first type of motion, where the first type of motion is a type of motion pre-set according to the characteristics of the frequency tracking algorithm model;

[0025] When the first motion is the first type of motion, determining whether the confidence level corresponding to the PPG signal is greater than a credible threshold;

[0026] When the confidence level corresponding to the PPG signal is greater than the credible threshold, the weight value w2 corresponding to the second heart rate data is increased to w2'.

[0027] In some possible implementations, the performing data fusion based on the first heart rate data and the second heart rate data to obtain a target heart rate value of the user during the first exercise includes:

[0028] When the first exercise is the first type of exercise, the first heart rate data and the second heart rate data are weightedly summed using the following equation to obtain the target heart rate value: Y=w1x1+w2'x2; or,

[0029] When the first exercise is not the first type of exercise, the first heart rate data and the second heart rate data are weightedly summed using the following equation to obtain the target heart rate value: Y=w1x1+w2x2.

[0030] In some possible implementations, the first type of exercise is walking or cycling.

[0031] In some possible implementations, before obtaining the first heart rate data based on the PPG signal and the first deep sequence neural network model, the method further includes:

[0032] The deep sequence neural network model is trained based on the data features provided by the first frequency tracking algorithm model to obtain the first deep sequence neural network model.

[0033] The data features include PPG main frequency data and acceleration ACC main frequency data.

[0034] In some possible implementations, the method further includes:

[0035] Based on the PPG signal and the time domain interval algorithm model, obtaining third heart rate data;

[0036] Data fusion is performed based on the first heart rate data, the second heart rate data, and the third heart rate data to obtain a target heart rate value of the user during the first exercise.

[0037] In some possible implementations, the performing data fusion based on the first heart rate data, the second heart rate data, and the third heart rate data to obtain a target heart rate value of the user during the first exercise includes:

[0038] The target heart rate value is obtained by weighted summing the first heart rate data, the second heart rate data and the third heart rate data using the following equation:

[0039] Y=w1x1+w2x2+w3x3,

[0040] Among them, x3 represents the third heart rate data, and w3 represents the weight value corresponding to the third heart rate data.

[0041] In some possible implementations, before performing weighted summation on the first heart rate data, the second heart rate data, and the third heart rate data, the method further includes:

[0042] Obtain second sample data of the first deep sequence neural network, the first frequency tracking algorithm model, and the time domain interval algorithm model in different motion scenes;

[0043] Based on the following equation, the Bayesian algorithm is used to perform deep learning on the second sample data:

[0044] Y=w1x1+w2x2+w3x3,

[0045] The values ​​of w1, w2 and w3 in different scenarios are obtained through training.

[0046] In some possible implementations, w1+w2+w3=1.

[0047] In some possible implementations, before performing data fusion based on the first heart rate data, the second heart rate data, and the third heart rate data, the method further includes:

[0048] Determining whether the first motion is a second type of motion, where the second type of motion is a type of motion pre-set according to the characteristics of the time domain interval algorithm model;

[0049] When the first motion is the second type of motion, determining whether the autocorrelation coefficient of the PPG time domain signal is greater than a coefficient threshold;

[0050] When the autocorrelation coefficient of the PPG time domain signal is greater than the coefficient threshold, the weight value w3 corresponding to the third heart rate data is increased to w3'.

[0051] In some possible implementations, the performing data fusion based on the first heart rate data, the second heart rate data, and the third heart rate data to obtain a target heart rate value of the user during the first exercise includes:

[0052] When the first exercise is the second type of exercise, the target heart rate value is obtained by weighted summing using the following equation: Y=w1x1+w2x2+w3'x3; or,

[0053] When the first exercise is not the second type of exercise, the target heart rate value is obtained by weighted summing using the following equation: Y=w1x1+w2x2+w3x3.

[0054] In some possible implementations, the second type of exercise is resting or sleeping.

[0055] In some possible implementations, before obtaining the first heart rate data based on the PPG signal and the first deep sequence neural network model, the method further includes:

[0056] Based on the first data feature provided by the first frequency tracking algorithm model and the second data feature provided by the time domain interval algorithm model, the deep sequence neural network model is trained to obtain the first deep sequence neural network model.

[0057] Among them, the first data features include PPG main frequency data and ACC main frequency data, and the second data features include interval duration and the number of peak points.

[0058] In some possible implementations, after obtaining the target heart rate value of the user during the first exercise, the method further includes:

[0059] The target heart rate value and information about the first exercise are displayed on a screen of the smart wearable device.

[0060] In some possible implementations, the method further includes:

[0061] Acquire a multi-scene sample set, where the multi-scene sample set is a data sample set obtained based on detection of multiple motion scenes;

[0062] Extracting acceleration ACC sample data, photoplethysmography PPG sample data and heart rate labels from the multi-scenario sample set;

[0063] Taking the ACC sample data and the PPG sample data as input, and the heart rate label and the motion scene label as target variables, training is performed through a deep sequence neural network;

[0064] The first deep sequence neural network is obtained, wherein the first deep sequence neural network has scene recognition and heart rate prediction functions.

[0065] In a second aspect, the present application provides a heart rate detection device based on AI dual algorithm engine decision making, the device comprising a unit for executing the method in the first aspect above. The device may correspond to executing the method described in the first aspect above, and the relevant description of the unit in the device can refer to the description of the first aspect above, which will not be repeated here for the sake of brevity.

[0066] The method described in the first aspect above can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions. For example, a processing module or unit, a display module or unit, etc.

[0067] In a third aspect, the present application provides an electronic device, comprising a processor, the processor being coupled to a memory, the memory being used to store computer programs or instructions, and the processor being used to execute the computer programs or instructions stored in the memory, so that the method in the first aspect is executed.

[0068] For example, the processor is used to execute the computer program or instructions stored in the memory, so that the device performs the method in the first aspect.

[0069] In a fourth aspect, the present application provides a computer-readable storage medium having stored thereon a computer program (also referred to as instructions or codes) for implementing the method in the first aspect.

[0070] For example, when the computer program is executed by a computer, the computer can perform the method in the first aspect.

[0071] In a fifth aspect, the present application provides a chip, including a processor. The processor is used to read and execute a computer program stored in a memory to perform the method in the first aspect and any possible implementation thereof.

[0072] Optionally, the chip also includes a memory, and the memory is connected to the processor via a circuit or wire.

[0073] In a sixth aspect, the present application provides a chip system, including a processor. The processor is used to read and execute a computer program stored in a memory to perform the method in the first aspect and any possible implementation thereof.

[0074] Optionally, the chip system also includes a memory, and the memory is connected to the processor via a circuit or wire.

[0075] In a seventh aspect, the present application provides a computer program product, comprising a computer program (also referred to as instructions or codes), wherein when the computer program is executed by a computer, the computer implements the method in the first aspect.

[0076] It can be understood that the beneficial effects of the second to seventh aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 A schematic diagram of an application scenario involved in an embodiment of the present application;

[0078] Figure 2 A schematic diagram of the structure of a smart wearable device provided in an embodiment of the present application;

[0079] Figure 3 A schematic diagram of the PPG heart rate detection principle provided in an embodiment of the present application;

[0080] Figure 4 A schematic diagram showing a large amount of noise introduced due to factors such as baseline drift and motion artifacts involved in an embodiment of the present application is shown;

[0081] Figure 5 A schematic diagram of PPG time domain and frequency domain signals in a resting state and a moving state in an embodiment of the present application is shown;

[0082] Figure 6 A schematic diagram of the structure of a PPG sensor in an embodiment of the present application is shown;

[0083] Figure 7 A schematic diagram of the process of a deep sequence neural network heart rate algorithm in an embodiment of the present application is shown;

[0084] Figure 8 A schematic diagram of the flow of the frequency tracking algorithm in an embodiment of the present application is shown;

[0085] Fig. 9 A schematic diagram of the process of the heart rate detection algorithm provided in the embodiment of the present application is shown;

[0086] Fig.10 A schematic diagram of the process of performing feature fusion by using a deep sequence neural network heart rate algorithm and a frequency tracking algorithm in an embodiment of the present application is shown;

[0087] Fig.11 A schematic diagram of a process for performing decision fusion using a deep sequence neural network heart rate algorithm and a frequency tracking algorithm in one embodiment of the present application is shown;

[0088] Fig.12A schematic diagram of a process of performing scene selection-based fusion using a deep sequence neural network heart rate algorithm and a frequency tracking algorithm in one embodiment of the present application is shown;

[0089] Fig.13 A schematic diagram of a process for determining a confidence level according to a PPG signal in an embodiment of the present application is shown;

[0090] Fig.14 A schematic diagram of a process for performing feature fusion using a deep sequence neural network heart rate algorithm and a frequency tracking algorithm in another embodiment of the present application is shown;

[0091] Fig.15 A schematic diagram of a process for performing decision fusion using a deep sequence neural network heart rate algorithm and a frequency tracking algorithm in another embodiment of the present application is shown;

[0092] Fig.16 A schematic diagram of a process of performing scene selection-based fusion by using a deep sequence neural network heart rate algorithm and a frequency tracking algorithm in another embodiment of the present application is shown;

[0093] Fig.17 A schematic diagram showing the benefits of fusing a deep sequence neural network heart rate algorithm with a frequency tracking algorithm provided in an embodiment of the present application is shown;

[0094] Fig.18 A schematic diagram showing the effect of the fusion solution provided by the present application when applied to a running scene;

[0095] Fig.19 A schematic diagram of a user interface when the present application solution is applied to a smart wearable device is shown;

[0096] Fig. 20 A schematic diagram of the software architecture involved in the technical solution provided in the embodiment of the present application is shown;

[0097] Fig.21 A structural schematic diagram of a heart rate detection device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0098] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0099] The term "and / or" in this article is a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " in this article indicates that the associated objects are in an or relationship, for example, A / B means A or B.

[0100] The terms "first" and "second" in the specification and claims herein are used to distinguish different objects rather than to describe a specific order of the objects. For example, the first heart rate data and the second heart rate data are used to distinguish different heart rate data rather than to describe a specific order of the heart rate data.

[0101] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0102] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. For example, multiple processing units refer to two or more processing units, etc.; multiple elements refer to two or more elements, etc.

[0103] To facilitate understanding of the embodiments of the present application, some terms used in the embodiments of the present application are explained below to facilitate understanding by those skilled in the art.

[0104] Photoplethysmography (PPG): PPG is a non-invasive detection technology that uses optical principles to convert human biological signals into electrical signals. Specifically, the light from a light emitting diode (LED) is directed toward the skin, and a photosensitive sensor is used to receive the light that is reflected or transmitted after being absorbed by the skin tissue. The electrical signal obtained by the photosensitive sensor is converted into a digital signal to obtain a PPG signal. Due to its advantages such as being non-invasive, simple, and portable, it is widely used in health monitoring of the human body's physiological heart rate, blood oxygen, pressure, etc. Among them, heart rate is one of the parameters to measure the heart's beating ability, and accurate detection of heart rate has important medical significance.

[0105] Figure 1 Schematic diagram of system architecture involved in various exemplary embodiments of the present application is shown. Figure 1As shown, the system architecture includes a smart wearable device 1, which has a PPG sensor. When a user wears the smart wearable device 1 and the smart wearable device is turned on, the smart wearable device 1 can use PPG technology to detect the user's heart rate value at each moment, and the smart wearable device can display the heart rate value. Among them, the heart rate refers to the number of heartbeats per minute of a normal person in a resting state, also called the resting heart rate, which is generally 60 to 100 beats per minute (bpm).

[0106] It can be understood that the smart wearable device 1 can be any electronic device capable of performing PPG detection on the user, such as a smart bracelet, a smart watch, glasses, a helmet, a headband, and other wearable electronic devices that support heart rate detection. For ease of explanation, the following description of the smart wearable device 1 is given by taking a smart watch as an example.

[0107] Figure 2 The hardware structure diagram of a smart wearable device provided in the embodiment of the present application is shown in FIG. Figure 2 As shown, the smart wearable device 200 may include a touch screen 201, a display screen 202, a processor 203, a micro control unit (MCU) 204, a heart rate detection sensor 205, a gravity acceleration (acceleration, ACC) sensor 206, a memory 207, a microphone (microphone, MIC) 208, a Bluetooth (blue tooth, BT) chip 209, a power management system 210, a power supply 211, etc. The following is an introduction to each functional component of the smart watch:

[0108] The touch screen 201, also known as the touch panel, can collect touch operations performed by the watch user thereon (such as operations performed by the user using fingers, stylus, or any other suitable object or accessory on or near the touch panel) and drive a corresponding connected device according to a pre-set program.

[0109] The display screen 202 can be used to display information input by the user or information provided to the user and various menus of the watch. Further, the touch screen 201 can cover the display screen 202. When the touch screen 201 detects a touch operation on or near it, it is transmitted to the processor 203 to determine the type of touch event, and then the processor 203 provides corresponding visual output on the display screen 202 according to the type of touch event.

[0110] The processor 203 is used for system scheduling, can control the display screen, touch screen and MCU 204, and supports the control of microphone 208 and Bluetooth chip 209, etc. The processor 203 can be connected to the memory 207. The memory 207 can be used to store program code and data. When the smart wearable device 200 is running, the processor 203 can execute the computer execution instructions in the memory 207 to perform the operation steps of the above method.

[0111] It should be understood that in the embodiment of the present application, the processor 203 can adopt a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. Or the processor 203 uses one or more integrated circuits to execute related programs to implement the technical solutions provided in the embodiment of the present application.

[0112] The micro control unit 204 is used to control sensors such as the heart rate detection sensor 205 and the ACC sensor 206, perform operations on sensor data, and communicate with the processor 203. For example, the micro control unit 204 receives instructions from the processor 203, or the micro control unit 204 feeds back information to the processor 203.

[0113] The micro control unit 204 can also control the lighting and extinguishing of each light source by controlling the driver of each light source (LED) in the heart rate detection sensor 205. At the same time, the micro control unit 204 can also receive signals from the heart rate detection sensor 205 and send signals to it.

[0114] Among them, the heart rate detection sensor 205 can be a PPG sensor, and the PPG sensor can detect and obtain a PPG signal, and the PPG signal can reflect the user's heart rate data. The PPG sensor may include multiple light sources and photoelectric sensors corresponding to the light sources to achieve PPG detection. Among them, the light source can be a green light source, a red light source, or an infrared light source, etc. Exemplarily, the PPG sensor may include an LED 2051 and a photosensor 2052. Specifically, the light of LED 2051 is directed toward the skin, and the photosensor 2052 receives the light reflected back or transmitted after being absorbed by the skin tissue, and then converts the electrical signal obtained by the photosensor into a digital signal, that is, obtaining a PPG signal. Optionally, the photosensor 2052 may be a photodiode (PD), or other photoelectric sensing elements may be used.

[0115] The ACC sensor 206 is used to receive signals sent by the micro control unit 204, and can also send signals to the micro control unit 204. The signal received by the ACC sensor 206 may include raw sensor data and / or filtered or processed sensor data. In an embodiment of the present application, the ACC sensor 206 can be used to sense the user's motion state, and then it can be known whether the user is in a quiet state when wearing the smart wearable electronic device. At the same time, the user's motion information can also be obtained. The heart rate value in the motion state can be detected based on the PPG signal and the ACC signal.

[0116] The memory 207 can be used to store the instructions executed by the microcontroller unit 204 and the intermediate data generated when executing the instructions, and can be used to store the detection data detected by the PPG sensor, accelerometer, etc. In addition, in some embodiments, the memory 105 can also be used to store the correspondence between the light source information of each light source and the identification of the tested user, wherein the light source information can include the light intensity and the light intensity sensing range of the light source, and the identification of the tested user can be used to indicate different tested users.

[0117] In addition, in some embodiments, the memory 207 may also store the personal information of the tested user (such as gender, age, etc.) for the first time and store the user's atrial fibrillation load data, heart rate and exercise information in the history records of the smart wearable device (such as a wristband). The memory 207 may also store information on the device type.

[0118] Optionally, the memory 207 may include a read-only memory and a random access memory, and provide instructions and data to the processor 203. A portion of the processor 203 may also include a nonvolatile random access memory.

[0119] The memory 207 is used to store software programs and data. The processor 203 executes various functional applications and data processing of the smart watch by running the software programs and data stored in the memory 207.

[0120] Microphone 208 is also called a microphone. Microphone 206 can convert the collected sound signals into electrical signals, which are then received by the audio circuit and converted into audio data; the audio circuit can also convert the audio data into electrical signals, which are then transmitted to the speaker and converted into sound signals for output.

[0121] Bluetooth chip 209, the smart watch can exchange information with other electronic devices (such as mobile phones, tablet computers, etc.) through the Bluetooth chip 209, and connect to the server after connecting to the network through the above electronic devices.

[0122] The power source 211 (such as a battery) can supply power to each component. Optionally, the power source 211 can be logically connected to the processor 203 through the power management system 210, so that the power management system 213 can manage charging, discharging, power consumption and other functions.

[0123] Optionally, the smart wearable device 200 may also include a user interface. The user interface is used to exchange information between the system and the user, and can realize user registration and login. Usually, the user interface refers to a software interface, which may include a command interface, a program interface, and a graphical interface. That is, software developed based on the hardware device interface of human-computer interaction.

[0124] Optionally, the smart wearable device 200 may also include a wireless communication module. The wireless communication module may generally include one or more modules to support wireless communication between devices. For example, the wireless communication module may include a wireless fidelity (Wi-Fi) module and a near field communication (NFC) module.

[0125] It should be noted that the hardware functional components of the above-mentioned smart wearable device can be changed according to user needs. It can be understood that the specific embodiment introduced above is only a specific implementation method of the present application. Other ways to implement the solution of the present application are also within the scope of protection of the present application and will not be elaborated here.

[0126] Optionally, the above system architecture may also include an electronic device 2, which can establish a wireless connection with the smart wearable device 1 and perform data communication. For example, the smart wearable device 1 can match with the electronic device 2 through its own Bluetooth, and after the match is successful, it can communicate data with the electronic device 2 through the Bluetooth communication link. Of course, data communication with the mobile phone can also be performed through other wireless communication methods, such as radio frequency identification technology, short-range wireless communication technology, etc. The electronic device 2 can receive and display the heart rate value sent by the smart wearable device 1 for the user to view.

[0127] It is understood that the electronic device 2 may include, but is not limited to: a laptop computer, a desktop computer, a tablet computer, a smart phone, a smart wearable device, a head-mounted display, a mobile email device, a portable game console, a portable music player, a reader device, a television set having one or more processors embedded or coupled thereto, or other electronic devices capable of accessing a network. For ease of description, the following description of the electronic device 2 is given by taking a mobile phone as an example.

[0128] The specific process of controlling the PPG sensor to perform PPG measurement using the technical solution of the present application is described in detail below.

[0129] Figure 3 Schematic diagram showing the principle of PPG heart rate detection. Figure 3 As shown, the PPG sensor may include an LED light and a photosensitive sensor. When the PPG sensor is close to or in contact with human skin, the microcontroller unit will call the LED driver to trigger the LED light to emit light. The LED light continuously projects light onto the skin, and the light passes through the skin tissue (including the epidermis and dermis) and is absorbed or reflected by the blood flow. At the same time, the photosensitive sensor receives the reflected light signal and converts the light signal into an electrical signal, thereby detecting the reflected light intensity change signal caused by the movement of blood flow in the dermal artery, such as Figure 3 The light intensity data graph of the light intensity data changing with time t in FIG. Among them, the absorption of light by bones, veins and other tissues is basically unchanged. Since the intensity of the reflected light is related to the blood flow velocity, and the blood flow velocity is affected by the periodic heart rhythm, that is, the collected signal is a PPG periodic signal corresponding to the heart rhythm, the PPG signal can reflect the heart rate information. Among them, the PPG signal can also be called a PPG heart rate signal.

[0130] Furthermore, the PPG signal can be amplified by an amplifier, and the signal waveform can be adjusted by a signal adjuster, and the adjusted PPG signal is sent to the MCU. The MCU processes the adjusted PPG signal to obtain Figure 3 The PPG signal data graph shown changes within a certain time period T.

[0131] Currently, a large amount of noise is introduced due to factors such as ambient light, baseline drift, motion artifacts, etc., resulting in distortion of the PPG signal.

[0132] On the one hand, when being worn, the smart bracelet may be displaced from the skin. At this time, when collecting the PPG signal, it will be collected together with the external ambient light. The ambient light, as a noise signal, will affect the accuracy of the PPG signal and cause inaccurate heart rate detection.

[0133] On the other hand, baseline drift refers to the noise with a frequency below 1Hz caused by the relative friction between human breathing and the skin surface and the PPG sensor. The PPG signal affected by baseline drift can be regarded as the result of the superposition of the characteristic waveform and the baseline drift signal. By performing appropriate filtering, the baseline drift signal can be separated. Figure 4 The middle dotted box shows the intensity characteristic waveform affected by the baseline drift signal. Affected by the baseline drift, the intensity characteristic waveform changes slowly, which will affect the waveform trend over a long period of time.

[0134] On the other hand, motion artifacts refer to the high-frequency noise caused by irregular movements such as walking and running, when the human body is always in a certain state of motion. Figure 4 As shown in the figure, high-frequency noise changes quickly and is aliased in the heart rate range, affecting the waveform changes in a short period of time. Motion noise is aliased in the heart rate frequency domain and is difficult to distinguish.

[0135] Under ideal circumstances, the PPG signal can accurately detect the heart rate value of the human body at any time. However, due to the influence of noise from complex scenes such as ambient light, baseline drift, and motion artifacts, the real PPG signal is distorted, which greatly restricts the accuracy of heart rate calculation. Among them, the most influential is motion artifacts, which will cause the absence or deformation of peaks and troughs in the PPG time domain signal, reducing the accuracy of the time domain counting method, and appearing as abnormal PPG spectrum peaks in the frequency domain. It is impossible to accurately locate the main frequency of the PPG signal at the current moment, and it is difficult to obtain accurate results. Therefore, how to adaptively eliminate complex noise in the PPG signal and improve the accuracy of heart rate detection is an important issue.

[0136] Figure 5 (a) shows the PPG time domain signal and frequency domain signal diagram in the resting state. Figure 5 (b) shows the PPG time domain signal and frequency domain signal diagram in motion state.

[0137] In the PPG time domain signal curve graph, the ordinate represents the PPG signal strength value, and the abscissa represents the sampling time. The PPG time domain signal curve graph is a time domain graph after bandpass filtering.

[0138] In the PPG frequency domain signal spectrum diagram, the ordinate represents the FFT amplitude value, and the abscissa represents the number of samples. The PPG frequency domain signal spectrum diagram is a spectrum diagram after bandpass filtering. The FFT amplitude value represented by the ordinate is normalized and has a value range of [0, 20]. The number of samples represented by the abscissa can be 35.

[0139] Assuming the sampling frequency is 25 Hz, the corresponding FFT abscissa unit interval resolution is 25 / 256 = 0.097 Hz. Assuming that the FFT abscissa point range in the filtered PPG spectrum is [5, 35]. Since the FFT abscissa unit interval resolution is 0.097 Hz, the frequency interval corresponding to the FFT abscissa point range [5, 35] is [5*0.097 Hz, 35*0.097 Hz], which is approximately [0.5 Hz, 3.5 Hz]. The spectrum diagram includes spectrum data in the frequency range [0.7 Hz, 3.5 Hz], and the frequency domain data obtained at this time is considered to be valid spectrum data.

[0140] Combination Figure 5 From (a) and (b), we can see that the PPG time domain signal is stable in the resting state, and the PPG frequency domain signal has a high energy ratio. The counting method and frequency domain peak search method can accurately predict the heart rate. However, the PPG time domain signal is distorted in the exercise state, and the energy ratio of the PPG frequency domain signal is very low. This is because the motion noise is mixed in the heart rate frequency domain, which is difficult to distinguish and affects the heart rate prediction. It can be seen that the heart rate detection result in the resting state is more accurate, but due to the interference of high-frequency noise in motion, the heart rate detection signal may be distorted in the exercise state, so the heart rate value detected by the traditional heart rate algorithm is not accurate enough.

[0141] At present, traditional technologies have carried out a lot of artificial noise preprocessing and rule formulation for signals, which requires a lot of domain knowledge as support, and can only achieve good results under specific parameters and specific scenarios. At present, it is impossible to find a technical solution to adaptively eliminate noise and compensate signals under the influence of various complex motion artifacts and noise.

[0142] In view of this, the present application provides a corresponding solution: enhancing the signal quality of the PPG optical path signal source, mining the target signal through a high-precision algorithm model, and achieving high-precision prediction.

[0143] For example, Figure 6As shown, the smart wearable device in the embodiment of the present application adopts an 8-channel PPG heart rate module. Among them, the 8-channel PPG heart rate module includes photosensor A1, photosensor A2, ... photosensor A8, a total of 8 photosensors. These 8 photosensors are distributed around one or more light sources, and two light sources B1 and B2 are used as examples for illustration. Optionally, light source B1 can be a green light source and / or a blue light source, and light source B2 can be a red light source and / or an infrared light source. For example, light source B1 can be a blue LED, and the second light source can be a red LED. In this way, the two light sources project light onto the skin, and the light is absorbed or reflected by the blood flow through the skin tissue. The 8 photosensors around the light source receive the reflected light signal and convert the light signal into an electrical signal, so that an 8-channel PPG signal can be obtained, which can enhance the signal quality of the PPG optical path signal source.

[0144] Among them, the smart wearable device can be a smart watch. By adopting an 8-channel PPG heart rate module, 8-channel data collection, 4-channel data processing, and AI fusion can be used to form a high-quality heart rate signal, which can effectively eliminate heart rate signal noise and greatly enhance anti-interference ability. Therefore, it can reduce the risk of unstable heart rate signals and improve the accuracy of dynamic heart rate.

[0145] Through the present application, when it is detected that a user wearing a smart wearable device is performing a first exercise, a PPG signal can be collected through the PPG sensor in the smart wearable device; the PPG signal is input into the first deep sequence neural network model to obtain the first heart rate data; the PPG signal is input into the first frequency tracking algorithm model to obtain the second heart rate data; the first heart rate data and the second heart rate data are fused to obtain the user's target heart rate value during the first exercise. Since the frequency tracking algorithm can quickly track changes in heart rate, it can compensate for scenarios that the deep sequence neural network cannot track in time when the heart rate suddenly changes. Therefore, the present application can improve the accuracy of heart rate prediction by fusing heart rate data obtained by different models.

[0146] The heart rate detection method provided in the embodiments of the present application is exemplarily described below in combination with the first embodiment and the second embodiment.

[0147] First embodiment

[0148] In a solution provided in an embodiment of the present application, a deep sequence neural network heart rate algorithm and a harmonic frequency tracking (FT) algorithm can be used simultaneously to process PPG signals collected in different scenarios to enhance the signal quality of the PPG optical path signal source. In this way, the target signal can be mined through a high-precision algorithm model to achieve high-precision prediction.

[0149] For example, the above Figure 6 The 8-channel PPG signal shown can be used for heart rate signal AI fusion using a deep sequence neural network heart rate algorithm and a frequency tracking algorithm. This allows the target signal to be mined through a highly accurate algorithm model to achieve high-precision prediction, thereby greatly improving the heart rate accuracy.

[0150] First, we introduce the deep sequence neural network heart rate algorithm. Figure 7 FIG. 4 shows a schematic structural block diagram of predicting heart rate using a deep sequence neural network heart rate algorithm. Figure 7 As shown, after the multi-dimensional feature data is input into the deep sequence neural network, the heart rate is predicted by the deep sequence neural network heart rate algorithm, and the target heart rate value can be output. Among them, the multi-dimensional feature data can include PPG and ACC time domain signals and PPG and ACC frequency domain signals; the deep sequence neural network can include a convolution layer, a normalization layer, a recurrent network layer, a residual layer and a fully connected layer. It can be understood that the components of the deep sequence neural network here are illustrative. In actual implementation, the deep sequence neural network can also include Figure 7 More or fewer components as shown.

[0151] On the one hand, the deep sequence neural network heart rate algorithm has the following advantages: First, the human heart rate changes are highly sequential and non-mutated, and the time series-based deep learning model can effectively model the changing patterns of heart rate; Second, the deep sequence neural network does not require a large amount of expert domain knowledge to do a large amount of data preprocessing, and can automatically learn features to achieve an end-to-end heart rate monitoring process. Third, it has strong anti-interference capabilities in the case of unstable signals or short-term noise mutations.

[0152] On the other hand, the deep sequence neural network heart rate algorithm has the following disadvantages: First, it cannot track sudden changes in heart rate (such as when a user wearing a smart wearable device stands up quickly). Second, it requires a large amount of training data.

[0153] Secondly, the harmonic frequency tracking algorithm (or adaptive filtering algorithm) is introduced. Figure 8 FIG. 4 shows a schematic block diagram of a harmonic frequency tracking algorithm. Figure 8 As shown in the figure, the signal x[n] collected at time n is input into K time-varying filters to obtain y1[n], y2[n], ...y K [n]; then, y1[n], y2[n], …y K [n] are input into the adaptive module for parameter adjustment, and w1[n], w2[n], ...w K [n]; then, w1[n], w2[n], …w K [n] are multiplied by the weight parameters W1[n], W2[n], ...WK [n], the three product values ​​are added together to obtain the sum value. Then, the sum value is used as the input of K time-varying bandpass filters through the feedback path, and the K coefficients of the filter are adjusted cyclically, and finally the target value w[n] is output.

[0154] The above filtering method is called adaptive filtering. In the process of adaptive filtering, it is not necessary to know the statistical characteristics of the input PPG signal and noise in advance. The filter itself can learn or estimate the statistical characteristics of the signal during operation, and adjust its own parameters based on this to achieve the optimal filtering effect under a certain criterion / cost function. Once the statistical characteristics of the signal change, it can also track this change and readjust the parameters to optimize the filtering performance again. Therefore, adaptive filtering is an effective means of processing non-stationary signals.

[0155] On the one hand, the harmonic frequency tracking algorithm has the following advantages: first, the frequency tracking algorithm can quickly track changes in heart rate; second, it does not require a large amount of data support and has a strong mathematical theoretical foundation.

[0156] On the other hand, the harmonic frequency tracking algorithm has the following disadvantages: first, it requires a lot of expert knowledge and cannot achieve end-to-end heart rate detection; second, it is sensitive to noise and has weak anti-interference ability.

[0157] Based on the above analysis of the advantages and disadvantages of the deep sequence neural network heart rate algorithm and the harmonic frequency tracking algorithm, the embodiment of the present application proposes that the deep sequence neural network heart rate algorithm and the harmonic frequency tracking algorithm can be fused and applied. Specifically, the advantages of the two algorithm models can be nonlinearly deeply fused by using feature fusion and model fusion methods, and the corresponding algorithm can be called an artificial intelligence AI fusion algorithm.

[0158] According to experimental data, the embodiment of the present application can have the following advantages by integrating the deep sequence neural network with the frequency tracking algorithm: based on the frequency tracking algorithm, it can quickly track the changes in heart rate, so that the AI ​​fusion algorithm can memorize and learn the heart rate trend, and has strong anti-interference and reasoning capabilities. Moreover, the AI ​​fusion algorithm uses feature fusion and model fusion to deeply integrate the advantages of the two models in a nonlinear way, greatly improving the algorithm's adaptability and detection capabilities in various scenarios.

[0159] Fig. 9 is a flow chart of a heart rate detection method provided in an embodiment of the present application. Fig. 9 As shown, the method includes the following steps S101-S106.

[0160] S101, determining that a user wearing a smart wearable device performs a first exercise, wherein a photoplethysmography (PPG) sensor is provided in the smart wearable device.

[0161] Exemplarily, the first exercise can be walking, running, swimming or cycling, etc. Cycling can also be divided into indoor cycling and outdoor cycling. Optionally, the first exercise scene can also be resting or sleeping, etc.

[0162] For example, a resting scene may refer to a state in which the user is sitting or standing, is motionless, or has small movements.

[0163] S102, collecting PPG signals through a PPG sensor.

[0164] S103: Obtain first heart rate data based on the PPG signal and a first deep sequence neural network model.

[0165] Specifically, the PPG signal can be input into a first deep sequence neural network model to obtain first heart rate data.

[0166] Among them, the above-mentioned first deep sequence neural network model is also called a deep sequence neural network heart rate algorithm, which can be simply referred to as a deep algorithm or a deep learning algorithm.

[0167] S104: Obtain second heart rate data based on the PPG signal and the first frequency tracking algorithm model.

[0168] Specifically, the PPG signal can be input into the first frequency tracking algorithm model to obtain the second heart rate data.

[0169] The first frequency tracking algorithm model refers to a specific frequency tracking algorithm (harmonic frequency tracking algorithm). Exemplarily, the frequency tracking algorithm (abbreviated as FT algorithm) can adopt an adaptive filtering algorithm, such as an adaptive filtering algorithm based on least mean square (LMS), referred to as LMS algorithm. In the LMS algorithm, a set of weight vectors is found to minimize the expected value of the square of the difference between the sample predicted output value and the actual output value.

[0170] Among them, the LMS algorithm uses an adaptive filter, which continuously adjusts the filter parameters according to the feedback of the previous and next result information during the filtering process. Different methods of updating parameters will result in different results. You can choose the filter parameter update method with the smallest error, which is called the cost function or objective function. Since the purpose of the objective function is to optimize the parameter calculation, it does not require prior knowledge of the characteristics of the filtered signal such as the frequency, so it can be well applied to PPG signal denoising.

[0171] S105: Perform data fusion based on the first heart rate data and the second heart rate data to obtain a target heart rate value of the user in the first sports scene.

[0172] S106: Display the target heart rate value and the first exercise information on the screen of the smart wearable device.

[0173] It should be noted that the embodiment of the present application does not limit the execution order of the above S103 and S104. For example, S103 may be executed first and then S104; S104 may be executed first and then S103; S103 and S104 may also be executed simultaneously. It is understandable that Fig. 9 The simultaneous execution of S103 and S104 is taken as an example for schematic illustration.

[0174] The following is an exemplary description of the data fusion method of the deep sequence neural network heart rate algorithm and the frequency tracking algorithm. Optionally, in the embodiment of the present application, the fusion method of the deep algorithm and the FT algorithm may include feature fusion, decision fusion, and fusion based on scene selection.

[0175] Among them, the deep sequence neural network model is first trained based on the characteristics provided by the FT algorithm and data such as PPG (fusion of multiple features) to obtain the optimized deep sequence neural network.

[0176] Then, based on a large number of sample databases of different motion scenes of the optimized deep sequence neural network and frequency tracking algorithm model, the Bayesian algorithm is used to learn the parameters (decision fusion) to obtain the weight factor w1 corresponding to the deep algorithm in different scenes and the weight factor w2 corresponding to the FT algorithm.

[0177] Finally, based on the PPG signal and the optimized deep sequence neural network, the first heart rate data is obtained; at the same time, based on the PPG signal and the frequency tracking algorithm model, the second heart rate data is obtained. Based on the first heart rate data and the second heart rate data, data weighted fusion is performed to obtain the user's target heart rate value during the first exercise.

[0178] The following is a detailed description of the fusion methods of the deep algorithm and the FT algorithm.

[0179] Method 1: Feature Fusion

[0180] Fig.10 FIG. 1 shows a schematic structural block diagram of feature fusion provided by the first embodiment of the present application. Fig.10As shown, the multidimensional feature data includes PPG and ACC time domain signal statistical features (such as mean variance, etc.), PPG and ACC frequency domain signals, and FT algorithm supply features, such as intermediate result data such as PPG main frequency and ACC main frequency. The multidimensional feature data is input into the deep sequence neural network, and the heart rate value obtained by the test such as electrocardiograph (ECG) is used as the target heart rate value to perform model training. Through parameter optimization and adjustment, the optimized deep sequence neural network is trained.

[0181] Among them, when the preset heart rate detection trigger condition is met, the smart wearable device will be triggered to perform heart rate detection and obtain PPG signals and ACC signals. For example, the smart wearable device will receive a heart rate detection command, which triggers the PPG sensor to collect PPG signals and triggers the acceleration sensor to collect ACC signals, so that the smart wearable device obtains PPG signals and ACC signals.

[0182] Optionally, in the embodiment of the present application, the preset heart rate detection trigger condition may be any one of the following: the smart wearable device receives an operation triggered by the user to start the heart rate detection function, the smart wearable device detects the heart rate in real time, the smart wearable device periodically detects the heart rate and starts the Nth cycle of heart rate detection. The specific conditions may be determined according to actual use requirements and are not limited in the embodiment of the present application.

[0183] Among them, the above FT algorithm supply characteristics can be through Figure 8 The intermediate result data such as PPG main frequency and ACC main frequency obtained by the harmonic frequency tracking algorithm shown in the figure should be noted that Figure 8 The harmonic frequency tracking algorithm shown is an exemplary enumeration. In actual implementation, the embodiment of the present application may also adopt any other frequency tracking algorithm that meets actual usage requirements.

[0184] Method 2: Decision Fusion

[0185] Fig.11 FIG. 4 shows a schematic structural block diagram of the decision fusion provided by the first embodiment of the present application. Fig.11 As shown, in an embodiment of the present application, assuming that the current heart rate output of the deep algorithm is x1 and the current heart rate output of the FT algorithm is x2, then decision fusion can be performed for the deep algorithm and the FT algorithm, that is, based on a large number of sample databases of different motion scenes obtained based on the deep algorithm and the FT algorithm respectively, the parameters are learned using the Bayesian algorithm, so as to obtain the weight values ​​of the deep algorithm and the weight values ​​of the FT algorithm in different scenes through training.

[0186] In the embodiment of the present application, training is performed by the following equations (1) and (2):

[0187] Y=w1x1+w2x2; (1)

[0188] w1+w2=1. (2)

[0189] Among them, Y is the heart rate value output by training, w1 is the weight factor corresponding to the deep algorithm, and w2 is the weight factor corresponding to the FT algorithm.

[0190] It should be noted that the different sports scenes mentioned in the embodiments of the present application include, for example, walking, running, swimming and other scenes, and of course, may also include specific scenes such as resting or sleeping, which can be determined based on actual usage requirements.

[0191] The following is a schematic diagram of the optimization results for different motion scenarios obtained through training:

[0192] For example, in the sleep scene, w1=0.20, w2=0.80; wherein the weight factor w2 corresponding to the FT algorithm is greater than the weight factor w1 corresponding to the deep algorithm. In the sleep scene, the FT algorithm is mainly used, and the deep algorithm is used as a supplement to perform heart rate detection.

[0193] For example, in a walking scene, w1=0.40, w2=0.60, wherein the weight factor w2 corresponding to the FT algorithm is greater than the weight factor w1 corresponding to the deep algorithm. In a walking scene, the FT algorithm is mainly used, and the deep algorithm is used as a supplement to perform heart rate detection.

[0194] For example, in the running scene, w1=0.55, w2=0.45, wherein the weight factor w1 corresponding to the deep algorithm is greater than the weight factor w2 corresponding to the FT algorithm. In the running scene, the deep algorithm is mainly used, and the FT algorithm is used as a supplement to perform heart rate detection.

[0195] For example, in the swimming scenario, w1=0.62, w2=0.38, wherein the weight factor w1 corresponding to the deep algorithm is greater than the weight factor w2 corresponding to the FT algorithm. In the swimming scenario, the deep algorithm is mainly used, and the FT algorithm is used as a supplement to perform heart rate detection.

[0196] In the embodiment of the present application, the deep algorithm and the FT algorithm respectively use weight factors corresponding to the scenes for decision fusion in different scenarios, which can better suppress the motion artifact noise existing in different motion scenes, thereby improving the accuracy of heart rate detection.

[0197] Method 3: Fusion based on scene selection

[0198] Fig.12 FIG. 4 shows a schematic structural block diagram of the fusion based on scene selection provided by the first embodiment of the present application. Fig.12As shown, the fusion process based on scene selection may include the following steps S301-S306.

[0199] S301: The smart wearable device detects that the user starts exercising.

[0200] S302, the smart wearable device determines whether the current scene is a FT strong scene.

[0201] Optionally, the FT dominant scene may be walking, resting, sleeping, etc. It is understood that this is an exemplary enumeration, and the specific example may be determined according to actual use requirements, and the embodiment of the present application is not limited thereto.

[0202] If the current scene is a FT-dominant scene, then continue to execute S303. If the current scene is not a FT-dominant scene, then continue to execute S305.

[0203] S303: The smart wearable device determines whether the FT confidence is greater than a trust threshold T.

[0204] In the embodiment of the present application, if the FT confidence is greater than the credible threshold T, it means that the signal quality corresponding to the FT algorithm in the current scenario is better, and the weight of the FT algorithm can be increased to improve the accuracy of the fusion algorithm in detecting heart rate.

[0205] The following is a brief description of how to obtain the FT confidence based on the PPG signal. Fig.13 As shown in the figure, after the PPG signal data is collected, it is converted into an analytical signal through signal processing modules such as deburring and bandpass filtering, and the PPG signal is subjected to Hilbert transform. Then, combined with ACC notch denoising, that is, the PPG signal is notched using the ACC main frequency, the motion noise can be eliminated. The PPG signal can be divided into multiple harmonic signals (i.e., decomposed harmonics) through multiple filters. For each harmonic signal, the relative error of the PPG signal after adaptive filtering is solved to obtain the harmonic weight. Through big data analysis, the thresholds of each confidence interval can be determined to obtain the final confidence.

[0206] In the process of decomposing harmonics and calculating weights, the PPG signal can be passed through multiple filters to obtain multiple harmonics, and the relative error of each signal relative to the ideal signal is calculated and divided by the sum of the relative errors of each harmonic signal to obtain the harmonic weight.

[0207] Among them, the confidence of threshold division can be obtained according to big data statistics, that is, a large amount of sample data (for example, more than 10,000 groups of sample data) is collected, and the confidence interval thresholds are divided into four levels according to quartiles, and the weights are converted into confidence output.

[0208] If the FT confidence is greater than the credible threshold T, then continue to execute S304. If the FT confidence is less than or equal to the credible threshold T, then continue to execute S305.

[0209] Optionally, the embodiment of the present application may also use other methods such as a neural network model to confirm the confidence of the FT.

[0210] S304: The smart wearable device increases the weight factor w2 corresponding to the FT algorithm.

[0211] That is to say, when the current scenario is a FT-dominant scenario and the FT confidence is greater than the trust threshold T, the weight factor w2 corresponding to the FT algorithm can be increased.

[0212] For example, the weight factor w2 corresponding to the FT algorithm can be amplified by multiplying the weight factor w2 corresponding to the FT algorithm by an amplification coefficient, that is, the weight of the FT algorithm is increased. For the sake of distinction, the increased weight factor w2 corresponding to the FT algorithm is recorded as w2'.

[0213] Optionally, the above-mentioned amplification factor can be 1.1, or 1.3, or 1.5, or any other value that meets actual usage requirements. It can be determined according to actual usage requirements and is not limited in the embodiments of the present application.

[0214] For example, in a walking scene, the weight factor w1 corresponding to the depth algorithm is 0.4, and the weight factor w2 corresponding to the FT algorithm is 0.6. After determining that the FT confidence is greater than the trust threshold T, the weight factor w2 corresponding to the FT algorithm can be multiplied by the amplification factor 1.1, that is, the weight of the FT algorithm is increased, and the corresponding weight factor w2' is 0.66.

[0215] S305, the smart wearable device will perform decision fusion on the deep algorithm and the FT algorithm.

[0216] It should be noted that after S302 or S303, S305 performs decision fusion according to Y=w1x1+w2x2 to calculate the heart rate value of the user when performing the current exercise.

[0217] After S304, S305 performs decision fusion according to Y=w1x1+x2w2' to calculate the heart rate value of the user during the current exercise.

[0218] S306: The smart wearable device outputs the current heart rate value obtained through decision fusion.

[0219] In an embodiment of the present application, the frequency tracking algorithm and the deep sequence neural network heart rate algorithm can be fused to synthesize accurate heart rate in segments. The present application scheme takes into account the user's behavior or motion state. The present application scheme can support the detection of the user's heart rate value when it is in different motion scenes (such as various motion states, including resting state). In addition, the heart rate value of the current scene and the current scene can be detected. Through the present application scheme, continuous and real-time high-precision heart rate monitoring can be achieved even in the presence of motion noise interference.

[0220] Second embodiment

[0221] In another solution provided in an embodiment of the present application, a deep sequence neural network heart rate algorithm, a frequency tracking algorithm, and a time domain interval calculation method can be used simultaneously to process PPG signals collected in different scenarios to enhance the signal quality of the PPG optical path signal source. In this way, the target signal can be mined through a high-precision algorithm model to achieve high-precision prediction.

[0222] Among them, the time domain interval calculation method has the following advantages: it can quickly track the trend of heart rate changes under low noise conditions, has low computational complexity, and is easy to implement. Optionally, the time domain interval calculation method can adopt a PP interval calculation method, wherein the PP interval corresponds to the distance between two adjacent P wave starting points displayed on the electrocardiogram.

[0223] Similar to the fusion method described in the first embodiment above, the solution provided in the second embodiment can also use feature fusion, decision fusion and scene selection-based fusion to perform AI fusion application on the deep sequence neural network heart rate algorithm, frequency tracking algorithm and time domain interval calculation method.

[0224] Among them, the deep sequence neural network model is first trained based on the time domain feature data such as the FT algorithm supply characteristics, PP interval duration and the number of peak points, and PPG data (multiple feature fusion) to obtain the optimized deep sequence neural network.

[0225] Then, based on a large number of sample databases of different motion scenes of the optimized deep sequence neural network, frequency tracking algorithm model and time domain interval calculation method model, the Bayesian algorithm is used to learn the parameters (decision fusion) to obtain the weight factor w1 corresponding to the deep algorithm in different scenes, the weight factor w2 corresponding to the FT algorithm, and the weight factor w3 corresponding to the time domain interval calculation method.

[0226] Finally, based on the PPG signal and the optimized deep sequence neural network, the first heart rate data is obtained; based on the PPG signal and the frequency tracking algorithm model, the second heart rate data is obtained; based on the PPG signal and the time domain interval calculation model, the third heart rate data is obtained. Based on the first heart rate data, the second heart rate data and the third heart rate data, data weighted fusion is performed to obtain the user's target heart rate value during the first exercise.

[0227] The following is a detailed description of the fusion methods of the depth algorithm, FT algorithm and time domain interval calculation method.

[0228] Method 1: Feature Fusion

[0229] Fig.14 FIG. 2 shows a schematic structural block diagram of feature fusion provided by the second embodiment of the present application. Fig.14 As shown, the multi-dimensional feature data includes PPG and ACC time domain signal statistical features (such as mean variance, etc.), PPG and ACC frequency domain signals, FT algorithm supply features (such as intermediate result data such as PPG main frequency and ACC main frequency), and PP interval duration, peak point number and other feature data. The multi-dimensional feature data is input into the deep sequence neural network, and the heart rate value obtained from the test such as electrocardiogram is used as the target heart rate value to perform model training. Through parameter optimization and adjustment, the optimized deep sequence neural network is trained.

[0230] The difference from the first embodiment is that the second embodiment uses feature data of three algorithms for fusion, so the accuracy of heart rate detection is higher.

[0231] Method 2: Decision Fusion

[0232] Fig.15 FIG. 4 shows a schematic structural block diagram of the decision fusion provided by the first embodiment of the present application. Fig.15 As shown, in an embodiment of the present application, assuming that the current heart rate output of the deep algorithm is x1, the current heart rate output of the FT algorithm is x2, and the current heart rate output of the time domain interval calculation method is x3, then decision fusion can be performed for the deep algorithm and the FT algorithm, that is, based on a large number of sample databases of different motion scenes obtained by the deep algorithm and the FT algorithm respectively, the parameters are learned using the Bayesian algorithm, so as to obtain the weight values ​​of the deep algorithm and the FT algorithm in different scenes through training, as well as the weight factor values ​​corresponding to the time domain interval calculation method.

[0233] In the embodiment of the present application, training is performed by the following equations (3) and (4):

[0234] Y=w1x1+w2x2+w3x3; (3)

[0235] w1+w2+w3=1. (4)

[0236] Among them, Y is the heart rate value of the training output, w1 is the weight factor corresponding to the deep algorithm, w2 is the weight factor corresponding to the FT algorithm, and w3 is the weight factor corresponding to the time domain interval calculation method.

[0237] In the embodiment of the present application, the Bayesian algorithm can be used to learn the parameters based on a large number of sample databases of different motion scenes, so as to train the algorithm weights w1, w2 and w3 in different scenes. For the description of decision fusion in the second embodiment, please refer to the relevant description of decision fusion in the first embodiment above, which will not be repeated here.

[0238] In the embodiments of the present application, the depth algorithm, FT algorithm and time domain interval calculation method respectively use weight factors corresponding to the scenes for decision fusion in different scenarios, which can better suppress the motion artifact noise existing in different motion scenes, thereby improving the accuracy of heart rate detection.

[0239] Method 3: Fusion based on scene selection

[0240] Fig.16 FIG. 2 shows a schematic structural block diagram of the fusion based on scene selection provided by the second embodiment of the present application. Fig.16 As shown, the fusion process based on scene selection may include the following steps S401-S409.

[0241] S401: The smart wearable device detects that the user starts exercising.

[0242] S402, the smart wearable device determines whether the current scene is a FT strong scene.

[0243] Optionally, the FT strong scene may be walking, riding, etc. It can be understood that this is an exemplary enumeration, and the specific example can be determined according to actual use requirements, and the embodiment of the present application is not limited thereto.

[0244] If the current scene is a FT-dominant scene, then continue to execute S403. If the current scene is not a FT-dominant scene, then continue to execute S405.

[0245] S403: The smart wearable device determines whether the FT confidence is greater than a trust threshold T1.

[0246] In the embodiment of the present application, if the FT confidence is greater than the credible threshold T1, it means that the signal quality corresponding to the FT algorithm in the current scenario is better, and the weight of the FT algorithm can be increased to improve the accuracy of the fusion algorithm in detecting heart rate.

[0247] In the second embodiment, for the description of the FT confidence, please refer to the detailed description of the FT confidence in the first embodiment, which will not be repeated here.

[0248] If the FT confidence is greater than the credible threshold T1, then continue to execute S404. If the FT confidence is less than or equal to the credible threshold T, then continue to execute S405.

[0249] S404: The smart wearable device increases the weight factor w2 corresponding to the FT algorithm.

[0250] For example, the weight factor w2 corresponding to the FT algorithm can be amplified by multiplying the weight factor w2 corresponding to the FT algorithm by an amplification coefficient a, that is, the weight of the FT algorithm is increased. For the sake of distinction, the increased weight factor w2 corresponding to the FT algorithm is recorded as w2'.

[0251] Optionally, the amplification factor a may be 1.1, or 1.3, or 1.5, or any other value that meets actual usage requirements. The specific value may be determined based on actual usage requirements and is not limited in the embodiments of the present application.

[0252] For example, in a walking scene, assuming that the weight factor w1 corresponding to the depth algorithm is 0.4, the weight factor w2 corresponding to the FT algorithm is 0.5, and the weight factor w3 corresponding to the time domain interval calculation method is 0.1, after judging that the FT confidence is greater than the credible threshold T, the weight factor w2 corresponding to the FT algorithm can be multiplied by the amplification factor 1.3, that is, the weight of the FT algorithm is increased, and the corresponding weight factor w2' is 0.65.

[0253] S405, the smart wearable device determines whether the current scene belongs to a strong scene of the time domain interval calculation method.

[0254] Exemplarily, the dominant scenario of the time domain interval calculation method may be a resting or sleeping state, etc.

[0255] If the current scene is a strong scene of the time domain interval calculation method, then continue to execute S406. If the current scene is not a strong scene of the time domain interval calculation method, then continue to execute S408.

[0256] S406: The smart wearable device determines whether the autocorrelation coefficient of the PPG time domain signal is greater than a threshold value T2.

[0257] Among them, the autocorrelation coefficient calculation can be used to evaluate the quality of the periodicity of the current PPG time domain signal.

[0258] It should be noted that the autocorrelation coefficient measures the correlation between the same event in two different periods, that is, it measures the impact of an object's past behavior on the object's present. The larger the autocorrelation coefficient, the more obvious the correlation.

[0259] For example, for two different sequences of the same length, covariance can be used to characterize their correlation. The larger the covariance value, the stronger the correlation between the two sequences.

[0260] The threshold T2 may be an autocorrelation coefficient determined based on a large amount of experimental data.

[0261] In an embodiment of the present application, if the autocorrelation coefficient of the time domain signal is greater than the threshold T2, it means that the signal quality corresponding to the domain interval calculation method in the current scenario is better, and the weight of the domain interval calculation method can be increased to improve the accuracy of the fusion algorithm in detecting heart rate.

[0262] If the autocorrelation coefficient of the time domain signal is greater than the threshold value T2, then continue to execute S407. If the autocorrelation coefficient of the time domain signal is less than or equal to the threshold value T2, then continue to execute S408.

[0263] S407, the smart wearable device increases the weight factor w3 corresponding to the time domain interval calculation method.

[0264] That is to say, when the current scene is a strong scene of the time domain interval calculation method and the autocorrelation coefficient of the time domain signal is greater than the threshold T2, the weight factor w3 corresponding to the time domain interval calculation method can be increased.

[0265] Exemplarily, the weight factor w3 corresponding to the time domain interval calculation method can be amplified by multiplying the weight factor w3 corresponding to the time domain interval calculation method by an amplification coefficient b, that is, increasing the weight of the time domain interval calculation method. For easy distinction, the enlarged weight factor w3 corresponding to the time domain interval calculation method is recorded as w3'.

[0266] Optionally, the above-mentioned amplification factor b can be 1.1, or 1.3, or 1.5, or any other value that meets actual usage requirements. It can be determined according to actual usage requirements and is not limited in the embodiments of the present application.

[0267] Optionally, the amplification factor a and the amplification factor b may be the same or different. Specifically, they may be determined according to actual use requirements and are not limited in the embodiments of the present application. For example, the amplification factor a may be 1.3 and the amplification factor b may be 1.1.

[0268] Exemplarily, in a sleep scenario, assuming that the weight factor w1 corresponding to the depth algorithm is 0.2, the weight factor w2 corresponding to the FT algorithm is 0.7, and the weight factor w3 corresponding to the time domain interval calculation method is 0.1, after determining that the autocorrelation coefficient of the time domain signal is greater than the threshold T2, the weight factor w3 corresponding to the time domain interval calculation method can be multiplied by the amplification factor 1.1, that is, the weight of the FT algorithm is increased, and the corresponding weight factor w3' is 0.11.

[0269] S408, the smart wearable device will perform decision fusion on the deep algorithm, FT algorithm and time domain interval calculation method.

[0270] It should be noted that after S405 (No) or S406 (No), the above S408 performs decision fusion according to Y=w1x1+w2x2+w3x3 to calculate the heart rate value of the user when performing the current exercise.

[0271] After S404, the above S408 can perform decision fusion according to Y=w1x1+w2x2'+w3x3 to calculate the heart rate value of the user when performing the current exercise.

[0272] After S407, the above S408 can perform decision fusion according to Y=w1x1+x2w2+x3w3' to calculate the heart rate value of the user when performing the current exercise.

[0273] S409, the smart wearable device outputs the current heart rate value obtained through decision fusion.

[0274] This application scheme takes into account the user's behavior or motion state. This application scheme can support the detection of the user's heart rate value when in different motion scenes (such as various motion states, of course including the resting state). In addition, the current scene and the heart rate value under the current scene can be detected. Through this application scheme, continuous and real-time high-precision heart rate monitoring can be achieved even in the presence of motion noise interference.

[0275] It can be seen from experimental data that the embodiment of the present application integrates the deep sequence neural network with the frequency tracking algorithm, and uses feature fusion and model fusion methods to perform nonlinear deep fusion of the advantages of the three algorithm models, thereby greatly improving the algorithm's adaptability and detection capabilities in various scenarios.

[0276] Fig.17 The schematic diagram of the benefits of combining the deep sequence neural network heart rate algorithm with the frequency tracking algorithm (FT algorithm) provided in the embodiment of the present application is shown. Fig.17 As shown in the figure, after adding the intermediate feature data of FT, the accuracy of the deep model increased by 1%, the accuracy of the deep model after fusion continued to increase by 1%, and the convergence speed of the neural network also increased accordingly (for example, 30%).

[0277] Fig.18 The schematic diagram shows the effect of the fusion solution provided by the present application when applied to a running scene. Fig.18As shown, the solid line is the heart rate curve of the calibration product (such as electrocardiogram detection), which can be used as the standard curve; the dotted line is the heart rate curve detected by the present application scheme. It can be seen that the heart rate curve detected by the present application scheme basically fits the standard curve, that is, the present application scheme can greatly improve the accuracy of heart rate detection of smart wearable devices.

[0278] The technical solution provided in the embodiment of the present application is applied to the heart rate evaluation report in the walking scene. By comparing the standard heart rate curve of the calibration product 1 (for example, electrocardiogram detection) and the heart rate curve detected by the product 2 of the present application, it can be seen that the heart rate curve detected by the product 2 of the present application basically fits the standard heart rate curve, that is, the heart rate detection accuracy of the smart wearable device in the walking scene can be greatly improved by the product 2 of the present application.

[0279] In the heart rate evaluation report when the solution provided by the embodiment of the present application is applied to a running scenario, by comparing the standard heart rate curve of the calibration product 1 (for example, electrocardiogram detection) and the heart rate curve detected by the product 2 of the present application, it can be seen that the heart rate curve detected by the product 2 of the present application basically fits the standard heart rate curve, that is, the heart rate detection accuracy of the smart wearable device in the running scenario can be greatly improved by using the product 2 of the present application.

[0280] Exemplarily, calibration product 1 may be a chest heart rate strap, and of course, may be any other possible heart rate detection device. Among them, the chest heart rate strap is a wearable ECG measuring device. Due to the advantages of small movement amplitude of the measuring part, large measuring area, and large correlation between ECG signal and heart pulse, the heart rate strap is very accurate in measuring heart rate and can be used as a calibration or standard product, but the disadvantage is that it is inconvenient to wear. Product 2 of the present application may be a smart wearable device such as a smart watch or a smart bracelet. This type of device is a measuring device based on photoelectric pulse signals, which is easy to wear and can be closely integrated with devices such as smart phones.

[0281] Fig.19 The following is a schematic diagram of the user interface when the present application solution is applied to a smart wearable device. Fig.19 As shown in (a), when a user wears a smart wearable device, the smart wearable device can be triggered to detect heart rate at any time in different sports scenes, wherein the heart rate detection can be automatically triggered by the system or by the user. Fig.19 As shown in (b), the smart wearable device displays the heart rate detection result: in the running scene, the user's heart rate is 117bpm. Fig.19 As shown in (c), the smart wearable device displays the heart rate detection result: in the walking scene, the user's heart rate value is 76bpm.

[0282] Fig. 20 Schematic diagram of the software architecture involved in the technical solution provided in the embodiment of the present application is shown. Fig. 20As shown, the software architecture involved in the present application scheme may include a user interface (UI) application layer, a system service layer (i.e., a framework layer), an algorithm library, a hardware abstraction layer (HAL), a kernel layer, and a hardware layer. Among them, the UI application layer may include a heart rate detection application and a motion recording application, etc. The system service layer may include a step counting service, a heart rate service, a calorie calculation service, and a heart health service, etc. The algorithm library may include a heart rate algorithm, a dimming algorithm, a sleep algorithm, and a wearing algorithm, etc. The HAL layer may include a PPG driver interface, a display interface, and a touch interface, etc. The kernel layer may include a PPG driver, a display driver, and a touch driver, etc. The hardware layer may include a PPG sensor, a liquid crystal display (LCD), and a motor, etc.

[0283] Among them, the present application scheme involves a heart rate detection application, a heart rate service, a heart rate algorithm, a PPG driver interface, a PPG driver, and a PPG sensor, which are used to detect the heart rate. Of course, the present application scheme also involves a display driver interface, a display driver, and an LCD, which can be used to display the heart rate data.

[0284] Understandably, Fig. 20 The software architecture shown is for illustrative purposes only and may be determined based on actual usage requirements, and is not limited in the embodiments of the present application.

[0285] The heart rate prediction algorithm provided in the embodiment of the present application adopts the decision fusion of the frequency tracking algorithm and the deep learning algorithm. Through multi-model fusion, a dynamic neural network is built using the attention mechanism to perform end-to-end scene recognition and heart rate prediction, which can improve the accuracy of heart rate prediction. As described above, the technical solution provided in the embodiment of the present application provides a new idea for multi-algorithm decision fusion, and proposes three decision fusion methods of traditional frequency tracking algorithms and deep learning algorithms to improve the accuracy of heart rate prediction.

[0286] It should also be noted that, in the embodiments of the present application, "greater than" can be replaced by "greater than or equal to", "less than or equal to" can be replaced by "less than", or "greater than or equal to" can be replaced by "greater than", and "less than" can be replaced by "less than or equal to".

[0287] The various embodiments described herein may be independent solutions or may be combined according to internal logic, and all of these solutions fall within the protection scope of this application.

[0288] It can be understood that the methods and operations implemented by the smart wearable device in the above-mentioned various method embodiments can also be implemented by components (such as chips or circuits) that can be used in smart wearable devices.

[0289] The above describes the method embodiment provided by the present application, and the following describes the device embodiment provided by the present application. It should be understood that the description of the device embodiment corresponds to the description of the method embodiment, so the content not described in detail can be referred to the method embodiment above, and for the sake of brevity, it will not be repeated here.

[0290] The above mainly describes the solution provided by the embodiment of the present application from the perspective of method steps. It is understandable that in order to achieve the above functions, the smart wearable device implementing the method includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should be aware that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of protection of this application.

[0291] The embodiment of the present application can divide the functional modules of the smart wearable device according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical functional division. There may be other feasible division methods in actual implementation. The following is an example of dividing each functional module according to each function.

[0292] Fig.21 A schematic block diagram of a heart rate detection device 800 based on deep learning provided in an embodiment of the present application. The device 800 can be used to perform the actions performed by the smart wearable device in the above method embodiment. The device 800 includes a detection unit 810, a signal acquisition unit 820 and a processing unit 830.

[0293] A detection unit 810 is used to detect that a user wearing a smart wearable device performs a first movement, and the smart wearable device is provided with a photoplethysmography (PPG) sensor;

[0294] A signal acquisition unit 820, configured to acquire a PPG signal through a PPG sensor;

[0295] The processing unit 830 is used to obtain first heart rate data based on the PPG signal and the first deep sequence neural network model; and obtain second heart rate data based on the PPG signal and the first frequency tracking algorithm model;

[0296] The processing unit 830 is further used to perform data fusion based on the first heart rate data and the second heart rate data to obtain a target heart rate value of the user during the first exercise.

[0297] Through the solution of this application, when it is detected that a user wearing a smart wearable device is performing a first exercise, a PPG signal can be collected through the PPG sensor in the smart wearable device; the PPG signal is input into the first deep sequence neural network model to obtain the first heart rate data; the PPG signal is input into the first frequency tracking algorithm model to obtain the second heart rate data; the first heart rate data and the second heart rate data are fused to obtain the user's target heart rate value in the first exercise scenario. Since the frequency tracking algorithm can quickly track changes in heart rate, it can compensate for scenarios that the deep sequence neural network cannot track in time when the heart rate suddenly changes. Therefore, the solution of this application can improve the accuracy of heart rate prediction by fusing heart rate data obtained by different models.

[0298] In some possible implementations, the processing unit 830 is specifically configured to:

[0299] The target heart rate value is obtained by performing a weighted summation of the first heart rate data and the second heart rate data using the following equation:

[0300] Y=w1x1+w2x2,

[0301] Among them, x1 represents the first heart rate data, w1 represents the weight value corresponding to the first heart rate data, x2 represents the second heart rate data, and w2 represents the weight value corresponding to the second heart rate data.

[0302] In some possible implementations, the processing unit 830 is further configured to:

[0303] Obtaining first sample data of the first deep sequence neural network and the first frequency tracking algorithm model respectively in different motion scenes;

[0304] Based on the following equation, the Bayesian algorithm is used to perform deep learning on the first sample data:

[0305] Y = w1x1 + w2x2;

[0306] The values ​​of w1 and w2 in different scenarios are obtained through training.

[0307] In some possible implementations, w1+w2=1.

[0308] In some possible implementations, the processing unit 830 is further configured to:

[0309] Determining whether the first motion is a first type of motion, where the first type of motion is a type of motion pre-set according to the characteristics of the frequency tracking algorithm model;

[0310] When the first motion is the first type of motion, determining whether the confidence level corresponding to the PPG signal is greater than a credible threshold;

[0311] When the confidence level corresponding to the PPG signal is greater than the credible threshold, the weight value w2 corresponding to the second heart rate data is increased to w2'.

[0312] In some possible implementations, the processing unit 830 is specifically configured to:

[0313] When the first exercise is the first type of exercise, the first heart rate data and the second heart rate data are weightedly summed using the following equation to obtain the target heart rate value: Y=w1x1+w2'x2; or,

[0314] When the first exercise is not the first type of exercise, the first heart rate data and the second heart rate data are weightedly summed using the following equation to obtain the target heart rate value: Y=w1x1+w2x2.

[0315] In some possible implementations, the first type of exercise is walking or cycling.

[0316] In some possible implementations, the processing unit 830 is further configured to:

[0317] Training a deep sequence neural network model based on the data features provided by the first frequency tracking algorithm model to obtain the first deep sequence neural network model;

[0318] The data features include PPG main frequency data and ACC main frequency data.

[0319] According to the embodiment of the present application, the device 800 may correspond to executing the method described in the embodiment of the present application, and the above and other operations and / or functions of the units in the device 800 are respectively for implementing the corresponding processes of the method, which will not be repeated here for the sake of brevity.

[0320] It should be understood that the device 800 in the embodiment of the present application may correspond to Figure 2 The above and other operations and / or functions of each unit in the apparatus 800 are respectively used to implement the corresponding processes of the above method, and for the sake of brevity, they are not repeated here.

[0321] Optionally, in some embodiments, the embodiments of the present application further provide a computer-readable medium, which stores program code. When the computer program code runs on a computer, the computer executes the methods in the above aspects.

[0322] Optionally, in some embodiments, the embodiments of the present application further provide a computer program product, which includes: a computer program code, and when the computer program code runs on a computer, the computer executes the methods in the above aspects.

[0323] In an embodiment of the present application, the smart wearable device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. Among them, the hardware layer may include hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory (also called main memory). The operating system of the operating system layer can be any one or more computer operating systems that implement business processing through processes, such as Linux operating system, Unix operating system, Android operating system, iOS operating system, or Windows operating system. The application layer may include applications such as browsers, address books, word processing software, and instant messaging software.

[0324] The embodiments of the present application do not specifically limit the specific structure of the execution subject of the method provided in the embodiments of the present application, as long as it can communicate according to the method provided in the embodiments of the present application by running a program that records the code of the method provided in the embodiments of the present application. For example, the execution subject of the method provided in the embodiments of the present application may be a smart wearable device, or a functional module in the smart wearable device that can call and execute a program.

[0325] Various aspects or features of the present application may be implemented as a method, apparatus, or article of manufacture using standard programming and / or engineering techniques. The term "article of manufacture" as used herein may encompass a computer program accessible from any computer-readable device, carrier, or medium. For example, a computer-readable medium may include, but is not limited to, a magnetic storage device (e.g., a hard disk, a floppy disk, or a magnetic tape, etc.), an optical disk (e.g., a compact disc (CD), a digital versatile disc (DVD), etc.), a smart card, and a flash memory device (e.g., an erasable programmable read-only memory (EPROM), a card, a stick, or a key drive, etc.).

[0326] The various storage media described herein may represent one or more devices and / or other machine-readable media for storing information. The term "machine-readable medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing and / or carrying instructions and / or data.

[0327] It should be understood that the processor mentioned in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0328] It should also be understood that the memory mentioned in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM). For example, RAM can be used as an external cache. By way of example and not limitation, RAM may include the following forms: static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0329] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, the memory (storage module) can be integrated into the processor.

[0330] It should also be noted that the memory described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0331] Those of ordinary skill in the art will appreciate that the units and steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of protection of this application.

[0332] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0333] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0334] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0335] In addition, each functional unit in each embodiment of the present application may be integrated into one unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0336] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a computer software product, which is stored in a storage medium. The computer software product includes several instructions that enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium may include, but is not limited to: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0337] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0338] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A heart rate detection method, It is characterized in that include: Determining that a user wearing a smart wearable device performs a first exercise, wherein a photoplethysmography (PPG) sensor is provided in the smart wearable device, wherein the PPG sensor is used to collect a PPG signal; Based on the multidimensional feature data and the first deep sequence neural network model, the first heart rate data is obtained; the multidimensional feature data includes the characteristics provided by the frequency tracking algorithm, the statistical characteristics of the PPG and acceleration ACC time domain signals, and the PPG and ACC frequency domain signals; Obtaining second heart rate data based on the PPG signal and the first frequency tracking algorithm model; When the first motion is not the first type of motion, based on Y= w 1 x 1 + w 2 x 2 fusing the first heart rate data and the second heart rate data to obtain a target heart rate value of the user during the first exercise; or, When the first motion is the first type of motion and the confidence level corresponding to the PPG signal is greater than a credible threshold, based on Y= w 1 x 1 + w 2 ' x 2 Performing data fusion to obtain the target heart rate value; Wherein, the first type of motion is a type of motion pre-set according to the characteristics of the frequency tracking algorithm model; Wherein, Y represents the target heart rate value, x 1 represents the first heart rate data, w 1 represents the weight value corresponding to the first heart rate data, x 2 represents the second heart rate data, w 2 represents the weight value corresponding to the second heart rate data, w 2 ' represents the modified weight value corresponding to the second heart rate data, w 2 'Greater than w 2 .

2. The method according to claim 1, It is characterized in that The frequency tracking algorithm provides features including PPG and ACC main frequency data provided based on the frequency tracking algorithm.

3. The method according to claim 1, It is characterized in that The method further comprises: Obtaining first sample data of the first deep sequence neural network and the first frequency tracking algorithm model respectively in different motion scenes; Based on the following equation, the Bayesian algorithm is used to perform deep learning on the first sample data: Y= w 1 x 1 + w 2 x 2 ； Training in different scenarios w 1 and w 2 The value of .

4. The method according to claim 1, It is characterized in that w 1 + w 2 =1。 5. The method according to claim 1, It is characterized in that Before performing data fusion, the method further includes: determining whether the first motion is the first type of motion; When the first motion is the first type of motion, determining whether the confidence level corresponding to the PPG signal is greater than a credible threshold; When the confidence level corresponding to the PPG signal is greater than the credible threshold, the weight value corresponding to the second heart rate data is w 2 Increase to w 2 '.

6. The method according to claim 1, It is characterized in that The first type of exercise is walking or cycling.

7. The method according to any one of claims 1 to 6, It is characterized in that The method further comprises: The deep sequence neural network model is trained based on the data features provided by the first frequency tracking algorithm model to obtain the first deep sequence neural network model. The data features provided by the first frequency tracking algorithm model include PPG main frequency data and acceleration ACC main frequency data.

8. The method according to any one of claims 1 to 6, It is characterized in that After obtaining the target heart rate value, the method further includes: The target heart rate value and information about the first exercise are displayed on a screen of the smart wearable device.

9. A heart rate detection method, It is characterized in that include: Determining that a user wearing a smart wearable device performs a first exercise, wherein a photoplethysmography (PPG) sensor is provided in the smart wearable device, wherein the PPG sensor is used to collect a PPG signal; Based on the multidimensional feature data and the first deep sequence neural network model, the first heart rate data is obtained; the multidimensional feature data includes the frequency tracking algorithm supply characteristics, the PP interval duration and the number of peak points, the PPG and acceleration ACC time domain signal statistical characteristics, and the PPG and ACC frequency domain signals; Obtaining second heart rate data based on the PPG signal and the first frequency tracking algorithm model; Based on the PPG signal and the time domain interval algorithm model, obtaining third heart rate data; When the first motion is not the second type of motion, based on Y= w 1 x 1 + w 2 x 2 + w 3 x 3 The first heart rate data, the second heart rate data and the third heart rate data are fused to obtain a target heart rate value of the user during the first exercise; or, when the first exercise is the second type of exercise and the autocorrelation coefficient of the PPG time domain signal corresponding to the PPG signal is greater than a coefficient threshold, Y= w 1 x 1 + w 2 x 2 + w 3 ' x 3 Performing data fusion to obtain the target heart rate value; Wherein, the second type of motion is a type of motion pre-set according to the characteristics of the time domain interval algorithm model; Wherein, Y represents the target heart rate value, x 1 represents the first heart rate data, w 1 represents the weight value corresponding to the first heart rate data, x 2 represents the second heart rate data, w 2 represents the weight value corresponding to the second heart rate data, x 3 represents the third heart rate data, w 3 represents the weight value corresponding to the third heart rate data, w 3 ' represents the modified weight value corresponding to the third heart rate data, w 3 'Greater than w 3 .

10. The method according to claim 9, It is characterized in that The frequency tracking algorithm provides features including PPG and ACC main frequency data provided based on the frequency tracking algorithm.

11. The method according to claim 9, It is characterized in that The method further comprises: Obtain second sample data of the first deep sequence neural network, the first frequency tracking algorithm model, and the time domain interval algorithm model in different motion scenes; Based on the following equation, the Bayesian algorithm is used to perform deep learning on the second sample data: Y= w 1 x 1 + w 2 x 2 + w 3 x 3 ， Training in different scenarios w 1 , w 2 and w 3 The value of .

12. The method according to claim 9, It is characterized in that w 1 + w 2 + w 3 =1。 13. The method according to claim 9, It is characterized in that Before performing data fusion, the method further includes: determining whether the first motion is a second type of motion; When the first motion is the second type of motion, determining whether an autocorrelation coefficient of a PPG time domain signal corresponding to the PPG signal is greater than the coefficient threshold; When the autocorrelation coefficient of the PPG time domain signal is greater than the coefficient threshold, the weight value corresponding to the third heart rate data is w 3 Increase to w 3 '.

14. The method according to claim 9, It is characterized in that The second type of exercise is resting or sleeping.

15. The method according to claim 9, It is characterized in that The method further comprises: Based on the first data feature provided by the first frequency tracking algorithm model and the second data feature provided by the time domain interval algorithm model, a deep sequence neural network model is trained to obtain the first deep sequence neural network model; Among them, the first data features include PPG main frequency data and ACC main frequency data, and the second data features include interval duration and the number of peak points.

16. The method according to any one of claims 9 to 15, It is characterized in that After obtaining the target heart rate value, the method further includes: The target heart rate value and information about the first exercise are displayed on a screen of the smart wearable device.

17. The method according to any one of claims 9 to 15, It is characterized in that The method further comprises: Acquire a multi-scene sample set, where the multi-scene sample set is a data sample set obtained based on detection of multiple motion scenes; Extracting acceleration ACC sample data, photoplethysmography PPG sample data and heart rate labels from the multi-scenario sample set; The ACC sample data and the PPG sample data, the duration of the PP interval and the number of peak points of the electrocardiogram, and the PPG and ACC main frequency data provided by the frequency tracking algorithm are used as input, and the heart rate label and the motion scene label are used as target variables, and training is performed through a deep sequence neural network; The first deep sequence neural network is obtained, wherein the first deep sequence neural network has scene recognition and heart rate prediction functions.

18. An electronic device, It is characterized in that The electronic device comprises a processor coupled to a memory, wherein the processor is configured to execute a computer program or instruction stored in the memory so that the electronic device implements the method according to any one of claims 1 to 17.

19. A chip system, It is characterized in that The chip system is coupled to a memory, and the chip system is used to read and execute a computer program stored in the memory to implement the method as claimed in any one of claims 1 to 17.

20. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 17.

21. A computer program product, It is characterized in that When the computer program product is run on a computer, the computer is caused to perform the method according to any one of claims 1 to 17.

Images