Respiratory rate processing method, device and computer readable storage medium

Abstract

The embodiment of the present invention proposes a method, device and medium for processing respiratory rate. The method part includes: firstly obtaining a PPG signal of a PPG sensor, obtaining an upper envelope and a lower envelope based on the PPG signal, and extracting features of the upper envelope and the lower envelope to obtain envelope features; then fusing the envelope features to obtain fused envelope features; secondly processing the fused envelope features to obtain target fused envelope features, converting the target fused envelope features from the time domain to the frequency domain to obtain a frequency domain energy map; finally, based on the frequency domain energy map, according to a pre-constructed frequency domain energy scoring model, selecting the frequency value with the highest score, and determining the respiratory rate value corresponding to the frequency value with the highest score, thereby improving the accuracy of the respiratory rate measurement value. The deviation in the accuracy of the respiratory rate measurement value caused by the error caused by a single envelope of the upper envelope or the lower envelope is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of wearable devices, and in particular to a method and device for processing respiratory rate and a computer-readable storage medium. Background Art

[0002] As people's living standards improve, health indicators have become one of the focuses of people's keen attention, and respiratory function has received more and more attention. The important parameter of respiratory function is respiratory rate, so it is particularly important to accurately monitor respiratory rate for the prevention and identification of respiratory diseases. Respiratory rate monitoring generally adopts direct measurement methods such as respiratory airflow method, including temperature, carbon dioxide, humidity content, etc., or respiratory sound measurement method, chest barrier method, etc. Indirect measurement methods include respiratory signals obtained based on electromyography, electrocardiography or infrared imaging. Most of the above methods are used for medical clinical respiratory monitoring and are difficult to use in daily life. Respiratory rate information can be indirectly obtained by using photoplethysmography (PPG) signal monitoring method. PPG is a photoelectric technology that can monitor the changes in blood volume in human tissue during the cardiac cycle. It has non-invasive characteristics and has the advantages of non-invasiveness and simple operation.

[0003] With the development of the smart wearable industry, users have higher and higher expectations for the intelligence of wearable devices. However, the current smart wearable devices measuring respiratory rate are easily affected by the user's environment, thus affecting the accuracy of measuring respiratory rate. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide a method for processing breathing rate to detect the breathing rate of a user. The method first obtains the PPG signal of the PPG sensor, obtains the upper envelope and the lower envelope based on the PPG signal, and extracts the features of the upper envelope and the lower envelope to obtain envelope features; then the envelope features are fused to obtain fused envelope features; secondly, the fused envelope features are processed to obtain target fused envelope features, and the target fused envelope features are converted from the time domain to the frequency domain to obtain a frequency domain energy map; finally, based on the frequency domain energy map, according to the pre-constructed frequency domain energy scoring model, the frequency value with the highest score is selected, and the breathing rate value corresponding to the frequency value with the highest score is determined, thereby improving the accuracy of the breathing rate measurement value.

[0005] In a first aspect of the present application, a respiratory rate processing method is provided, the method comprising: obtaining a PPG signal from a PPG sensor; obtaining an upper envelope and a lower envelope based on the PPG signal, and performing feature extraction on the upper envelope and the lower envelope to obtain envelope features; fusing the envelope features to obtain fused envelope features; processing the fused envelope features to obtain target fused envelope features, and converting the target fused envelope features from the time domain to the frequency domain to obtain a frequency domain energy map; based on the frequency domain energy map, according to a pre-constructed frequency domain energy scoring model, selecting the frequency value with the highest score, and determining the respiratory rate value corresponding to the frequency value with the highest score.

[0006] Optionally, in combination with the first aspect, in a possible implementation method, the envelope features are fused to obtain a fused envelope feature, including: flipping the lower envelope curve within the same window period upward, aligning it with the upper envelope curve according to the period, and performing weighted fusion, wherein the weight value and the window period are determined according to the PPG signal frequency and correlation; the correlation is the correlation between the upper envelope feature or the lower envelope feature and the respiratory rate value.

[0007] Optionally, in combination with the first aspect, in a possible implementation method, the envelope features are fused to obtain a fused envelope feature, including: flipping the upper envelope curve within the same window period downward, aligning it with the lower envelope curve according to the period, and performing weighted fusion, wherein the weight value and the window period are determined according to the PPG signal frequency and correlation; the correlation is the correlation between the upper envelope feature or the lower envelope feature and the respiratory rate value.

[0008] Optionally, in combination with the first aspect, in a possible implementation, before obtaining the PPG signal of the PPG sensor, it also includes: if it is a first measurement mode, determining whether the wearable device is stationary according to the acceleration signal, and if it is stationary, turning on the PPG signal; if it is a second measurement mode, identifying whether the wearable device has fallen asleep according to a sleeping algorithm, and if it has fallen asleep, turning on the PPG signal of a second time at intervals of a first time; the second time is less than the first time.

[0009] Optionally, in combination with the first aspect, in a possible implementation, the PPG signal is a green light signal, and it also includes: judging the signal quality of the PPG signal according to the interval characteristics of the PPG signal; the interval characteristics of the PPG signal are obtained by the interval difference between adjacent peaks; when adjacent intervals suddenly change continuously, it is judged that the signal quality does not meet the preset conditions; if the signal quality meets the preset conditions, reading the signal, preprocessing the PPG signal to remove the baseline signal to obtain a preprocessed PPG signal.

[0010] Optionally, in combination with the first aspect, in a possible implementation, obtaining a first signal feature of at least one dimension based on the PPG signal includes: obtaining the peaks and troughs of the preprocessed PPG signal; and interpolating the peaks and troughs to obtain upper and lower envelope features of the green light signal.

[0011] Optionally, in combination with the first aspect, in a possible implementation manner, obtaining the peaks and troughs of the preprocessed PPG signal includes: determining whether the PPG point of the preprocessed PPG signal is a maximum value or a minimum value; if it is a maximum value, determining whether it is the maximum value of an adjacent window length of N; if it is a maximum value, it is a valid peak; wherein, the selection of N is determined according to the frequency of the PPG signal; if it is a minimum value, determining whether it is the minimum value of an adjacent window length of N; if it is a minimum value, it is a valid trough; wherein, the selection of N is determined according to the frequency of the PPG signal.

[0012] Optionally, in combination with the first aspect, in a possible implementation manner, the PPG sensor includes one or more green LEDs and a photodetector, and acquiring the PPG signal of the PPG sensor includes: acquiring the PPG signal of the green LED.

[0013] Optionally, in combination with the first aspect, in a possible implementation, the PPG sensor includes one or more green LEDs, one or more red LEDs, one or more infrared LEDs and a photodetector, and obtaining the PPG signal of the PPG sensor includes: detecting a scene of the wearable device; determining signal qualities of green light, red light and infrared light according to the scene; and turning on at least one of the green LED, red LED and infrared LED according to the signal quality.

[0014] In a second aspect of the present application, a respiratory rate processing device is provided, the device comprising: an acquisition module, configured to acquire a PPG signal from a PPG sensor; an extraction module, configured to acquire an upper envelope and a lower envelope based on the PPG signal, and perform feature extraction on the upper envelope and the lower envelope to obtain envelope features; a fusion module, configured to fuse the envelope features to obtain fused envelope features; a conversion module, configured to process the fused envelope features to obtain target fused envelope features, and convert the target fused envelope features from the time domain to the frequency domain to obtain a frequency domain energy map; a determination module, configured to select the frequency value with the highest score based on the frequency domain energy map and according to a pre-constructed frequency domain energy scoring model, and determine the respiratory rate value corresponding to the frequency value with the highest score.

[0015] Optionally, in combination with the second aspect, in a possible implementation manner, the fusion module includes:

[0016] The first fusion unit is configured to flip the lower envelope curve within the same window period upward, align it with the upper envelope curve according to the period, and perform weighted fusion, wherein the weight value and the window period are determined according to the PPG signal frequency and correlation; the correlation is the correlation between the upper envelope feature or the lower envelope feature and the respiratory rate value.

[0017] Optionally, in combination with the second aspect, in a possible implementation manner, the fusion module includes:

[0018] The second fusion unit is configured to flip the upper envelope curve within the same window period downward, align it with the lower envelope curve according to the period, and perform weighted fusion, wherein the weight value and the window period are determined according to the PPG signal frequency and correlation; the correlation is the correlation between the upper envelope feature or the lower envelope feature and the respiratory rate value.

[0019] Optionally, in combination with the second aspect, in a possible implementation, the respiratory rate processing device further includes:

[0020] A first determination module is configured to determine whether the wearable device is stationary according to the acceleration signal if the wearable device is in the first measurement mode, and to turn on the PPG signal if the wearable device is stationary;

[0021] The second judgment module is configured to, if in the second measurement mode, identify whether the user has fallen asleep according to a sleeping algorithm, and if the user has fallen asleep, turn on the PPG signal for a second time at intervals of a first time; the second time is shorter than the first time.

[0022] Optionally, in combination with the second aspect, in a possible implementation, the respiratory rate processing device further includes:

[0023] A signal quality judgment module is configured to judge the signal quality of the PPG signal according to the interval characteristics of the PPG signal; the interval characteristics of the PPG signal are obtained by the interval difference between adjacent peaks;

[0024] The preprocessing module is configured to determine that the signal quality does not meet the preset conditions when mutations occur continuously in adjacent intervals; if the signal quality meets the preset conditions, the signal is read, and the PPG signal is preprocessed to remove the baseline signal to obtain a preprocessed PPG signal.

[0025] Optionally, in combination with the second aspect, in a possible implementation, the acquisition module includes: an acquisition unit, configured to acquire the peaks and troughs of the preprocessed PPG signal; and a difference unit, configured to interpolate the peaks and troughs respectively to obtain upper and lower envelope features of the green light signal.

[0026] Optionally, in combination with the second aspect, in a possible implementation manner, the acquiring unit includes:

[0027] A judging subunit, configured to judge whether the PPG point of the preprocessed PPG signal is a maximum value or a minimum value;

[0028] If it is a maximum value, determine whether it is the maximum value of the adjacent window length N; if it is the maximum value, it is a valid peak; wherein, the selection of N is determined according to the frequency of the PPG signal; if it is a minimum value, determine whether it is the minimum value of the adjacent window length N; if it is the minimum value, it is a valid trough; wherein, the selection of N is determined according to the frequency of the PPG signal.

[0029] Optionally, in combination with the second aspect, in a possible implementation manner, the acquisition module includes:

[0030] The green light acquisition unit is configured to acquire the PPG signal of the green light LED.

[0031] Optionally, in combination with the second aspect, in a possible implementation manner, the acquisition module includes:

[0032] a detection unit configured to detect a scene of the wearable device;

[0033] A quality judgment unit, configured to determine the signal quality of green light, red light and infrared light according to the scene;

[0034] The control unit is configured to turn on at least one of the green light LED, the red light LED and the infrared light LED according to the signal quality.

[0035] In a third aspect of the present application, a wearable device is provided, comprising a processor and a memory, wherein the memory stores a computer program that can be executed by the processor, and when the computer program is executed by the processor, the method in any possible implementation manner of the first aspect to the first aspect is implemented.

[0036] In a fourth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that when the computer program is executed by a processor, the method in any possible implementation manner of the first aspect to the first aspect is implemented.

[0037] In the embodiment provided by the present application, the method includes: firstly obtaining a PPG signal of a PPG sensor, obtaining an upper envelope and a lower envelope based on the PPG signal, and extracting features of the upper envelope and the lower envelope to obtain envelope features; then fusing the envelope features to obtain fused envelope features; secondly processing the fused envelope features to obtain target fused envelope features, converting the target fused envelope features from the time domain to the frequency domain to obtain a frequency domain energy map; finally, based on the frequency domain energy map, according to a pre-constructed frequency domain energy scoring model, selecting the frequency value with the highest score, and determining the respiratory rate value corresponding to the frequency value with the highest score, thereby improving the accuracy of the respiratory rate measurement value. The deviation of the accuracy of the respiratory rate measurement value caused by the error caused by a single envelope of the upper envelope or the lower envelope is avoided. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 A schematic diagram of the waveform modulation effect of the breathing process on the PPG signal provided in an embodiment of the present application;

[0040] Figure 2 A flow chart of a method for processing respiratory rate data provided in an embodiment of the present application;

[0041] Figure 3 A schematic diagram of an upper envelope frequency domain energy diagram provided in an embodiment of the present application;

[0042] Figure 4 A schematic diagram of the structure of a respiratory rate data processing device provided in an embodiment of the present application;

[0043] Figure 5 A module schematic diagram of a wearable device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0045] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0046] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0047] Respiratory rate is the number of breaths per unit time. The amount of light absorbed by the muscles and venous blood of the human body is constant, but when the heart beats, the volume of arterial blood also absorbs light periodically. When the heart beats strongly and the blood content in the blood vessels is high, the blood absorbs more light, and the detected projected or reflected light intensity is smaller; when the heart beats weakly and the blood content in the blood vessels is low, the blood absorbs less light, and the detected projected or reflected light is strong. The speed of blood flow will change with the change of respiratory rate, and the intensity of the received PPG signal will also change. Therefore, the respiratory rate can be obtained through the PPG signal.

[0048] In a calm state, the normal respiratory rate of an adult is about 16 to 20 times per minute. Women have an average respiratory rate that is 2 to 3 times faster than men. The fastest respiratory rate of a newborn is about 40 to 50 times per minute, and for children under 1 year old, it is about 30 to 40 times per minute. The respiratory rate of children over 7 years old is basically in line with that of adults.

[0049] Specifically, the measurement principle of respiratory rate is: use a light-emitting diode to illuminate the measured part, and then use a photodiode to receive the transmitted / reflected light and convert the optical signal into an electrical signal.

[0050] See also Figure 1The modulation effects of the breathing process on the PPG waveform include baseline modulation (BW) and pulse amplitude modulation (AM). Baseline modulation is that during the entire respiratory cycle, the changing intrathoracic pressure will cause venous blood return. The intrathoracic pressure will decrease during inspiration, causing a decrease in central venous pressure and an increase in venous return. The exhalation process is the opposite of the inhalation process, that is, the intrathoracic pressure will increase during exhalation, causing an increase in central venous pressure and a decrease in venous return. When more blood continues to be shunted to the low-pressure venous system, the modulation effect will act on the baseline (the baseline is No mod), causing the baseline waveform to move up and down. Pulse amplitude modulation is due to the change in intrathoracic pressure during inspiration. With each ventricular beat, the blood stroke volume will decrease, causing the respiratory pulse amplitude to decrease, such as Figure 1 As shown in the AM waveform, the amplitude of some respiratory waves is reduced compared to the baseline.

[0051] As an embodiment of the present invention, refer to Figure 2 , provides a method for processing respiratory rate, which is applied to wearable devices, and the method specifically includes:

[0052] S101: Acquire a PPG signal from a PPG sensor.

[0053] Among them, the PPG sensor may include one or more green light LEDs and photodetectors. The reason for choosing green light as the light source is that the following characteristics are taken into consideration: the melanin in the skin absorbs a large amount of waves with shorter wavelengths; the moisture on the skin also absorbs a large amount of UV and IR light; most of the green light (500nm)--yellow light (600nm) entering the skin tissue will be absorbed by red blood cells; red light and light close to IR are easier to pass through the skin tissue than other wavelengths of light; blood absorbs more light than other tissues; green (green-yellow) light can be absorbed by oxygenated hemoglobin and deoxygenated hemoglobin compared to red light. In summary, both green light and red light can be used as measurement light sources. The signal obtained by using green light as a light source is better, and the signal-to-noise ratio is better than other light sources, so green light is used as the light source in this example. However, considering the different skin conditions (skin color, sweat), as a preferred embodiment, the PPG sensor includes one or more green light LEDs, one or more infrared light LEDs and photodetectors, and can automatically use and switch multiple light sources such as green light, red light and IR according to the situation. The PPG sensor includes one or more green LEDs, one or more red LEDs, one or more infrared LEDs and a photodetector. The acquisition of the PPG signal of the PPG sensor includes: detecting the scene of the wearable device; determining the signal quality of green light, red light and infrared light according to the scene; turning on at least one of the green LED, red LED and infrared LED according to the signal quality. For example, in a high temperature environment, when the user is sweating in the gym, the moisture on the skin surface increases. Since more green light has been absorbed, it is more difficult to detect the green light reflected under the skin. The green light can be automatically turned off and switched to infrared light to adapt to various scenes and ensure the accuracy of the measured breathing rate.

[0054] S102: Acquire an upper envelope and a lower envelope based on the PPG signal, and perform feature extraction on the upper envelope and the lower envelope to obtain envelope features.

[0055] The PPG signal is generally expressed as a waveform, and the upper envelope and the lower envelope can be obtained by performing correlation processing on the waveform. The correlation processing method may include but is not limited to filtering, which is not limited here.

[0056] S103: Fusing the envelope features to obtain fused envelope features.

[0057] For S103, there are two implementation methods, including but not limited to the following:

[0058] Method 1: Flip the lower envelope curve within the same window period upward, align it with the upper envelope curve according to the period, and perform weighted fusion, wherein the weight value and the window period are determined according to the PPG signal frequency and correlation; the correlation is the correlation between the upper envelope feature (or the lower envelope feature) and the respiratory rate value.

[0059] Method 2: Flip the upper envelope curve within the same window period downward, align it with the lower envelope curve according to the period, and perform weighted fusion, wherein the weight value and the window period are determined according to the PPG signal frequency and correlation; the correlation is the correlation between the upper envelope feature (or the lower envelope feature) and the respiratory rate value.

[0060] S104: Processing the fused envelope feature to obtain a target fused envelope feature, and converting the target fused envelope feature from the time domain to the frequency domain to obtain a frequency domain energy map.

[0061] The processing method may be filtering, including but not limited to IIR, FIR or sliding filtering. After filtering, the target fusion envelope feature is in the respiratory rate band of 3 to 50 times / min, and then the target fusion envelope feature can be converted from the time domain to the frequency domain through Fourier transform to obtain a frequency domain energy map.

[0062] S105: Based on the frequency domain energy graph and according to a pre-constructed frequency domain energy scoring model, the frequency value with the highest score is selected, and the respiratory rate value corresponding to the frequency value with the highest score is determined.

[0063] refer to Figure 3 , shows a schematic diagram of the envelope frequency domain energy diagram, the horizontal axis is the time point, the vertical axis is the frequency value, the clearer curve is the frequency with the highest frequency domain energy value score, and each frequency value corresponds to the breathing rate value. The score of the frequency domain energy value corresponding to each time point on the frequency domain energy diagram of the target fusion envelope feature is calculated by the following formula.

[0064] Frequency domain energy value score = frequency continuity score * first weight value + energy value score * second weight value.

[0065] Among them, the frequency continuity score is that if the absolute difference between the frequency value at the current time point and the frequency value at the previous time point is less than the threshold, the continuity score is high, otherwise the score is low; the energy value score is to sort the energy values, the greater the energy, the higher the score; the first weight value and the second weight value are selected according to the signal-to-noise ratio of the corresponding frequency at each time point on the frequency energy diagram. In the frequency domain energy diagram, the energy value can be represented by brightness or clarity. The greater the brightness or the clearer the clarity, the greater the energy value, and vice versa.

[0066] As an embodiment of the present invention, the PPG signal of the PPG sensor is first obtained, the upper envelope and the lower envelope are obtained based on the PPG signal, and the upper envelope and the lower envelope are subjected to feature extraction to obtain envelope features; then the envelope features are fused to obtain fused envelope features; secondly, the fused envelope features are processed to obtain target fused envelope features, and the target fused envelope features are converted from the time domain to the frequency domain to obtain a frequency domain energy map; finally, based on the frequency domain energy map, according to a pre-constructed frequency domain energy scoring model, the frequency value with the highest score is selected, and the respiratory rate value corresponding to the frequency value with the highest score is determined, thereby improving the accuracy of the respiratory rate measurement value. The deviation in the accuracy of the respiratory rate measurement value caused by the error caused by a single envelope of the upper envelope or the lower envelope is avoided.

[0067] It should be pointed out that the above-mentioned respiratory rate processing method can be applied to wearable devices, such as bracelets and watches. During the application process, the method may include but is not limited to two usage scenarios. First, click on the respiratory rate measurement. That is, when the user clicks on the respiratory rate measurement from the graphical interface of the wearable device, the user's respiratory rate is measured. Second, sleep respiratory rate measurement. That is, when the user enters a sleep state, the user's respiratory rate is measured.

[0068] As an embodiment of the present invention, before S101, the method further includes:

[0069] S106: If it is the first measurement mode, determine whether the wearable device is stationary according to the acceleration signal, and if it is stationary, turn on the PPG signal.

[0070] The first measurement mode may be a click breathing rate measurement mode.

[0071] S107: If it is the second measurement mode, whether the user has fallen asleep is identified according to the sleeping algorithm. If the user has fallen asleep, a PPG signal of a second time is turned on at intervals of a first time; the second time is shorter than the first time.

[0072] The second measurement mode may be a sleep breathing rate measurement mode. The falling asleep algorithm may be completed through ACC amplitude recognition, which is not limited here. For ease of explanation, the second time may be three minutes and the first time may be ten minutes. If it is determined that the user falls asleep, a three-minute green light signal is turned on every ten minutes for signal extraction.

[0073] As an embodiment of the present invention, the PPG signal is a green light signal, and before S101, the method further includes:

[0074] S108: Determine the signal quality of the PPG signal according to the interval characteristics of the PPG signal; the interval characteristics of the PPG signal are obtained by the interval difference between adjacent peaks.

[0075] S109: When mutations occur continuously in adjacent intervals, it is determined that the signal quality does not meet the preset conditions; if the signal quality meets the preset conditions, the signal is read, and the PPG signal is preprocessed to remove the baseline signal to obtain a preprocessed PPG signal.

[0076] Based on the above corresponding embodiments of S108 and S109, as an embodiment of the present invention, S102 specifically includes:

[0077] S201: Acquire the peaks and troughs of the preprocessed PPG signal.

[0078] S202: interpolating the peaks and troughs to obtain upper and lower envelope features of the green light signal.

[0079] The interpolation method may include but is not limited to cubic spline interpolation, discrete smoothing interpolation, etc.

[0080] As an embodiment of the present invention, S201 specifically includes:

[0081] S301: Determine whether the PPG point of the preprocessed PPG signal is a maximum value or a minimum value.

[0082] S302: If it is a maximum value, determine whether it is the maximum value of the adjacent window length N; if it is a maximum value, it is a valid peak; wherein the selection of N is determined according to the frequency of the PPG signal.

[0083] S303: If it is a minimum value, determine whether it is the minimum value of the adjacent window length N; if it is a minimum value, it is a valid trough; wherein the selection of N is determined according to the frequency of the PPG signal.

[0084] For S301~S303, determine whether the pre-processed PPG point is larger than the adjacent points, and then determine whether it is the maximum value of the adjacent window length N. If it is the maximum value, it is a valid peak. Determine whether the pre-processed PPG point is smaller than the adjacent points, and then determine whether it is the minimum value of the adjacent window length N. If it is the minimum value, it is a valid trough. The selection of N is determined according to the PPG signal frequency. This method has strong anti-interference ability to noise and is easy to calculate.

[0085] refer to Figure 4 As one embodiment of the present invention, a respiratory rate processing device 40 is provided, comprising:

[0086] An acquisition module 410 is configured to acquire a PPG signal from a PPG sensor;

[0087] An extraction module 420 is configured to obtain an upper envelope and a lower envelope based on the PPG signal, and perform feature extraction on the upper envelope and the lower envelope to obtain envelope features;

[0088] A fusion module 430 is configured to fuse the envelope features to obtain fused envelope features;

[0089] The conversion module 440 is configured to process the fused envelope feature to obtain a target fused envelope feature, and convert the target fused envelope feature from the time domain to the frequency domain to obtain a frequency domain energy map;

[0090] The determination module 450 is configured to select the frequency value with the highest score based on the frequency domain energy graph and a pre-constructed frequency domain energy scoring model, and determine the respiratory rate value corresponding to the frequency value with the highest score.

[0091] Optionally, in a possible implementation, the fusion module 430 includes:

[0092] The first fusion unit is configured to flip the lower envelope curve within the same window period upward, align it with the upper envelope curve according to the period, and perform weighted fusion, wherein the weight value and the window period are determined according to the PPG signal frequency and correlation; the correlation is the correlation between the upper envelope feature or the lower envelope feature and the respiratory rate value.

[0093] Optionally, in a possible implementation, the fusion module 430 includes:

[0094] The second fusion unit is configured to flip the upper envelope curve within the same window period downward, align it with the lower envelope curve according to the period, and perform weighted fusion, wherein the weight value and the window period are determined according to the PPG signal frequency and correlation; the correlation is the correlation between the upper envelope feature or the lower envelope feature and the respiratory rate value.

[0095] Optionally, in a possible implementation, the respiratory rate processing device further includes:

[0096] A first determination module is configured to determine whether the wearable device is stationary according to the acceleration signal if the wearable device is in the first measurement mode, and to turn on the PPG signal if the wearable device is stationary;

[0097] The second judgment module is configured to, if in the second measurement mode, identify whether the user has fallen asleep according to a sleeping algorithm, and if the user has fallen asleep, turn on the PPG signal for a second time at intervals of a first time; the second time is shorter than the first time.

[0098] Optionally, in a possible implementation, the respiratory rate processing device further includes:

[0099] A signal quality judgment module is configured to judge the signal quality of the PPG signal according to the interval characteristics of the PPG signal; the interval characteristics of the PPG signal are obtained by the interval difference between adjacent peaks;

[0100] The preprocessing module is configured to determine that the signal quality does not meet the preset conditions when mutations occur continuously in adjacent intervals; if the signal quality meets the preset conditions, the signal is read, and the PPG signal is preprocessed to remove the baseline signal to obtain a preprocessed PPG signal.

[0101] Optionally, in a possible implementation, the acquisition module includes:

[0102] an acquisition unit, configured to acquire peaks and troughs of the preprocessed PPG signal;

[0103] The difference unit is configured to interpolate the peaks and troughs to obtain upper and lower envelope features of the green light signal.

[0104] Optionally, in a possible implementation manner, the acquiring unit includes:

[0105] A judging subunit, configured to judge whether the PPG point of the preprocessed PPG signal is a maximum value or a minimum value;

[0106] If it is a maximum value, determine whether it is the maximum value of the adjacent window length N; if it is the maximum value, it is a valid peak; wherein the selection of N is determined according to the frequency of the PPG signal;

[0107] If it is a minimum value, it is determined whether it is the minimum value of the adjacent window length N; if it is a minimum value, it is a valid trough; wherein the selection of N is determined according to the frequency of the PPG signal.

[0108] Optionally, in combination with the second aspect, in a possible implementation manner, the acquisition module includes:

[0109] The green light acquisition unit is configured to acquire the PPG signal of the green light LED.

[0110] Optionally, in combination with the second aspect, in a possible implementation manner, the acquisition module includes:

[0111] a detection unit configured to detect a scene of the wearable device;

[0112] A quality judgment unit, configured to determine the signal quality of green light, red light and infrared light according to the scene;

[0113] The control unit is configured to turn on at least one of the green light LED, the red light LED and the infrared light LED according to the signal quality.

[0114] In one embodiment provided by the present application, the PPG signal of the PPG sensor is first obtained, the upper envelope and the lower envelope are obtained based on the PPG signal, and the upper envelope and the lower envelope are subjected to feature extraction to obtain envelope features; then the envelope features are fused to obtain fused envelope features; secondly, the fused envelope features are processed to obtain target fused envelope features, and the target fused envelope features are converted from the time domain to the frequency domain to obtain a frequency domain energy map; finally, based on the frequency domain energy map, according to a pre-constructed frequency domain energy scoring model, the frequency value with the highest score is selected, and the respiratory rate value corresponding to the frequency value with the highest score is determined, thereby improving the accuracy of the respiratory rate measurement value. The deviation in the accuracy of the respiratory rate measurement value caused by the error caused by a single envelope of the upper envelope or the lower envelope is avoided.

[0115] like Figure 5 As shown, the wearable device 100 may include one or more processors 101, memory 102, communication module 103, sensor module 104, display screen 105, audio module 106, speaker 107, microphone 108, camera module 109, motor 110, button 111, indicator 112, battery 113, power management module 114. These components may communicate via one or more communication buses or signal lines.

[0116] The processor 101 is the final execution unit for information processing and program operation, and can run an operating system or application program to execute various functional applications and data processing of the wearable device 100. The processor 101 may include one or more processing units, for example: the processor 101 may include a central processing unit (CPU), a graphics processing unit (GPU), an image signal processor (ISP), a sensor hub processor or a communication processor (CP) application processor (AP), etc. In some embodiments, the processor 101 may include one or more interfaces. The interface is used to couple a peripheral device to the processor 101 to transmit instructions or data between the processor 101 and the peripheral device. In an embodiment of the present application, the processor 101 is also used to identify the target motion type corresponding to the motion data collected by the acceleration sensor and the gyroscope sensor, for example, walking / running / riding / swimming, etc. Specifically, processor 101 compares the motion waveform characteristics corresponding to the received motion data with the motion waveform characteristics corresponding to the target motion type to identify the target motion type corresponding to the motion data. Processor 101 is also used to determine whether the motion data within a preset time period all meet the preset motion intensity requirements associated with the target motion type. When it is determined that the motion data within the preset time period all meet the preset motion intensity requirements associated with the target motion type, processor 101 controls the sensor group associated with the target motion type to be turned on.

[0117] The memory 102 can be used to store computer executable program codes, and the executable program codes include instructions. The memory 102 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the wearable device 100, such as the motion parameters of each user's exercise, such as the number of steps, stride, pace, heart rate, breathing rate, blood sugar concentration, energy consumption (calories), etc. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. In an embodiment of the present application, the memory 102 can store sensor waveform regularity characteristic data corresponding to target sports such as walking, running, cycling, and swimming.

[0118] The communication module 103 can support the wearable device 100 to communicate with the network and the mobile terminal through wireless communication technology. The communication module 103 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. The communication module 103 may include one or more of a cellular mobile communication module, a short-range wireless communication module, a wireless Internet module, and a location information module. The mobile communication module can send or receive wireless signals based on the technical standards of mobile communications, and can use any mobile communication standard or protocol, including but not limited to the Global System for Mobile Communications (GSM), Code Division Multiple Access (CDMA), Code Division Multiple Access 2000 (CDMA2000), Wideband CDMA (WCDMA), Time Division Synchronous Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), LTE-A (Advanced Long Term Evolution), etc. The wireless Internet module can send or receive wireless signals via a communication network according to wireless Internet technology, including wireless LAN (WLAN), wireless fidelity (Wi-Fi), Wi-Fi direct, Digital Living Network Alliance (DLNA), wireless broadband (WiBro), etc. The short-range wireless communication module can send or receive wireless signals according to short-range communication technologies, including Bluetooth, radio frequency identification (RFID), infrared data communication (IrDA), ultra-wideband (UWB), ZigBee, near field communication (NFC), wireless fidelity (Wi-Fi), Wi-Fi direct connection, wireless USB (wireless universal serial bus), etc. The location information module can obtain the location of the wearable device based on the global navigation satellite system (GNSS), which can include one or more of the global positioning system (GPS), the global satellite navigation system (Glonass), the Beidou satellite navigation system and the Galileo satellite navigation system.

[0119] The sensor module 104 is used to measure physical quantities or detect the operating state of the wearable device 100. The sensor module 104 may include an acceleration sensor 104A, a gyroscope sensor 104B, an air pressure sensor 104C, a magnetic sensor 104D, a biometric sensor 104E, a proximity sensor 104F, an ambient light sensor 104G, a touch sensor 104H, etc. The sensor module 104 may also include a control circuit for controlling one or more sensors included in the sensor module 104.

[0120] Among them, the acceleration sensor 104A can detect the acceleration magnitude of the wearable device 100 in all directions. When the wearable device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the wearable device 100, and is applied to applications such as horizontal and vertical screen switching and pedometers. In one embodiment, the acceleration sensor 104A can be combined with the gyroscope sensor 104B to monitor the user's stride, step frequency, and pace during exercise.

[0121] The gyro sensor 104B may be used to determine the motion posture of the wearable device 100. In some embodiments, the angular velocity of the wearable device 100 around three axes (ie, x, y, and z axes) may be determined by the gyro sensor 104B.

[0122] The air pressure sensor 104C is used to measure air pressure. In some embodiments, the wearable device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 104C to assist positioning and navigation.

[0123] The GPS sensor 104D may be used to record user activity tracks to determine the user's location.

[0124] The biometric sensor 104E is used to measure the physiological parameters of the user, including but not limited to a photoplethysmography (PPG) sensor, an ECG sensor, an EMG sensor, a blood glucose sensor, and a temperature sensor. For example, the wearable device 100 can measure the user's heart rate, respiratory rate, and blood pressure data through the signals of the photoplethysmography sensor and / or the ECG sensor, and identify the user's blood glucose value based on the data generated by the blood glucose sensor. In an embodiment of the present application, the PPG sensor is used to detect the user's heart rate. Specifically, after being turned on, the PPG sensor can continuously detect signal data related to the user's heart rate and transmit it to the processor 101, and then the processor 101 calculates the heart rate value through a heart rate algorithm. In an embodiment of the present application, the temperature sensor is used to detect the first temperature of the user's wrist skin. Specifically, after being turned on, the temperature sensor can continuously obtain the temperature data of the user's wrist skin and transmit it to the processor 101, and then the processor 101 calculates the corresponding temperature value in physical sense through the temperature algorithm using the electrical signal data of the temperature sensor.

[0125] The proximity sensor 104F is used to detect the presence of an object near the wearable device 100 without any physical contact. In some embodiments, the proximity sensor 104F may include a light emitting diode and a light detector. The light emitting diode may be an infrared light, and the wearable device 100 uses the light detector to detect reflected light from a nearby object. When the reflected light is detected, it can be determined that there is an object near the wearable device 100. The wearable device 100 can use the proximity sensor 104F to detect its wearing state.

[0126] The ambient light sensor 104G is used to sense the brightness of the ambient light. In some embodiments, the wearable device 100 can adaptively adjust the brightness of the display screen according to the perceived brightness of the ambient light to reduce power consumption.

[0127] The touch sensor 104H is used to detect a touch operation applied thereto or in the vicinity thereof, and is also referred to as a “touch control device.” The touch sensor 104H may be disposed on the display screen 105 , and the touch sensor 104H and the display screen 105 form a touch screen.

[0128] The display screen 105 is used to display a graphical user interface (UI), which may include graphics, text, icons, videos, and any combination thereof. The display screen 105 may be a liquid crystal display (LCD), an organic light-emitting diode (OLED) display, and the like. When the display screen 105 is a touch display screen, the display screen 105 can collect a touch signal on or above the surface of the display screen 105, and input the touch signal as a control signal to the processor 101.

[0129] The audio module 106, the speaker 107, and the microphone 108 provide audio functions between the user and the wearable device 100, such as listening to music or making calls; for example, when the wearable device 100 receives a notification message from a mobile terminal, the processor 101 controls the audio module 106 to output a preset audio signal, and the speaker 107 emits a sound to remind the user. The audio module 106 converts the received audio data into an electrical signal and sends it to the speaker 107, which converts the electrical signal into sound; or the microphone 108 converts the sound into an electrical signal and sends it to the audio module 106, which then converts the audio electrical signal into audio data.

[0130] The camera module 111 is used to capture still images or videos. The camera module 111 may include an image sensor, an image signal processor (ISP), and a digital signal processor (DSP). The image sensor converts an optical signal into an electrical signal, the image signal processor converts the electrical signal into a digital image signal, and the digital signal processor converts the digital image signal into an image signal in a standard format (RGB, YUV). The image sensor may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS).

[0131] The motor 110 can convert electrical signals into mechanical vibrations to produce vibration effects. The motor 110 can be used for vibration prompts of incoming calls and messages, and can also be used for touch vibration feedback. The button 109 includes a power button, a volume button, etc. The button 109 can be a mechanical button (physical button) or a touch button. The indicator 112 is used to indicate the status of the wearable device 100, for example, to indicate the charging status, power changes, and can also be used to indicate messages, missed calls, notifications, etc. In some embodiments, the wearable device 100 provides vibration feedback after receiving a notification message from a mobile terminal application.

[0132] The battery 113 is used to provide power to various components of the wearable device 100. The power management module 114 is used to manage the charge and discharge of the battery, and monitor parameters such as battery capacity, battery cycle number, battery health status (whether it is leaking, impedance, voltage, current and temperature). In some embodiments, the power management module 114 can charge the battery by wire or wirelessly.

[0133] It should be understood that in some embodiments, the wearable device 100 may be composed of one or more of the aforementioned components, and the wearable device 100 may include more or fewer components than shown, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0134] It should be understood that in some embodiments, the wearable device may be composed of one or more of the aforementioned components, and the wearable device may include more or fewer components than shown, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0135] The present application also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the above-mentioned respiratory rate processing method is implemented.

[0136] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0137] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0138] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0139] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for processing respiratory rate, It is characterized in that The method comprises: Get the PPG signal from the PPG sensor; Acquire an upper envelope and a lower envelope based on the PPG signal, and perform feature extraction on the upper envelope and the lower envelope to obtain envelope features; fusing the envelope features to obtain fused envelope features; Processing the fused envelope feature to obtain a target fused envelope feature, and converting the target fused envelope feature from the time domain to the frequency domain to obtain a frequency domain energy map; Based on the frequency domain energy graph, according to a pre-constructed frequency domain energy scoring model, selecting the frequency value with the highest score, and determining the respiratory rate value corresponding to the frequency value with the highest score; The step of fusing the envelope features to obtain fused envelope features comprises: The lower envelope curve within the same window period is flipped upward, aligned with the upper envelope curve according to the period, and weighted fusion is performed, wherein the weight value and the window period are determined according to the PPG signal frequency and correlation; the correlation is the correlation between the upper envelope feature or the lower envelope feature and the respiratory rate value; or, The upper envelope curve within the same window period is flipped downward, aligned with the lower envelope curve according to the period, and weighted fusion is performed, wherein the weight value and the window period are determined according to the PPG signal frequency and correlation; the correlation is the correlation between the upper envelope feature or the lower envelope feature and the respiratory rate value.

2. The processing method according to claim 1, It is characterized in that Before obtaining the PPG signal of the PPG sensor, it also includes: If it is the first measurement mode, it is determined whether the wearable device is stationary according to the acceleration signal, and if it is stationary, the PPG signal is turned on; If it is the second measurement mode, whether the user has fallen asleep is identified based on the sleeping algorithm. If the user has fallen asleep, the PPG signal of the second time is turned on every first time interval; the second time is shorter than the first time.

3. The processing method according to claim 2, It is characterized in that If the PPG signal is a green light signal, the method further includes: Determining the signal quality of the PPG signal according to the interval characteristics of the PPG signal; the interval characteristics of the PPG signal are obtained by the interval difference between adjacent peaks; When mutations occur continuously in adjacent intervals, it is judged that the signal quality does not meet the preset conditions; if the signal quality meets the preset conditions, the signal is read, and the PPG signal is preprocessed to remove the baseline signal to obtain a preprocessed PPG signal.

4. The processing method according to claim 3, It is characterized in that Acquiring a first signal feature of at least one dimension based on the PPG signal includes: Obtaining peaks and troughs of the preprocessed PPG signal; The peaks and troughs are interpolated to obtain upper and lower envelope features of the green light signal.

5. The processing method according to claim 4, It is characterized in that The obtaining of the peaks and troughs of the preprocessed PPG signal comprises: Determine whether the PPG point of the preprocessed PPG signal is a maximum value or a minimum value; If it is a maximum value, determine whether it is the maximum value of the adjacent window length N; if it is the maximum value, it is a valid peak; wherein the selection of N is determined according to the frequency of the PPG signal; If it is a minimum value, it is determined whether it is the minimum value of the adjacent window length N; if it is a minimum value, it is a valid trough; wherein the selection of N is determined according to the frequency of the PPG signal.

6. The processing method according to claim 1, It is characterized in that The PPG sensor includes one or more green LEDs and a photodetector, and the method of acquiring a PPG signal of the PPG sensor includes: Get the PPG signal of the green LED.

7. The processing method according to claim 1, It is characterized in that The PPG sensor includes one or more green LEDs, one or more red LEDs, one or more infrared LEDs, and a photodetector. The method of acquiring the PPG signal of the PPG sensor includes: Detect wearable device scenarios; Determine the signal quality of green light, red light and infrared light according to the scenario; At least one of the green LED, the red LED and the infrared LED is turned on according to the signal quality condition.

8. A respiratory rate processing device, It is characterized in that The device comprises: An acquisition module, configured to acquire a PPG signal from a PPG sensor; an extraction module, configured to obtain an upper envelope and a lower envelope based on the PPG signal, and perform feature extraction on the upper envelope and the lower envelope to obtain envelope features; A fusion module is configured to fuse the envelope features to obtain fused envelope features; A conversion module is configured to process the fused envelope feature to obtain a target fused envelope feature, and convert the target fused envelope feature from the time domain to the frequency domain to obtain a frequency domain energy map; A determination module is configured to select the frequency value with the highest score based on the frequency domain energy graph and a pre-constructed frequency domain energy scoring model, and determine the respiratory rate value corresponding to the frequency value with the highest score; The fusion module includes a first fusion unit or a second fusion unit: A first fusion unit is configured to flip the lower envelope curve within the same window period upward, align it with the upper envelope curve according to the period, and perform weighted fusion, wherein the weight value and the window period are determined according to the PPG signal frequency and correlation; the correlation is the correlation between the upper envelope feature or the lower envelope feature and the respiratory rate value; The second fusion unit is configured to flip the upper envelope curve within the same window period downward, align it with the lower envelope curve according to the period, and perform weighted fusion, wherein the weight value and the window period are determined according to the PPG signal frequency and correlation; the correlation is the correlation between the upper envelope feature or the lower envelope feature and the respiratory rate value.

9. A wearable device, It is characterized in that The method comprises a processor and a memory, wherein the memory stores a computer program executable by the processor, and when the computer program is executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images