Vital sign detection method and electronic device
By performing frequency domain fusion and deep neural network processing on PPG signals from multiple channels, the problem of noise interference in PPG signal detection is solved, and the accuracy of heart rate detection is significantly improved.
Patent Information
- Application Number
- CN202111669683.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2021-11-01
- Filing Date
- 2021-12-30
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2041-12-30
AI Technical Summary
Heart rate detection based on photovoltaic pulse wave schema (PPG) signals is greatly disturbed by noise, resulting in a decrease in detection accuracy.
By obtaining the PPG signal and acceleration signal of multiple channels, calculating the correlation coefficient matrix and variance. If the noise interference is large, frequency domain fusion will be performed, and the input deep neural network will be input for interference filtering to improve detection accuracy.
It effectively reduces noise interference and improves the accuracy of heart rate detection, especially in high noise environments.
Smart Images

Figure CN116058815B_ABST
Abstract
Description
[0001] This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office on November 1, 2021, with application number 202111285028.0 and invention name “A heart rate prediction method and device”, the entire contents of which are incorporated by reference in this application. Technical Field
[0002] The present application relates to the field of intelligent terminals, and in particular to a vital sign detection method and electronic equipment. Background Art
[0003] Electronic devices that support heart rate detection can detect heart rate based on photoplethysmography (PPG) signals. The basic principle of detecting heart rate based on PPG signals is as follows: a light-emitting diode (LED) illuminates the skin tissue, and the light reflected back through the skin tissue is received by a photodiode (PD) and converted into a PPG signal. Since the absorption of light by human tissues such as muscles, bones, and veins is basically unchanged, the periodic changes in blood flow in the arteries will cause periodic changes in the PPG signal, so the heart rate can be detected based on the PPG signal.
[0004] However, the PPG signal is greatly affected by noise interference, which causes PPG signal distortion and seriously affects the accuracy of heart rate detection. Summary of the invention
[0005] The embodiments of the present application provide a vital sign detection method and an electronic device for improving the accuracy of vital sign detection.
[0006] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:
[0007] In a first aspect, a vital sign detection method is provided, including: obtaining an acceleration signal and a photoplethysmography (PPG) signal of multiple channels within a preset time period; calculating a correlation coefficient matrix between the PPG signals of the multiple channels; if there is at least one correlation coefficient in the correlation coefficient matrix that is less than or equal to a first threshold, calculating the variance of the PPG signal of each of the multiple channels; if the variances of the PPG signals of all channels are greater than or equal to a second threshold, performing FFT on the PPG signals of the multiple channels and then performing frequency domain fusion to obtain a frequency domain signal of the fused PPG signal; performing FFT on the acceleration signal to obtain a frequency domain signal of the acceleration signal; inputting the frequency domain signal of the fused PPG signal and the frequency domain signal of the acceleration signal into a deep neural network to obtain a numerical value of the vital sign; and displaying the numerical value of the vital sign.
[0008] The vital sign detection method provided in the embodiment of the present application calculates the correlation coefficient matrix between the PPG signals of multiple channels; if there is at least one correlation coefficient in the correlation coefficient matrix that is less than or equal to the first threshold, that is, there are at least two channels with low correlation between the PPG signals, calculate the variance of the PPG signal of each of the multiple channels; if the variance of the PPG signals of all channels is greater than or equal to the second threshold, that is, the PPG signals of all channels are greatly interfered by noise, perform frequency domain fusion after FFT on the PPG signals of multiple channels to obtain the frequency domain signal of the fused PPG signal; perform fast Fourier transform on the acceleration signal to obtain the frequency domain signal of the acceleration signal; input the frequency domain signal of the fused PPG signal and the frequency domain signal of the acceleration signal into the deep sequence neural network to obtain the value of the vital sign; and display the value of the vital sign. The interference in the PPG signal is filtered out by using the strong anti-interference characteristic of the deep neural network to improve the accuracy of vital sign detection.
[0009] In a possible implementation, the method further includes: if each correlation coefficient in the correlation coefficient matrix is greater than a first threshold, adding the PPG signals of multiple channels to a set; performing time domain fusion on the PPG signals of each channel belonging to the set, and performing fast Fourier transform on the time domain fused signal to obtain a frequency domain signal of the fused PPG signal. At this time, the correlation between the PPG signals of any two channels is high, and the noise interference of the PPG signal is small. Time domain fusion can reduce the computational workload without reducing the accuracy.
[0010] It should be noted that the PPG signals of multiple channels can be fused in the time domain, or the PPG signals of multiple channels can be fused in the frequency domain, either of which can be performed arbitrarily; or the two can be performed conditionally, that is, when the first condition is met, the PPG signals of multiple channels are fused in the time domain, and when the second condition is met, the PPG signals of multiple channels are fused in the frequency domain. For example, when the noise interference in the PPG signals of multiple channels is small (for example, less than a preset threshold, that is, the quality is good), the PPG signals of multiple channels are fused in the time domain, and when the noise interference in the PPG signals of multiple channels is large (for example, greater than a preset threshold, that is, the quality is poor), the PPG signals of multiple channels are fused in the frequency domain.
[0011] In a possible implementation, the method further includes: if the variance of the PPG signal of any channel is less than the second threshold, calculating the autocorrelation coefficient of the PPG signal of each channel whose variance is less than the second threshold. The autocorrelation coefficient indicates the correlation between the PPG signals of the same channel at different times (i.e., the quality of the periodicity), that is, the influence of the past PPG signal of the same channel on the current PPG signal. The better the periodicity of the PPG signal, the larger the autocorrelation coefficient, and the worse the periodicity of the PPG signal, the smaller the autocorrelation coefficient.
[0012] In a possible implementation, if the autocorrelation coefficient of the PPG signal of any channel is greater than the third threshold, the PPG signals of the channels belonging to the set are fused in the time domain, and the time-domain fused signal is fast Fourier transformed to obtain a frequency domain signal of the fused PPG signal. The better the periodicity of the PPG signal, the larger the autocorrelation coefficient, the smaller the noise interference of the PPG signal, and the time domain fusion can reduce the computational workload without reducing the accuracy.
[0013] In a possible implementation, the method further includes: if the autocorrelation coefficients of the PPG signals of all channels are less than or equal to the third threshold, performing fast Fourier transform on the PPG signals of multiple channels and then performing frequency domain fusion to obtain a frequency domain signal of the fused PPG signal. The worse the periodicity of the PPG signal, the smaller the autocorrelation coefficient, and the accuracy can be improved by frequency domain fusion.
[0014] In one possible implementation, the PPG signals of the channels belonging to the set are fused in the time domain, including: averaging the PPG signals of the channels belonging to the set. The averaging can reduce the computational workload without reducing the accuracy.
[0015] In a possible implementation, performing fast Fourier transform on PPG signals of multiple channels and then performing frequency domain fusion to obtain a frequency domain signal of a fused PPG signal, including: obtaining a frequency domain measurement value corresponding to the numerical value of the previous vital sign; performing fast Fourier transform on the PPG signals of multiple channels to obtain frequency domain signals of the PPG signals of multiple channels; obtaining a near peak in the frequency domain signal of the PPG signal of each channel, where the near peak refers to the peak value in the frequency domain signal of the PPG signal of each channel that is closest to the frequency domain measurement value; taking the maximum value of the near peak in the frequency domain signal of the PPG signals of multiple channels, and for the remaining values in the frequency domain signal of the PPG signals of multiple channels, taking the minimum value at the same time to obtain the frequency domain signal of the fused PPG signal.
[0016] By finding the peak close to the frequency domain measurement value corresponding to the value of the previous vital sign, the valid signal can be screened out. By determining the maximum value from the peak close to the value and then taking the minimum value of the remaining values, the maximum value of the valid signal is actually selected, and other invalid signals (including noise interference) are suppressed, thereby improving the signal-to-noise ratio.
[0017] In a possible implementation manner, the method further includes: increasing the number of channels in response to the first operation; or decreasing the number of channels in response to the second operation.
[0018] Increasing the number of channels means selecting PPG signals of more (e.g., all) channels for performing the vital sign detection method, thereby improving accuracy. Reducing the number of channels means selecting PPG signals of fewer channels for performing the vital sign detection method, thereby reducing data storage and calculation pressure, thereby reducing power consumption.
[0019] In a possible implementation, the vital sign is heart rate. The vital sign may also be blood oxygen saturation, etc., which is not limited in this application.
[0020] In one possible implementation, the deep neural network is a deep sequential neural network, which includes a recurrent network layer for accumulating a time series of input feature vectors.
[0021] When interference occurs suddenly, the recurrent network layer can effectively filter out the interference and improve the robustness of the system.
[0022] In a second aspect, an electronic device is provided, comprising: a processor, at least one light emitting diode and a plurality of photodiodes; the light emitting diode is used to emit light to human tissue; the plurality of photodiodes are used to respectively receive light reflected by human tissue and output a plurality of channels of photoplethysmography (PPG) signals; the processor is used to execute the method described in the first aspect and any one of its embodiments. The technical effect of the electronic device can refer to the technical effect of the first aspect and any one of its embodiments.
[0023] In one possible implementation, the number of photodiodes is 8. The present application does not limit the number of light emitting diodes 1031. For example, the number of light emitting diodes 1031 may be 2, 4, or another number. The more light emitting diodes 1031 there are, the higher the light intensity is, and the better the performance against ambient light interference is.
[0024] In one possible implementation, the number of light emitting diodes is 2. The present application does not limit the number of photodiodes 1032. For example, the number of photodiodes 1032 may be 2, 4, 8, or other numbers. The more photodiodes 1032 there are, the more channels of the output PPG signal, and the higher the detection accuracy.
[0025] In a possible implementation, a plurality of photodiodes are arranged in a ring around a light emitting diode in an area where the electronic device contacts human tissue, so that each photodiode receives light from the light emitting diode in a balanced manner.
[0026] In one possible implementation, the electronic device is a smart watch. The electronic device may be a wearable device, such as a smart watch, a smart bracelet, smart glasses, etc.; or the electronic device may be a terminal device, such as a mobile phone, a heart rate monitor, a blood oximeter, etc.
[0027] According to a third aspect, a computer-readable storage medium is provided, comprising instructions, which, when executed on an electronic device, enable the electronic device to execute the method according to the first aspect and any embodiment thereof.
[0028] In a fourth aspect, a computer program product comprising instructions is provided. When the instructions are executed on the electronic device, the electronic device executes the method as described in the first aspect and any embodiment thereof.
[0029] In a fifth aspect, a chip system is provided, the device comprising a processor and an interface circuit, the interface circuit is used to read instructions stored in a memory and send the instructions to the processor. When the instructions are executed by the processor, the electronic device can execute the method as described in the first aspect and any embodiment thereof, and the chip system can include chips and other discrete devices.
[0030] Among them, the technical effects brought about by any implementation of the third to fifth aspects can refer to the technical effects brought about by the first aspect and any implementation of it, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 A schematic diagram of a deep sequence neural network provided in an embodiment of the present application;
[0032] Figure 2 A schematic diagram of the structure of a device for generating a PPG signal provided in an embodiment of the present application;
[0033] Figure 3 A schematic diagram of a principle of detecting heart rate based on a PPG signal provided in an embodiment of the present application;
[0034] Figure 4 A schematic diagram of another principle of detecting heart rate based on PPG signals provided in an embodiment of the present application;
[0035] Figure 5 A schematic diagram of the principle of the influence of ambient light interference on PPG signals provided in an embodiment of the present application;
[0036] Figure 6 A schematic diagram of a user wearing an electronic device running provided in an embodiment of the present application;
[0037] Figure 7 A schematic diagram of the effect of baseline drift and motion artifacts on PPG signals provided in an embodiment of the present application;
[0038] Figure 8 A schematic diagram of the effect of a static state and a motion state on a frequency domain signal of a PPG signal provided in an embodiment of the present application;
[0039] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;
[0040] Fig.10 A schematic diagram of the structure of a smart watch provided in an embodiment of the present application;
[0041] Fig.11 A schematic diagram of a processor running software architecture provided in an embodiment of the present application;
[0042] Fig.12 A schematic diagram of a flow chart of a vital sign detection method provided in an embodiment of the present application;
[0043] Fig.13 A schematic diagram of a switch for a vital sign detection function and a switch for an enhanced function of vital sign detection provided in an embodiment of the present application;
[0044] Fig.14 A schematic diagram of another switch for a vital sign detection function and a switch for an enhanced function of vital sign detection provided in an embodiment of the present application;
[0045] Fig.15 A schematic diagram of obtaining an acceleration signal within a preset time period and a PPG signal of multiple channels by using a sliding time window provided in an embodiment of the present application;
[0046] Fig.16 A schematic diagram of a flow chart of another vital sign detection method provided in an embodiment of the present application;
[0047] Fig.17 A schematic diagram of a correlation coefficient matrix provided in an embodiment of the present application;
[0048] Fig.18 A schematic diagram of a flow chart of another vital sign detection method provided in an embodiment of the present application;
[0049] Fig.19 A schematic diagram of a frequency domain signal of a fused PPG signal obtained according to a close peak provided in an embodiment of the present application;
[0050] Fig. 20 A schematic diagram of anti-interference of a deep sequence neural network provided in an embodiment of the present application;
[0051] Fig.21 A schematic diagram of the comparison of heart rates obtained by a vital sign detection method provided in an embodiment of the present application;
[0052] Fig. 22 A schematic diagram showing vital signs provided in an embodiment of the present application;
[0053] Fig.23A schematic diagram of the structure of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION
[0054] It should be noted that the terms "first", "second", etc. involved in the embodiments of the present application are only used to distinguish features of the same type and cannot be understood as indicating relative importance, quantity, order, etc.
[0055] The terms "exemplary" or "for example" and the like in the embodiments of the present application are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the terms "exemplary" or "for example" is intended to present the related concepts in a specific way.
[0056] The terms "coupling" and "connection" involved in the embodiments of the present application should be understood in a broad sense. For example, they may refer to a direct physical connection, or an indirect connection achieved through electronic devices, such as a connection achieved through resistors, inductors, capacitors or other electronic devices.
[0057] First, some concepts involved in this application are described:
[0058] Variance: Assume that there is a time series X1, X2, X3, ..., X i (i=1,2,...,n), then the time series X1,X2,X3,......,X i The variance of (i=1,2,...,n) is denoted as δ 2 =D(X i )=E((X i -μ X ) 2 ). Among them, E() represents the mean, μ X represents the time series X1,X2,X3,......,X i The mean of (i=1,2,...,n).
[0059] Pearson correlation coefficient: The Pearson correlation coefficient between variable X and variable Y is shown in the following formula 1:
[0060]
[0061] Among them, cov(X,Y) represents the covariance between variable X and variable Y, δ X represents the standard deviation of variable X, δ Y represents the standard deviation of variable Y, μ X represents the mean of variable X, μ Y represents the mean of variable Y, and E() represents the mean.
[0062] Expand the above formula, assuming that the number of sample points is n, then n sample points X1, X2, X3, ..., X i (i=1,2,...,n) and n sample points Y1,Y2,Y3,......,Y i The Pearson correlation coefficient between (i=1,2,...,n) is shown in the following formula 2:
[0063]
[0064] in, Represents n sample points X1,X2,X3,......,X i The mean of (i=1,2,...,n), Represents n sample points Y1,Y2,Y3,......,Y i (i=1,2,...,n), where n represents the number of sample points.
[0065] n sample points X i (i=1,2,...,n) and n sample points Y i The Pearson correlation coefficient between (i=1,2,...,n) can also be expressed by estimating the mean of the standard scores of the sample points, as shown in the following formula 3:
[0066]
[0067] in, Represents n sample points X1,X2,X3,......,X i The standard score of (i=1,2,...,n), Represents n sample points Y1,Y2,Y3,......,Y i The standard score of (i=1,2,...,n), Represents n sample points X1,X2,X3,......,X i The mean of (i=1,2,...,n), δ X Represents n sample points X1,X2,X3,......,X i The standard deviation of (i=1,2,...,n), Represents n sample points Y1,Y2,Y3,......,Y i The mean of (i=1,2,...,n), δ Y Represents n sample points Y1,Y2,Y3,......,Y i The standard deviation of (i=1,2,...,n).
[0068] Autocorrelation coefficient: Assuming there is a time series X1, X2, X3, ..., X i (i=1,2,...,n), n represents the time series X1,X2,X3,......,X i (i=1,2,...,n) length, then there are time series X1,X2,X3,......,X i The mean of (i=1,2,...,n) is μ X =E(X i ), time series X i The variance of (i=1,2,...,n) is δ 2 , time series X1,X2,X3,......,X i The standard deviation of (i=1,2,...,n) is δ. For two time series X1,X2,X3,......,X i (i=1,2,...,n) and Y1,Y2,Y3,...,Y i (i=1,2,...,n), can be obtained by covariance cov(X i ,Y i )=E((X i -μ X )(Y i -μ Y )) to represent the correlation between the two time series. The larger the absolute value of the covariance, the greater the correlation between the two time series (when the covariance is greater than 0, it is positive correlation, and when the covariance is less than 0, it is negative correlation). If one of the two time series (for example, Y i (i=1,2,...,n)) is another time series X1,X2,X3,......,X i (i=1,2,...,n) time series with k lag points (i.e. Y i =X i-k ), we can get the time series X1,X2,X3,......,X i The biased estimate of the autocovariance of (i=1,2,...,n) is shown in Formula 4, and we get the time series X1,X2,X3,......,X i The unbiased estimate of the autocovariance of (i=1,2,...,n) is shown in Formula 5. According to Formula 4 and Formula 5, we can get the time series X1,X2,X3,......,X i The autocorrelation coefficient of (i=1,2,...,n) is shown in Formula 6.
[0069]
[0070]
[0071]
[0072] Deep Sequential Neural Network: A deep neural network that can accumulate time series and effectively eliminate sudden interference in time series. Figure 1 As shown, the deep sequence neural network includes a first fully connected layer 11, a normalization layer 12, a recurrent network layer 13 and a second fully connected layer 14, and optionally a residual layer 15, each layer includes at least one neuron. The neurons of the first fully connected layer 11 are fully connected to the neurons of the normalization layer 12. The first fully connected layer 11 inputs a feature vector and outputs a feature vector after nonlinear processing to the normalization layer 12. The normalization layer 12 normalizes the feature vector and outputs it to the recurrent network layer 13 to accelerate the convergence of the deep sequence neural network. The recurrent network layer 13 accumulates the time series of the normalized feature vector and outputs it to the second fully connected layer 14. The neurons of the second fully connected layer 14 are fully connected to the neurons of the recurrent network layer 13, and the second fully connected layer is used to output the final calculation result. The residual layer 14 can be located between any two adjacent layers among the first fully connected layer 11, the normalization layer 12, the recurrent network layer 13 and the second fully connected layer 14, and is used to reduce the quantity of the output of the previous layer and output it to the next layer to accelerate the convergence of the deep sequence neural network.
[0073] The training method of the deep sequence neural network is: assume that there are multiple sets of training data, each set of training data includes a training feature vector and a correct first calculation result corresponding to the feature vector. During training, each training feature vector is input into the neural network and a second calculation result is obtained. The loss function is calculated based on the first calculation results and the second calculation results corresponding to all the training feature vectors. When the loss function does not meet the conditions, the parameters of the neural network are adjusted according to the gradient descent method. The above training process is repeated until the loss function meets the conditions and the parameters of the neural network are stopped to complete the training of the neural network.
[0074] Photoplethysmography (PPG): Figure 2 As shown, the light emitting diode 21 illuminates the skin tissue 23, and the light reflected by the skin tissue 23 is received by the photodiode 22 and converted into an electrical signal of one channel, and the electrical signal is then converted into a digital signal through analog-to-digital conversion. It should be noted that in the embodiment of the present application, the electrical signal of one channel output by the photodiode is collectively referred to as a PPG signal.
[0075] Because blood absorbs more green light, the light-emitting diode 21 usually uses green light so that the intensity change of the reflected light can better reflect the change of blood. Since the absorption of light by human tissues such as muscles, bones, and veins is basically unchanged, the electrical signal output by the photodiode 22 when detecting the light reflected by this part of human tissue is a DC signal. Figure 3 As shown in the figure, due to the periodic changes (diastole and contraction) of blood flow in the artery, the electrical signal output by the photodiode when detecting the light reflected by the arterial blood is an AC signal. Therefore, the PPG signal output by the photodiode includes a DC signal and an AC signal. By extracting the AC signal, the detection of vital signs can be achieved. Taking the heart rate as an example, Figure 3 As shown, a time domain analysis method is to obtain the heart rate N*60 / T per minute by counting the number N of peaks (maximum intensity) of the AC signal in the PPG signal within a unit time T, such as Figure 4 As shown, another frequency domain analysis method is to perform fast Fourier transform (FFT) on the PPG signal to obtain the frequency domain signal of the PPG signal, where the energy is strongest near 0 Hz, which corresponds to the measurement value of human tissues such as muscles, bones, and veins. The second strongest energy peak appears at A, which corresponds to the heart rate measurement value, and the heart rate B = A*60 per minute can be calculated.
[0076] The difficulty of PPG is that due to factors such as ambient light interference, baseline drift, and motion artifacts, a large amount of noise will be introduced, causing PPG signal distortion, making it impossible to achieve accurate recognition. Figure 5 As shown in FIG. 1 , ambient light interference refers to: when the photodiode 22 is not in close contact with the skin tissue 23, the photodiode 22 will not only detect the light emitted by the light-emitting diode 21, but also the ambient light, and the intensity of the ambient light is much greater than the intensity of the light emitted by the photodiode 21, thereby causing serious interference to the PPG signal. Baseline drift refers to the interference caused by human breathing and the surface of the skin and the PPG sensor ( Figure 2 The noise with a frequency below 1 Hz generated by the relative friction between the light emitting diode 21 and the photodiode 22 in the PPG signal is superimposed on the AC signal corresponding to the heart rate. By appropriately filtering the PPG signal, the baseline drift signal can be separated. Figure 6 As shown, motion artifacts refer to high-frequency noise aliased in the PPG signal when the user wearing the electronic device 60 is in motion (such as walking, running, etc.) (see Figure 7 ).
[0077] For example, Figure 7As shown in the figure, baseline drift will produce low-frequency noise (slowly changing), which will affect the characteristic waveform changes over a long period of time; motion artifacts will produce high-frequency noise (faster changing) that is aliased with the characteristic waveform of the heart rate, affecting the characteristic waveform changes in a short period of time. Figure 8 As shown, A is the time domain signal of the PPG signal in a stationary state, B is the frequency domain signal of the PPG signal in a stationary state, C is the time domain signal of the PPG signal in a moving state, and D is the frequency domain signal of the PPG signal in a moving state. By comparing A and C, it can be seen that the time domain signal of the PPG signal in a stationary state is relatively regular, while the time domain signal of the PPG signal in a moving state is distorted. By comparing B and D, it can be seen that the energy proportion of the heart rate measurement value of the frequency domain signal of the PPG signal in a stationary state is high, and the peak can be accurately determined. The energy proportion of the heart rate measurement value of the time domain signal of the PPG signal in a moving state is very low, and the peak cannot be accurately determined.
[0078] The present application embodiment provides an electronic device for executing the vital sign detection method provided in the present application embodiment. The electronic device may be a wearable device, such as Fig.10 The smart watch, smart bracelet, smart glasses, etc. shown in the figure; or, the electronic device can be a terminal device, such as a mobile phone, a heart rate meter, an oximeter, etc. The embodiment of the present application takes the electronic device as a smart watch as an example, but is not intended to be limited to this. The vital signs involved in the embodiment of the present application may include but are not limited to heart rate, blood oxygen saturation (including oxygenated hemoglobin and reduced hemoglobin), etc.
[0079] like Fig. 9 As shown, the electronic device 100 includes a processor 101, a memory 102, a PPG circuit 103, an acceleration sensor 104 and a display screen 105. Among them, the PPG circuit 103 includes at least one light-emitting diode 1031, at least one photodiode 1032, a light-emitting diode driving circuit 1033, an amplifier 1034, and a signal waveform processing (signal conditioning) circuit 1035. Among them, the memory 102 and the processor 101 can be integrated in a system on chip (SoC) chip or can be separately arranged. The light-emitting diode 1031 and the photodiode 1032 can be integrated on a micro-displacement stage (microtranslation stage) or can be separately arranged, and the number of the light-emitting diodes 1031 and the number of the photodiodes 1032 can be the same or different.
[0080] The PPG circuit 103 is used to output a PPG signal to the processor 101 , and the acceleration sensor 104 is used to output an acceleration signal (at least one of the X / Y / Z axis acceleration signals) to the processor 101 .
[0081] The light emitting diode 1031 can emit light of a specific wavelength, for example, it can emit green light, red light, infrared light, etc. respectively by time-division multiplexing, or it can emit a combination of these lights, which is not limited in this application. When the light emitting diode 1031 emits green light, the electronic device 100 can detect the heart rate (i.e., output the heart rate), when the light emitting diode 1031 emits red light, the electronic device 100 can detect oxygenated hemoglobin (i.e., output the ratio of oxygenated hemoglobin), and when the light emitting diode 1031 emits infrared light, the electronic device 100 can detect reduced hemoglobin (i.e., output the ratio of reduced hemoglobin). It should be noted that the embodiment of the present application is described by taking heart rate detection as an example, but it is not intended to be limited to this.
[0082] The LED driving circuit 1033 is used to provide a driving current to the LED 1031. Because the driving capability of each pin of the processor 101 is usually limited and it is difficult to drive the LED 1031 to emit light, in the embodiment of the present application, the processor 101 can output a control signal to the LED driving circuit 1033 to control the LED driving circuit 1033 to provide a driving current to the LED 1031.
[0083] Each photodiode 1032 is used to detect the light from the light emitting diode 1031 reflected by the human tissue and output a PPG signal of one channel, so multiple photodiodes 1032 can output PPG signals of multiple channels. The size of the PPG signal output by the photodiode 1032 is related to the intensity of the light detected by the photodiode 1032. The greater the intensity of the light detected by the photodiode 1032, the greater the output PPG signal, and the smaller the intensity of the light detected by the photodiode 1032, the smaller the output PPG signal.
[0084] like Fig.10 As shown in A and B, taking the electronic device as a smart watch as an example, the light-emitting diode 1031 and the photodiode 1032 can be located at the bottom of the watch so as to contact human tissue. There can be multiple light-emitting diodes 1031 and photodiodes 1032, and the application does not limit the number of light-emitting diodes 1031 and photodiodes 1032. For example, the number of light-emitting diodes 1031 can be 2, 4 or other numbers, and the number of photodiodes 1032 can be 2, 4, 8 or other numbers. The more light-emitting diodes 1031 there are, the higher the light intensity is, and the better the performance against ambient light interference is; the more photodiodes 1032 there are, the more channels of the output PPG signal is, and the higher the detection accuracy is.
[0085] The light emitting diodes 1031 are as far apart as possible to reduce correlation, and the photodiodes 1032 are as far apart as possible to reduce correlation, thereby improving measurement accuracy. Fig.10 As shown in A, multiple (for example, 2) light-emitting diodes 1031 are relatively arranged, and multiple (for example, 8) photodiodes 1032 are arranged in a ring (symmetrically) around the light-emitting diode 1031 in the area where the electronic device contacts the human tissue (for example, the bottom of a smart watch), so that each photodiode 1032 receives light from the light-emitting diode 1031 in a balanced manner.
[0086] Each photodiode 1032 can output a PPG signal of one channel. In order to reduce the number of channels, reduce data storage and computing pressure, and thus reduce power consumption, the PPG signals output by multiple (e.g., 2, 4, 8) photodiodes 1032 can be combined. For example, Fig.10 As shown in B, the PPG signals output by two (i.e., paired) photodiodes 1032 located at symmetrical positions among the eight photodiodes 1032 arranged in a circular symmetry can be combined (current summation), and then combined into a PPG signal of four channels, and output to the processor 101. When the user wearing the smart watch is exercising, even if one of the two paired photodiodes 1032 is far away from human tissue, the other photodiode 1032 can be close to human tissue, and the PPG signals output by the two photodiodes 1032 can be combined to achieve the purpose of comprehensive noise reduction.
[0087] The amplifier 1034 is used to amplify the PPG signal output by the photodiode 1032 .
[0088] The signal waveform processing circuit 1035 is used to perform noise reduction, bandpass filtering and other processing on the PPG signal.
[0089] The display screen 105 can be used to display the values of the vital signs detected. The display screen 105 can also have a touch function and feed back the user's touch position in the display screen 105 to the processor 101.
[0090] The memory 102 may include at least one of a volatile memory and a non-volatile memory. The non-volatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable programmable ROM (EEPROM) or a flash memory. The volatile memory may be a random access memory (RAM), for example, a static RAM (SRAM), a dynamic RAM (DRAM), a synchronous DRAM (SDRAM), a double data rate synchronous DRAM (DDR SDRAM), an enhanced synchronous DRAM (ESDRAM), a synchronous link DRAM (SLDRAM) and a direct rambus RAM (DR RAM), etc. The memory 102 is used to store programs, instructions, and related data. When the programs and instructions are executed by the processor 101, the vital sign detection method provided in the embodiment of the present application can be executed.
[0091] The processor 101 may be a chip, for example, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD), or other integrated chips. The processor 101 executes the vital sign detection method provided in the embodiment of the present application by executing the program and instructions stored in the memory 102.
[0092] The program executed by the processor 101 may be based on an operating system, such as Android. Apple (iOS) Harmony Windows operating system, etc. Fig.11 As shown, the program executed by the processor 101 is based on For example, the programs run by the processor 101 are layered according to their functions, and may include an application layer, a system service layer, an algorithm library hardware abstraction layer, a kernel layer, and a driver layer.
[0093] The driver layer is used to drive hardware resources. The driver layer may include PPG driver, acceleration sensor driver and motor driver, etc. Among them, the PPG driver is used to drive the PPG circuit 103 and provide PPG signals to the upper layer software (such as the PGG module in the hardware abstraction layer), and the acceleration sensor driver is used to drive the acceleration sensor 104 and provide acceleration signals to the upper layer software (such as the acceleration sensor module in the hardware abstraction layer).
[0094] The kernel layer includes the operating system (OS) kernel, which is used to manage the system's processes, memory, drivers, file systems, and network systems.
[0095] The hardware abstraction layer (HAL) is used to abstract the hardware. It includes the PGG module, the accelerometer module, the display module, the touch module, etc. Among them, the PGG module is used to abstract the PPG hardware, and the abstracted PPG circuit is used by the vital signs detection algorithm in the algorithm library to call the PPG signal to the vital signs detection algorithm. The accelerometer module is used to abstract the accelerometer sensor, and the abstracted accelerometer sensor is used by the vital signs detection algorithm in the algorithm library to call the acceleration signal to the vital signs detection algorithm.
[0096] The algorithm library may include vital sign detection algorithms (such as heart rate algorithms, blood oxygen saturation algorithms), dimming algorithms, sleep algorithms, wearing algorithms, etc. Among them, the vital sign detection algorithm is used to execute the vital sign detection method provided in the embodiment of the present application.
[0097] The system service layer may include vital sign detection services (such as heart rate services, blood oxygen saturation services), step counting services, calorie services, health services, etc. Among them, the vital sign detection service is used to provide a service for calling a vital sign detection algorithm to the vital sign detection program.
[0098] The application layer may include a vital sign detection program (eg, a heart rate program, a blood oxygen saturation program), a dial program, a motion recording program, a call program, etc. Among them, the vital sign detection program is used to output the result of the vital sign detection.
[0099] The working principle of the electronic device for performing the vital sign detection method is as follows: the processor 101 controls the light emitting diode 1031 to emit light to the skin tissue through the light emitting diode driving circuit 1033, and the light reflected back through the skin tissue 23 is received by the photodiode 1032 and converted into a PPG signal. The amplifier 1034 amplifies the PPG signal and outputs it to the signal waveform processing circuit 1035. The signal waveform processing circuit 1035 performs noise reduction, bandpass filtering and other processing on the PPG signal and then sends it to the processor 101. The processor 101 may have a built-in analog-to-digital conversion circuit or the electronic device may also include an analog-to-digital conversion circuit (not shown in the figure). The analog-to-digital conversion circuit performs analog-to-digital conversion on the PPG signal to obtain a digital PPG signal, and the analog-to-digital conversion circuit performs analog-to-digital conversion on the acceleration signal output by the acceleration sensor 104 to obtain a digital acceleration signal. The processor 101 obtains the value of the vital sign (such as heart rate, blood oxygen saturation, etc.) according to the digital PPG signal and the acceleration signal, and controls the display screen 105 to display the value of the vital sign.
[0100] The following describes how the processor 101 performs the vital sign detection method. Fig.12 As shown, the method includes steps S101-S105:
[0101] S101, acquiring an acceleration signal (at least one of the X / Y / Z axis acceleration signals) and PPG signals of multiple channels within a preset time period.
[0102] The PPG signals of multiple channels may be PPG signals output by all photodiodes, or may be PPG signals output by some photodiodes, or may be PPG signals of multiple channels obtained by merging PPG signals output by multiple photodiodes. How to select the PPG signals of multiple channels is controlled by the switch of the vital sign detection function and the switch of the enhancement function of the vital sign detection:
[0103] Taking the electronic device as a smart watch and the vital sign detection as heart rate detection as an example, for example, Fig.13 and Fig.14 As shown, you can Fig.13 ) or the shortcut menu ( Fig.14 ) displays a switch of a vital sign detection function (e.g., a "heart rate detection" switch) or a switch of an enhanced function of vital sign detection (e.g., a "heart rate detection enhancement" switch). The switch of the enhanced function of vital sign detection may depend on the switch of the vital sign detection function. When the switch of the vital sign detection function is turned on, the user can control the switch of the enhanced function of vital sign detection.
[0104] The processor 101 can turn on or off the vital signs detection (such as heart rate detection) function in response to the user clicking the switch of the vital signs detection function. When the switch is turned on, the electronic device can execute the vital signs detection method. When the switch is turned off, the electronic device does not execute the vital signs detection method.
[0105] The processor 101 can turn on or off the enhanced function of vital sign detection (such as heart rate detection) in response to the user clicking the switch of the enhanced function of vital sign detection. When the switch is turned on (i.e., in response to the first operation), the processor 101 can increase the number of channels, i.e., select PPG signals of more (e.g., all) channels for executing the vital sign detection method, thereby improving the accuracy. For example, Fig.10 As shown in A, the PPG signals output by all eight photodiodes 1032 are selected to perform the vital sign detection method. When the switch is turned off (i.e., in response to the second operation), the processor 101 can reduce the number of channels, i.e., select PPG signals of fewer channels to perform the vital sign detection method, so as to reduce data storage and calculation pressure, thereby reducing power consumption. For example, Fig.10 As shown in FIG. 1B , the electronic device may combine the PPG signals output by the plurality of photodiodes 1032 , or the electronic device may select some of the photodiodes 1032 (eg Fig.10 The PPG signals output by (a, b, c, d) in (B).
[0106] The acceleration signal comes from the acceleration sensor mentioned above.
[0107] The PPG signal and acceleration signal here can be time domain signals obtained after noise reduction, bandpass filtering and other processing, and can be digital signals obtained after analog-to-digital conversion. Exemplarily, the preset time period can be 12 seconds, the number of sampling points of the acceleration signal and the PPG signal of each channel in the preset time period can be 300, and the bandwidth of the bandpass filtering of the PPG signal can be [0.3Hz, 4Hz], so as to filter out the DC signal mentioned above and retain only the AC signal corresponding to the vital signs (such as heart rate).
[0108] It should be noted that the processor 101 may acquire the acceleration signal and the PPG signals of multiple channels within a preset time period by means of a sliding time window. Fig.15 As shown, the processor 101 can obtain the acceleration signal and the PPG signal of multiple channels in the time window 1 (0-12 seconds) at the 12th second, obtain the acceleration signal and the PPG signal of multiple channels in the time window 2 (1-13 seconds) at the 13th second, and obtain the acceleration signal and the PPG signal of multiple channels in the time window 3 (2-14 seconds) at the 14th second.
[0109] S102, performing time domain fusion on the PPG signals of multiple channels to obtain a frequency domain signal of the fused PPG signal; or performing frequency domain fusion on the PPG signals of multiple channels to obtain a frequency domain signal of the fused PPG signal.
[0110] The PPG signals of multiple channels are fused in the time domain, or the PPG signals of multiple channels are fused in the frequency domain, and either of them can be performed arbitrarily; or the two can be performed conditionally, that is, when the first condition is met, the PPG signals of multiple channels are fused in the time domain, and when the second condition is met, the PPG signals of multiple channels are fused in the frequency domain. For example, when the noise interference in the PPG signals of multiple channels is small (for example, less than a preset threshold, that is, the quality is good), the PPG signals of multiple channels are fused in the time domain, and when the noise interference in the PPG signals of multiple channels is large (for example, greater than a preset threshold, that is, the quality is poor), the PPG signals of multiple channels are fused in the frequency domain.
[0111] Perform time domain fusion on PPG signals of multiple channels: first, fuse the PPG signals of multiple channels that meet certain conditions in the time domain to obtain the time domain signal of the fused PPG signal, and then perform FFT on the time domain signal of the fused PPG signal to obtain the frequency domain signal of the fused PPG signal.
[0112] Perform frequency domain fusion on the PPG signals of multiple channels: first perform FFT on the PPG signals of multiple channels to obtain frequency domain signals of the PPG signals of multiple channels, and then fuse these frequency domain signals to obtain frequency domain signals of the fused PPG signals.
[0113] In a possible implementation, Fig.16 As shown, the steps may include S1021-S1025:
[0114] S1021. Calculate the correlation coefficient matrix between the PPG signals of multiple channels.
[0115] For example, take the PPG signal of four channels as an example, Fig.17 As shown in , the correlation coefficient matrix can be a symmetric matrix, each row and each column of the correlation coefficient matrix represents a channel, and any numerical value in the correlation coefficient matrix represents the correlation coefficient between the PPG signals of two channels. The correlation coefficient represents the correlation between the PPG signals of the two channels. The larger the correlation coefficient, the higher the correlation between the PPG signals of the two channels, and the smaller the correlation coefficient, the lower the correlation between the PPG signals of the two channels.
[0116] In a possible implementation, the correlation coefficient matrix may be a Pearson correlation coefficient matrix, and any value in the Pearson correlation coefficient matrix represents the Pearson correlation coefficient between the PPG signals of the two channels. The PPG signals of any two channels are respectively used as n sample points X i (i=1,2,...,n) and n sample points Y i (i=1,2,...,n), according to formula 2, the Pearson correlation coefficient between the PPG signals of any two channels can be obtained, and then the Pearson correlation coefficient matrix between the PPG signals of multiple channels can be obtained. For example, the number of sample points n in formula 2 is 300,
[0117] If each correlation coefficient in the correlation coefficient matrix is greater than the first threshold, that is, the correlation between the PPG signals of any two channels is high, then it is considered that the first condition is met, and the PPG signals of multiple channels are added to the set and step S1024 is executed; otherwise (there is at least one correlation coefficient in the correlation coefficient matrix that is less than or equal to the first threshold, that is, there are at least two channels with low correlation between the PPG signals), step S1022 is executed.
[0118] S1022. Calculate the variance of the PPG signal of each of the multiple channels.
[0119] The n sample points X1, X2, X3, ..., X i (i=1,2,...,n), we can use the formula The variance of the PPG signal of the channel is calculated, where μ represents the mean of the n sample points. For example, the number of sample points n is 300. The variance of the PPG signal of each channel represents the degree of dispersion of the PPG signal of the channel, that is, the degree to which each sampling point deviates from the mean. When the variance is large, it means that the PPG signal of the channel is more dispersed and is more affected by noise. When the variance is small, it means that the PPG signal of the channel is less dispersed and is less affected by noise.
[0120] If the variance of the PPG signal of any channel is less than the second threshold, that is, the PPG signal of any channel is less affected by noise, step S1023 is executed; otherwise (the variance of the PPG signals of all channels is greater than or equal to the second threshold, that is, the PPG signals of all channels are more affected by noise), it is considered that the second condition is met, and step S1025 is executed.
[0121] S1023. For the PPG signal of each channel whose variance is less than the second threshold, calculate the autocorrelation coefficient of the PPG signal.
[0122] The autocorrelation coefficient of the PPG signal can be calculated according to Formula 4-Formula 6. The autocorrelation coefficient represents the correlation between the PPG signals of the same channel at different times (i.e., the quality of the periodicity), that is, the influence of the past PPG signal of the same channel on the current PPG signal. The better the periodicity of the PPG signal, the larger the autocorrelation coefficient, and the worse the periodicity of the PPG signal, the smaller the autocorrelation coefficient. For example, the time series length n in Formula 4-Formula 6 is 300, and k is 25, that is, the autocorrelation coefficient represents the correlation between the PPG signal of a channel 25 sampling points ago and the current PPG signal.
[0123] If the autocorrelation coefficient of the PPG signal of any channel is greater than the third threshold, that is, the correlation between the PPG signals of any channel at different times is high, then the first condition is considered to be met, and the PPG signal of the channel is added to the set and step S1024 is executed; otherwise (the autocorrelation coefficients of the PPG signals of all channels are less than or equal to the third threshold, that is, the correlation between the PPG signals of all channels at different times is low), then the second condition is considered to be met, and step S1025 is executed.
[0124] S1024. Perform time domain fusion on the PPG signals of each channel belonging to the set, and perform FFT on the time domain fused signal to obtain a frequency domain signal of the fused PPG signal.
[0125] The time domain fusion of the PPG signals of each channel belonging to the set includes: averaging the PPG signals of each channel belonging to the set to obtain the time domain signal of the fused PPG signal, and performing FFT on the time domain signal of the fused PPG signal to obtain the frequency domain signal of the fused PPG signal. When the noise interference of the PPG signal is small, the time domain fusion can reduce the computational workload without reducing the accuracy.
[0126] It should be noted that step S1024 does not depend on steps S1021-S1023 described above, that is, step S1024 can be directly executed. If step S1024 is directly executed, the above set includes the PPG signals of the multiple channels obtained in step S101, that is, the PPG signals of the multiple channels obtained in step S101 are fused in the time domain to obtain the frequency domain signal of the fused PPG signal.
[0127] S1025, performing FFT on the PPG signals of multiple channels and then performing frequency domain fusion to obtain a frequency domain signal of the fused PPG signal.
[0128] It should be noted that step S1025 does not depend on the steps S1021-S1023 described above, that is, it is not necessary to perform conditional judgment through steps S1021-S1023, and step S1024 or step S1025 can be directly executed.
[0129] In a possible implementation, Fig.18 As shown, step S1025 includes steps S10251-S10254:
[0130] S10251. Obtain a frequency domain measurement value A corresponding to the numerical value of the previous vital sign.
[0131] The value of the last vital sign refers to the value of the vital sign obtained during the last vital sign detection. Since the value of the last vital sign in the time domain can be mapped to the corresponding frequency domain, the frequency domain measurement value corresponding to the value of the last vital sign can be obtained by calculating the corresponding frequency domain measurement value according to the value of the last vital sign. For example, Fig.19 As shown in Figure 4 Similarly, assuming that the value of the previous heart rate is B, the frequency domain measurement value corresponding to the value B of the previous heart rate is A=B / 60.
[0132] S10252. Perform FFT on the PPG signals of the multiple channels to obtain frequency domain signals of the PPG signals of the multiple channels.
[0133] For example, Fig.19 As shown, assuming there are three channels of PPG signals, curves 1 to 3 respectively represent the frequency domain signals of the PPG signals of the three channels. Among them, S10252 and S10251 have no timing relationship, and can be performed at the same time, or S10252 can be performed before S10251.
[0134] S10253. Obtain a near peak in the frequency domain signal of the PPG signal of each channel.
[0135] The close peak refers to the peak value in the frequency domain signal of the PPG signal of the channel that is closest to the frequency domain measurement value A. For example, Fig.19 As shown, the peaks closest to the frequency domain measurement value A in curves 1 to 3 are P1, P2 and P3 respectively.
[0136] S10254. Take the maximum value close to the peak in the frequency domain signals of the PPG signals of multiple channels, and for the remaining values in the frequency domain signals of the PPG signals of multiple channels, take the minimum value at the same time to obtain the frequency domain signal of the fused PPG signal.
[0137] For example, Fig.19 As shown, the value of P1 among the peaks P1, P2 and P3 is the largest, and the minimum values of the other values in the frequency domain signals of the PPG signals of these three channels at the same time are combined, so that curve 4 can be obtained, which is the frequency domain signal of the fused PPG signal.
[0138] Since the vital signs change slowly and there is no large mutation, steps S10251-S10253 can filter out valid signals by finding the peak close to the frequency domain measurement value corresponding to the value of the previous vital sign. In step S10254, by determining the maximum value from the peak close to the frequency domain measurement value and then taking the minimum value of the remaining values, the maximum value of the valid signal is actually selected, and other invalid signals (including noise interference) are suppressed, thereby improving the signal-to-noise ratio.
[0139] S103 , performing FFT on the acceleration signal to obtain a frequency domain signal of the acceleration signal.
[0140] It should be noted that there is no requirement for the execution order of step S103 and step S102. For example, step S102 can be executed first and then step S103, or step S103 can be executed first and then step S102. Or, for a processor that supports multi-threading or has multiple cores, step S102 and step S103 can be executed simultaneously.
[0141] S104: Inputting the frequency domain signal of the fused PPG signal and the frequency domain signal of the acceleration signal into a deep neural network to obtain the numerical value of the vital sign.
[0142] The present application does not limit the specific form of the deep neural network, and mainly uses the anti-interference characteristics of the deep neural network to filter out noise interference in the PPG signal. In a possible implementation, the embodiment of the present application adopts the deep sequence neural network described above.
[0143] For example, Fig. 20 As shown, curve 1 represents the signal that has not been processed by the deep sequence neural network, curve 2 represents the signal that has been processed by the deep sequence neural network, and curve 3 represents the reference signal. Fig. 20 A in the middle shows that when the noise interference is not large, both curve 1 and curve 2 can fit well with curve 3. Fig. 20 Figures B and C show that when the noise interference is large, curve 1 is greatly distorted, while curve 2 can still fit well with curve 3. It can be seen that the deep sequence neural network has good anti-interference performance for signals.
[0144] In the embodiment of the present application, the frequency domain signal of the fused PPG signal and the frequency domain signal of the acceleration signal can be spliced into a feature vector, and the feature vector is input as follows: Figure 1 The deep sequence neural network shown in FIG. 1 is a deep sequence neural network, and the calculation result output by the deep sequence neural network is the value of the vital sign (such as heart rate). The role of the acceleration signal is to remove the motion noise in the PPG signal.
[0145] Since vital signs change slowly and do not undergo major mutations, the recurrent network layer 13 in the deep sequence neural network accumulates the time series of feature vectors. When interference suddenly occurs, the recurrent network layer 13 can effectively filter out the interference and improve the robustness of the system.
[0146] It should be noted that when the neural network is trained, the feature vector in each set of training data is a feature vector formed by splicing the frequency domain signal of the PPG signal and the acceleration signal, and the calculation result in each set of training data is the value of the vital sign corresponding to the feature vector. The specific training process is described above and will not be repeated here.
[0147] For example, Fig.21 As shown, curves 1 to 3 are the heart rate values calculated by using the PPG signals of the three channels alone, curve 4 is the heart rate value obtained by using the vital sign detection method provided in the embodiment of the present application, and curve 5 is the true heart rate value. It can be seen that when there is severe interference in the time period T, the heart rate value calculated by using the PPG signals of the three channels alone is greatly distorted, while the heart rate value obtained by using the vital sign detection method provided in the embodiment is not greatly distorted.
[0148] S105. Displaying the values of vital signs.
[0149] For example, taking the vital sign as heart rate, Fig. 22 As shown, the processor can control the display screen to display the heart rate value.
[0150] After applying the technical solution provided by this application to existing products, the accuracy of heart rate detection is significantly improved, with an average improvement of about 2.5%. For a single scene, the accuracy of scenes with large noise interference such as treadmill speed running, outdoor walking, and indoor elliptical machine is significantly improved, especially in the indoor elliptical machine scene, the accuracy is improved by nearly 14%. That is, the greater the noise interference, the more obvious the improvement in accuracy.
[0151] The technical solution provided by the present application improves the accuracy of heart rate detection because: on the one hand, the interference in the PPG signal is filtered out by utilizing the strong anti-interference characteristics of the deep neural network. On the other hand, when the noise interference is large, the PPG signals of multiple channels are fused in the frequency domain to obtain a frequency domain signal of the fused PPG signal; wherein, the near peaks of the PPG signals of each channel are determined based on the value of the previous vital sign, and the maximum value is determined from the near peaks, and the minimum value is determined from the remaining values, thereby obtaining the frequency domain signal of the fused PPG signal, selecting the maximum value of the valid signal, and suppressing other invalid signals (including noise interference).
[0152] The vital signs detection method and electronic device provided in the embodiment of the present application obtain acceleration signals and PPG signals of multiple channels within a preset time period; perform time domain fusion on the PPG signals of multiple channels to obtain frequency domain signals of fused PPG signals, or perform frequency domain fusion on the PPG signals of multiple channels to obtain frequency domain signals of fused PPG signals; perform fast Fourier transform on the acceleration signal to obtain frequency domain signals of the acceleration signal; input the frequency domain signals of the fused PPG signal and the frequency domain signals of the acceleration signal into a deep neural network to obtain the values of vital signs; and display the values of vital signs. The strong anti-interference characteristics of the deep sequence neural network are used to filter the interference in the PPG signal, thereby improving the accuracy of vital signs detection.
[0153] The present application also provides a chip system, such as Fig.23 As shown, the chip system 2300 includes at least one processor 2301 and at least one interface circuit 2302. The processor 2301 and the interface circuit 2302 can be interconnected via a line. For example, the interface circuit 2302 can be used to receive signals from other devices (such as a memory of an electronic device). For another example, the interface circuit 2302 can be used to send signals to other devices (such as the processor 2301). Exemplarily, the interface circuit 2302 can read instructions stored in the memory and send the instructions to the processor 2301. When the instructions are executed by the processor 2301, the electronic device can execute the various steps in the above embodiments, such as executing Fig.12 , Fig.16 , Fig.18 Of course, the chip system may also include other discrete devices, which are not specifically limited in the embodiments of the present application.
[0154] The present application also provides a computer-readable storage medium, which includes instructions. When the instructions are executed on the electronic device, the electronic device executes each function or step performed by the electronic device in the method embodiment, such as executing Fig.12 , Fig.16 , Fig.18 The method shown.
[0155] The present application also provides a computer program product including instructions. When the instructions are executed on the electronic device, the electronic device executes the functions or steps executed by the electronic device in the method embodiment, such as executing Fig.12 , Fig.16 , Fig.18 The method shown.
[0156] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0157] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0158] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0159] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.
[0160] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one device or distributed on multiple devices. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.
[0161] In addition, each functional module in each embodiment of the present application may be integrated into one device, or each module may exist physically separately, or two or more modules may be integrated into one device.
[0162] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing a computer program instruction on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or may contain one or more servers, data centers and other data storage devices that can be integrated with a medium. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).
[0163] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A method for detecting vital signs, characterized in that: include: Acquire acceleration signals and photoplethysmography (PPG) signals of multiple channels within a preset time period; Calculating a correlation coefficient matrix between the PPG signals of the multiple channels; If there is at least one correlation coefficient in the correlation coefficient matrix that is less than or equal to a first threshold, calculating the variance of the PPG signal of each of the multiple channels; If the variances of the PPG signals of all channels are greater than or equal to the second threshold, performing fast Fourier transform on the PPG signals of the multiple channels and then performing frequency domain fusion to obtain a frequency domain signal of the fused PPG signal; Performing a fast Fourier transform on the acceleration signal to obtain a frequency domain signal of the acceleration signal; Inputting the frequency domain signal of the fused PPG signal and the frequency domain signal of the acceleration signal into a deep neural network to obtain a value of a vital sign; The numerical values of the vital signs are displayed.
2. The method according to claim 1, characterized in that Also includes: If each correlation coefficient in the correlation coefficient matrix is greater than the first threshold, adding the PPG signals of the multiple channels to a set; The PPG signals of each channel belonging to the set are fused in the time domain, and the time domain fused signals are fast Fourier transformed to obtain the frequency domain signal of the fused PPG signal.
3. The method according to claim 2, characterized in that Also includes: If there is any channel whose variance of the PPG signal is smaller than the second threshold, the autocorrelation coefficients of the PPG signals of the channels whose variances are smaller than the second threshold are calculated.
4. The method according to claim 3, characterized in that: Also includes: If the autocorrelation coefficient of the PPG signal of any channel is greater than the third threshold, the PPG signals of each channel belonging to the set are fused in the time domain, and the time domain fused signal is fast Fourier transformed to obtain the frequency domain signal of the fused PPG signal.
5. The method according to claim 3, characterized in that: Also includes: If the autocorrelation coefficients of the PPG signals of all channels are less than or equal to the third threshold, the PPG signals of the multiple channels are subjected to fast Fourier transform and then subjected to frequency domain fusion to obtain a frequency domain signal of the fused PPG signal.
6. The method according to claim 2 or 4, characterized in that: The performing time domain fusion on the PPG signals of each channel belonging to the set includes: The PPG signals of the channels belonging to the set are averaged.
7. The method according to any one of claims 1 to 5, characterized in that: The method of performing fast Fourier transformation on the PPG signals of the multiple channels and then performing frequency domain fusion to obtain a frequency domain signal of the fused PPG signal includes: Obtain the frequency domain measurement value corresponding to the value of the previous vital sign; Performing fast Fourier transform on the PPG signals of the multiple channels to obtain frequency domain signals of the PPG signals of the multiple channels; Obtaining a close peak in the frequency domain signal of the PPG signal of each channel, wherein the close peak refers to a peak value in the frequency domain signal of the PPG signal of each channel that is closest to the frequency domain measurement value; The maximum value close to the peak in the frequency domain signals of the PPG signals of the multiple channels is taken, and for the remaining values in the frequency domain signals of the PPG signals of the multiple channels, the minimum value at the same time is taken to obtain the frequency domain signal of the fused PPG signal.
8. The method according to any one of claims 1 to 5, characterized in that: Also includes: In response to a switch of the enhancement function of vital sign detection being turned on, the number of channels is increased; or, In response to the vital sign detection the enhanced function switch is closed, reducing the number of channels.
9. The method according to any one of claims 1 to 5, characterized in that: The vital sign is heart rate.
10. The method according to any one of claims 1 to 5, characterized in that: The deep neural network is a deep sequence neural network, which includes a recurrent network layer for accumulating the time series of input feature vectors.
11. An electronic device, characterized in that: include: a processor, at least one light emitting diode, and a plurality of photodiodes; The light emitting diode is used to emit light to human tissue; The plurality of photodiodes are used to respectively receive light reflected by the human tissue and output photoplethysmography (PPG) signals of multiple channels; The processor is configured to execute the method according to any one of claims 1 to 10.
12. The electronic device according to claim 11, characterized in that: The number of the photodiodes is 8.
13. The electronic device according to any one of claims 11 to 12, characterized in that: The number of the light emitting diodes is 2.
14. The electronic device according to any one of claims 11 to 12, characterized in that: The plurality of photodiodes are arranged in a ring shape around the light emitting diode in the area where the electronic device contacts the human tissue.
15. The electronic device according to any one of claims 11 to 12, characterized in that: The electronic device is a smart watch.
16. A computer-readable storage medium, characterized in that: The method comprises instructions, which, when executed on an electronic device, cause the electronic device to execute the method according to any one of claims 1 to 10.
Citation Information
Cited By
Vital sign detection method and electronic device
WO2023071550A1
