Signal processing device, signal processing method, program, and learning device

Abstract

The heart rate can be estimated with high precision. The signal processing device estimates a first signal quality of the input biometric signal; estimating a second signal quality of the output signal from the noise reduction processor; estimating a heart rate based on the output signal from the noise reduction processor, and estimating a reliability of the estimated heart rate based on a result of the estimated first signal quality and a result of the estimated second signal quality; and correcting the estimated heart rate based on the reliability of the estimated heart rate. The present technology can be applied to a wearable cardiotachometer.

Description

Technical Field

[0001] The present technology relates to a signal processing device, a signal processing method, a program, and a learning device, and more particularly, to a signal processing device, a signal processing method, a program, and a learning device capable of estimating a heart rate with high accuracy.

[0002] <Cross-reference to related applications>

[0003] This application claims priority from Japanese Patent Application JP 2022-149751 filed on September 21, 2022 and priority from Japanese Patent Application JP 2023-094921 filed on June 8, 2023, the entire contents of which are incorporated herein by reference. Background Art

[0004] Currently, wearable devices equipped with heart rate measurement functions are widely used in various applications such as healthcare and heart rate training.

[0005] The heart rate measurement function mounted on a wearable device can use photoplethysmography (hereinafter referred to as the PPG method). The PPG method is a method in which light emitted from a light-emitting unit (e.g., a light-emitting diode (LED)) is absorbed, scattered, and reflected by blood and subcutaneous tissue a few millimeters below the skin, and the amount of reflected light is measured by a light-receiving unit (e.g., a photodetector), thereby measuring a signal indicating changes in blood flow of capillaries distributed under the skin (hereinafter referred to as a pulse wave signal).

[0006] Heart rate variability is a physiological index known to reflect autonomic nervous activity. In the PPG method, heart rate variability is analyzed by time series data during the time interval of the peak position of the pulse wave signal.

[0007] In the case of the PPG method, the pulse wave signal can be measured with relatively high accuracy in a resting state where the measurement site hardly moves, but when the measurement site moves, noise (hereinafter referred to as body motion noise) appears in the pulse wave signal. Body motion noise in a wristband-type PPG heart rate sensor (hereinafter referred to as a PPG sensor) is a change in the contact state between the PPG sensor and the measurement site, and the following are four main factors that cause body motion noise.

[0008] (1) Mixing of unwanted reflected light on the skin surface;

[0009] (2) Mixing of external light transmitted beneath the skin;

[0010] (3) Artifacts in blood flow changes due to movement of the measurement site, which can occur even when the contact between the PPG sensor and the measurement site is good; and

[0011] (4) Changes in light absorption due to deformation of subcutaneous tissue associated with movement of the fingers or wrist.

[0012] Due to the combined effect of these four factors, a false peak signal is mixed into the pulse wave signal, and it is difficult to determine which peak is the actual peak signal originating from the heartbeat or the false peak signal, which reduces the accuracy of heart rate estimation.

[0013] So far, various methods for reducing body motion noise (e.g., adaptive filters, frequency analysis, blind signal separation methods, etc.) have been proposed (see, e.g., PTL 1 and PTL 2). Generally, as a preprocessing of the noise reduction method, a bandpass filtering process that limits the pulse wave signal band is applied to the pulse wave signal, and the noise reduction method is further applied to the pulse wave signal after the bandpass filtering process.

[0014] Citation List

[0015] Patent Literature

[0016] PTL1: JP 2021-145930A

[0017] PTL2: JP 2021-503309A Summary of the invention

[0018] Technical issues

[0019] However, even when the peak intervals are correctly detected in the pulse wave signal after noise reduction processing, the error compared to the heart rate interval calculated by the electrocardiograph as a reference will still increase. This is because the signal-to-noise ratio (S / N) of the input signal is poor, and the peak position after the filtering process is shifted, even if the noise reduction process is performed as expected. Therefore, even when the time series data of the peak intervals (heart rate) of the pulse wave signal is used to analyze the heart rate variability, it will be greatly deviated from the analysis result of the heart rate variability obtained by the electrocardiograph as a reference, thereby reducing the accuracy of medical care applications using heart rate variability.

[0020] The present technology has been proposed in view of such circumstances, and can estimate the heart rate with high accuracy.

[0021] Solution to the problem

[0022] According to one aspect of the present technical content, a signal processing device includes a first signal quality estimator, which estimates a first signal quality of an input biometric signal; a second signal quality estimator, which estimates a second signal quality estimate of an output signal from a noise reduction processor; and a heart rate estimator, which estimates a heart rate based on the output signal from the noise reduction processor and estimates the reliability of the estimated heart rate based on a result of the estimated first signal quality and a result of the estimated second signal quality.

[0023] In one aspect of the present technology, the estimated first signal quality is composed of an input biometric signal, the estimated second signal quality is composed of an output signal from a noise reduction processor, the heart rate is estimated based on the output signal from the noise reduction processor, and the reliability of the estimated heart rate is based on the result of the estimated first signal quality and the result of the estimated second signal quality. Then, the heart rate is corrected based on the reliability estimated for the estimated heart rate.

[0024] According to another aspect of the present technical content, a learning device includes a first signal quality estimator, which estimates a first signal quality of an input biometric signal; a second signal quality estimator, which estimates a second signal quality of an output signal from a noise reduction processor; a heart rate estimator, which estimates the heart rate based on the output signal from the noise reduction processor and estimates the reliability of the heart rate based on the result of the estimated first signal quality and the result of the estimated second signal quality; a correction processor, which corrects the heart rate based on the reliability estimated for the estimated heart rate; and an estimation model learning unit, which uses the estimated heart rate and the reliability estimated for the estimated heart rate to learn an estimation model for estimating a person's emotional state.

[0025] In another aspect of the present technical content, a first signal quality estimate is formed by an input biometric signal, a second signal quality estimate is formed by an output signal from a noise reduction processor, a heart rate is estimated based on the output signal from the noise reduction processor, and a reliability of the estimated heart rate is estimated based on a result of the estimated first signal quality and a result of the estimated second signal quality. Then, the heart rate is corrected based on the reliability estimated for the estimated heart rate, and an estimation model learning unit learns an estimation model for estimating a person's emotional state using the estimated heart rate and the reliability estimated for the estimated heart rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] [ Figure 1 ] Figure 1 : is a block diagram showing a configuration example of a wearable heart rate meter according to a first embodiment of the present technology.

[0027] [ Figure 2 ] Figure 2 It shows Figure 1 FIG. 4 is a block diagram of a configuration example of a signal quality estimation unit (also referred to herein as a signal quality estimator) in FIG.

[0028] [ Figure 3 ] Figure 3 is a block diagram showing a configuration example of a signal quality estimation model learning unit.

[0029] [ Figure 4 ] Figure 4 is a block diagram showing a configuration example of a signal quality label generation unit (also referred to herein as a signal quality label generator).

[0030] [ Figure 5 ] Figure 5 is a diagram showing an example of a pulse wave signal.

[0031] [ Figure 6 ] Figure 6 is a graph showing the maximum correlation coefficient for the lag with the maximum correlation.

[0032] [ Figure 7 ] Figure 7 is a graph showing the lag at which the correlation is maximized over time.

[0033] [ Figure 8 ] Figure 8 It shows that according to Figure 1 Flowchart of processing of wearable heart rate meter signals for estimating heart rate.

[0034] [ Fig. 9 ] Fig. 9 AB shows an example of the reliability of the final output heart rate.

[0035] [ Fig.10 ] Fig.10 : is a block diagram showing an example of a wearable heart rate meter according to a second embodiment of the present technology.

[0036] [ Fig.11 ] Fig.11 It shows that according to Fig.10 Flowchart of processing of wearable heart rate meter signals for estimating heart rate.

[0037] [ Fig.12 ] Fig.12 It shows that according to Fig.10 Flow chart of a second processing of a wearable heart rate meter signal to estimate heart rate.

[0038] [ Fig.13 ] Fig.13is a block diagram showing an exemplary configuration of an apparatus for learning an arousal estimation model.

[0039] [ Fig.14 ] Fig.14 is a block diagram showing an exemplary configuration of a wakefulness estimator.

[0040] [ Fig.15 ] Fig.15 is a block diagram showing a configuration example of a computer. DETAILED DESCRIPTION

[0041] Hereinafter, a mode for implementing the present technical content will be described. The description is given in the following order.

[0042] 1. First Implementation Method

[0043] 2. Second Implementation

[0044] 3. Third Implementation Method

[0045] 4. Others

[0046] <1. First Embodiment>

[0047] <Configuration example of a wearable heart rate monitor>

[0048] Figure 1 : is a block diagram showing an exemplary configuration of a wearable heart rate meter according to a first embodiment of the present technology.

[0049] Figure 1 The wearable heart rate meter 11 may be included in a smart watch or the like equipped with a heart rate measurement function of the PPG method. Figure 1 In this case, the measurement site of the wearable heart rate monitor 11 can be the wrist.

[0050] Note that the measurement site is not limited to the wrist, and for example, in the case where the wearable heart rate meter 11 is an in-ear headphone or a headband, the ear may be the measurement site. In the case where the wearable heart rate meter 11 is a pair of virtual reality (VR) glasses, the forehead may be the measurement site. In the case where the wearable heart rate meter 11 is a band-type device, the arm or foot on which the band is worn may be the measurement site. In the case where the wearable heart rate meter 11 is a patch-type device, the chest may be the measurement site.

[0051] exist Figure 1In the embodiment, the wearable heart rate meter 11 includes a signal quality estimation unit (also referred to as a signal quality estimator in this article) 21, a noise reduction processing unit (also referred to as a noise reduction processor in this article) 22, a signal quality estimation unit (also referred to as a signal quality estimator in this article) 23, a cycle analysis unit (also referred to as a cycle analyzer in this article) 24, a heart rate estimation unit (also referred to as a heart rate estimator in this article) 25, a correction processing unit (also referred to as a correction processor in this article) 26 and an overall control unit (also referred to as an overall controller in this article) 27.

[0052] For example, in the wearable heart rate meter 11, the heart rate is measured by observing the pulse wave signal in the biometric signal.

[0053] Can be Figure 1 The wearable heart rate meter 11 measures a pulse wave signal, an acceleration signal, and an angular velocity (gyro) signal. The pulse wave signal, the acceleration signal, and the angular velocity (gyro) signal are provided to a signal quality estimation unit 21 and a noise reduction processing unit 22.

[0054] The signal quality estimation unit 21 estimates the signal quality of the pulse wave signal supplied from the preceding-stage (not shown) heart rate meter 11 , and supplies information indicating the estimated signal quality to the overall control unit 27 .

[0055] Note that, generally, when estimating the signal quality of a pulse wave signal, an S / N value is used as the heart rate band (S component) and other bands (N component). Since the heart rate band of the observed pulse wave signal is known, the heart rate band estimated from the pulse wave signal is used. However, in the case where body motion noise is mixed into the pulse wave signal, it is difficult to estimate the correct heart rate band.

[0056] In the signal quality estimation unit 21, a signal quality estimation (DNN) model is used, which is constructed by machine learning using a data set of pulse wave signals of various S / N values ​​generated in advance. The signal quality estimation model is a model that estimates a signal quality label (for example, two categories of good or bad). The signal quality label is defined by comparing the calculated heart rate with the heart rate measured by the electrocardiogram and determining whether the difference between the two is equal to or less than a preset error. This will be referred to later. Figure 2 The details of the signal quality estimation unit 21 are described.

[0057] The signal quality estimation unit 21 supplies information indicating the signal quality of the pulse wave signal (the signal quality label and the estimation value) to the overall control unit 27 .

[0058] The noise reduction processing unit 22 reduces the noise superimposed on the pulse wave signal according to the pulse wave signal, acceleration signal and angular velocity signal provided by the previous stage. For example, in the noise reduction processing unit 22, an adaptive filter is used to perform noise reduction, and the adaptive filter uses the acceleration signal and the angular velocity signal as noise reference signals. The noise reduction processing unit 22 outputs the pulse wave signal after the noise reduction processing to the signal quality estimation unit 23, the cycle analysis unit 24 and the heart rate estimation unit 25.

[0059] Note that although various methods have been proposed as noise reduction methods, the present technical content is a framework that does not rely on noise reduction processing, so that an arbitrary noise reduction method can be used in the noise reduction processing unit 22 .

[0060] The signal quality estimation unit 23 estimates the signal quality of the output signal from the noise reduction processing unit 22 .

[0061] Note that, although various noise reduction processes have been proposed as described above, it is difficult to completely separate noise, and therefore, the pulse wave signal after the noise reduction process may include residual noise.

[0062] For example, when noise reduction is performed by an adaptive filter, high-precision noise reduction can be performed when the frequency of body motion is constant. However, when the body motion is non-periodic and / or single motion, the optimal filter coefficient may not be obtained and complete noise reduction may not be obtained, and residual noise may occur.

[0063] Therefore, in order to evaluate the residual noise amount, the signal quality estimation unit 23 estimates the signal quality of the pulse wave signal after the noise reduction process (also referred to herein as the output signal from the noise reduction processing unit). At this time, a signal quality estimation model can be used.

[0064] The signal quality estimation unit 23 supplies information indicating the signal quality of the pulse wave signal after the noise reduction process (the signal quality label and the estimation value) to the overall control unit 27 .

[0065] The period analysis unit 24 analyzes the periodicity of the output signal from the noise reduction processing unit 22 with reference to the information indicating the signal quality (before and after the noise reduction processing) provided from the overall control unit 27. Since the heart rate has high periodicity, the period analysis unit 24 determines whether the periodicity of the pulse wave signal is high by using the characteristic that the periodicity of the pulse wave signal derived from the heart rate is high. The period analysis unit 24 outputs the analysis result of the periodicity to the overall control unit 27.

[0066] For example, the cycle analysis unit 24 performs cycle analysis using autocorrelation and detects the hysteresis when the correlation is maximized. The cycle analysis unit 24 determines that there is periodicity (high) when the maximum correlation coefficient is greater than a preset threshold value, and determines that there is no periodicity (low) when the maximum correlation coefficient value is less than the preset threshold value.

[0067] In addition, as another example of periodicity analysis, the lag with the maximum correlation can be stored, and it can be determined that there is periodicity when the difference in time series changes between the current lag and the past lag is less than a preset threshold, or it can be determined that there is no periodicity when the difference is less than the threshold.

[0068] The heart rate estimation unit 25 estimates the heart rate of the output signal from the noise reduction processing unit 22 with reference to the periodic analysis result supplied from the overall control unit 27 .

[0069] The heart rate estimation unit 25 detects the peak value of the pulse wave signal. The heart rate estimation unit 25 detects the peak value of the pulse wave and estimates the heart rate based on the peak interval which is the interval between the detected peak values. Although various methods have been proposed as peak detection methods, the present technical content is a framework that does not rely on the peak detection method, so that any peak detection method can be used in the heart rate estimation unit 25.

[0070] For example, when the peak interval is within the heart rate band of the person, the heart rate estimation unit 25 detects the peak interval based on maximum value detection and uses the detected maximum value as the peak position. In addition, for example, the heart rate estimation unit 25 can use the peak interval and the peak position with a similar lag detected by the cycle analysis unit 24 as candidates for the peak interval.

[0071] In addition, the heart rate estimation unit 25 estimates the reliability of the heart rate at the peak position based on the signal quality estimation results of the signal quality estimation unit 21 and the signal quality estimation unit 23. The signal quality before and after the noise reduction process is called the reliability estimate of the heart rate. For example, in the case where the signal quality labels of the signal quality estimation unit 21 and the signal quality estimation unit 23 at the peak position are both good (high quality), the reliability of the heart rate at the peak position is estimated to be high.

[0072] The present technology estimates the signal quality of the peak value of the pulse wave signal for each heartbeat to detect the heart rate from the pulse wave signal.

[0073] That is, in the present technical content, when estimating the reliability of the heart rate at the peak position in the detection of the peak value, it is not determined whether the peak intensity or peak interval is the intensity or band derived from the heartbeat, but whether the signal waveform of one heartbeat (each heartbeat) is derived from the heartbeat, that is, whether the reliability is high. With this arrangement, the reliability of the heart rate at the peak position can be estimated with high accuracy.

[0074] It should be noted that the heart rate estimation unit 25 can determine the reliability of the heart rate in consideration of the result of the period analysis. With this arrangement, in addition to the signal quality of the waveform, it is possible to determine whether the periodicity is the periodicity derived from the heartbeat, that is, whether the reliability of the periodicity is high, and therefore, the reliability of the heart rate can be estimated with high accuracy.

[0075] The heart rate estimation unit 25 outputs the estimated heart rate and the reliability of the estimated heart rate to the correction processing unit 26 .

[0076] The correction processing unit 26 performs correction processing on the heart rate supplied from the heart rate estimation unit 25 based on the reliability of the heart rate supplied from the heart rate estimation unit 25. For example, the correction processing unit 26 regards heart rate data whose reliability is less than or equal to a preset threshold value (i.e., the reliability is low) as a missing value, and compensates for the missing value using nearby (past or future) data having high reliability.

[0077] Specifically, for example, heart rate data with low reliability is compensated by pre-holding using the heart rate with high reliability that is closest in time or by linear interpolation using the nearby heart rate with high reliability. Since the heart rate with low reliability is a heart rate calculated from a pulse wave signal on which body motion noise may be superimposed, an erroneously detected heart rate is rejected and the accuracy of the final output heart rate is improved.

[0078] Note that the correction processing unit 26 may change the type of correction processing according to the degree of signal quality.

[0079] The correction processing unit 26 outputs the heart rate subjected to the correction processing and the reliability of the heart rate to a subsequent stage (not shown).

[0080] The overall control unit 27 controls data exchange between the various units.

[0081] <Configuration of Signal Quality Estimation Unit>

[0082] Figure 2 It shows Figure 1 FIG. 2 is a block diagram of an exemplary configuration of the signal quality estimation unit 21. FIG.

[0083] exist Figure 2 In the example, the signal quality estimation unit 21 includes an analysis window setting unit 31 , a time feature quantity calculation unit 32 , a frequency feature quantity calculation unit 33 , a signal quality estimation processing unit 34 , and a signal quality estimation model storage unit 35 .

[0084] The pulse wave signal, the acceleration signal, and the angular velocity signal are input to the analysis window setting unit 31. Typically, the feature quantity is extracted by an analysis window (sliding window) of about several seconds. The analysis window setting unit 31 sets an analysis window of, for example, four seconds for the input signal to output information about the set analysis window to the time feature quantity calculation unit 32 and the frequency feature quantity calculation unit 33.

[0085] The time feature amount calculation unit 32 calculates the feature amount of the time component within the analysis window based on the information about the analysis window supplied from the analysis window setting unit 31 , and outputs the calculated feature amount of the time component to the signal quality estimation processing unit 34 .

[0086] The frequency feature amount calculation unit 33 calculates the feature amount of the frequency component within the analysis window based on the information about the analysis window supplied from the analysis window setting unit 31 , and outputs the calculated feature amount of the frequency component to the signal quality estimation processing unit 34 .

[0087] The signal quality estimation processing unit 34 receives the feature quantity of the time component in the analysis window provided by the time feature quantity calculation unit 32 and the feature quantity of the frequency component in the analysis window provided by the frequency feature quantity calculation unit 33, estimates the signal quality using the signal quality estimation model, and outputs a signal quality label and an estimated value.

[0088] The signal quality estimation model storage unit 35 stores the signal quality estimation model used by the signal quality estimation processing unit 34. The signal quality estimation model is obtained by referring to Figure 3 The signal quality estimation model learning unit 51 described later performs learning and stores in the signal quality estimation model storage unit 35 .

[0089] <Signal quality estimation model learning unit>

[0090] Figure 3 is a block diagram showing an exemplary configuration of the signal quality estimation model learning unit 51 .

[0091] Figure 3 The signal quality estimation model learning unit 51 in the wearable heart rate meter 11 learns the signal quality estimation model stored in the signal quality estimation model storage unit 35. The signal quality estimation model learning unit 51 may be included in the wearable heart rate meter 11, or may be included in another signal processing device. Figure 3 In, with Figure 2 Corresponding components are denoted by the same reference numerals and their description will be omitted due to duplication.

[0092] The signal quality estimation model learning unit 51 includes an analysis window setting unit 31 , a time feature quantity calculation unit 32 , a frequency feature quantity calculation unit 33 , a signal quality estimation model learning unit 61 , and a data set storage unit 62 .

[0093] The data set of the pulse wave signal, acceleration signal, angular velocity signal (x) and signal quality label (y) is stored in the data set storage unit 62 and input to Figure 3 The analysis window setting unit 31.

[0094] The signal quality estimation model learning unit 61 learns a signal quality estimation model by using a signal quality label defined in advance for the data set of the data set storage unit 62 and taking as input the feature amount of the time component in the analysis window supplied from the time feature amount calculation unit 32 and the feature amount of the frequency component in the analysis window supplied from the frequency feature amount calculation unit 33. The learned signal quality estimation model is used for Figure 1 In the signal quality estimation units 21 and 23 in the etc.

[0095] The data set storage unit 62 stores data sets of pulse wave signals, acceleration signals, angular velocity signals (x), and signal quality labels (y), as well as signal quality labels (for example, good or bad) defined in advance for each data set. The signal quality labels are obtained by referring to Figure 4 The signal quality label generation unit (also referred to herein as a signal quality label generator) 71 described later is defined and stored in the data set storage unit 62 .

[0096] <Signal Quality Label Generation Unit>

[0097] Figure 4 is a block diagram showing an exemplary configuration of the signal quality label generation unit 71 .

[0098] For example, by performing various operations when performing simultaneous measurement using an electrocardiograph and the wearable heart rate meter 11 , a data set of pulse wave signals having various S / N values ​​is constructed in advance.

[0099] The data set of the pulse wave signal includes the pulse wave signal, the acceleration signal and the angular velocity signal measured by the wearable heart rate meter 11 and the heart rate measured by the electrocardiograph.

[0100] Figure 4 The signal quality label generation unit 71 defines a signal quality label for each data set, and stores the defined signal quality label together with each corresponding data set. Figure 5 The signal quality label generation unit 71 may be included in the wearable heart rate meter 11, or may be included in another signal processing device.

[0101] The signal quality label generation unit 71 includes a noise reduction processing unit 81 , a heart rate estimation unit 82 , an arithmetic unit 83 , and a comparison determination unit 84 .

[0102] The pulse wave signal, acceleration signal, and angular velocity signal of the data set are input to the noise reduction processing unit 81. The noise reduction processing unit 81 performs noise reduction processing on the pulse wave signal and outputs the signal to the heart rate estimation unit 82.

[0103] The heart rate estimation unit 82 detects a peak value, a peak interval, etc. from the pulse wave signal after the noise reduction process, and estimates the heart rate based on the detected peak interval. The heart rate estimation unit 82 outputs the estimated heart rate to the arithmetic unit 83.

[0104] The heart rate estimated by the heart rate estimation unit 82 and the reference heart rate of the data set are supplied to the arithmetic unit 83. The arithmetic unit 83 outputs the difference between the heart rate estimated by the heart rate estimation unit 82 and the reference heart rate to the comparison determination unit 84.

[0105] The comparison determination unit 84 defines a signal quality label (for example, two categories of good or bad) based on whether the difference provided from the arithmetic unit 83 is equal to or less than a preset error. That is, in the case where the difference is equal to or less than the preset error, the signal quality label is defined in the good category, and in the case where the difference is greater than the preset error, the signal quality label is defined in the bad category.

[0106] Next, an example in which the cycle analysis unit 24 performs cycle analysis using autocorrelation will be described.

[0107] Figure 5 is a diagram showing an example of a pulse wave signal.

[0108] exist Figure 5 In the figure, the vertical axis represents the pulse wave and the horizontal axis represents time. Figure 5 A basic window (solid line) set by the cycle analysis unit 24 and a reference window (dashed line) set at a position shifted in the past direction from the position of the basic window are shown.

[0109] Figure 6 is a graph showing the correlation coefficients for the lags with the largest correlation.

[0110] exist Figure 6 In , the vertical axis represents the correlation coefficient and the horizontal axis represents the lag.

[0111] like Figure 5 As shown, the period analysis unit 24 sets a basic window for the pulse wave signal and sets a reference window at a position shifted by the hysteresis in the past direction. Then, the correlation coefficient between the pulse wave signal of the set basic window and the pulse wave signal of the reference window is calculated, thereby detecting the hysteresis with the maximum correlation, as shown in FIG. Figure 6 shown.

[0112] Note that, in the case where the correlation coefficient of the lag having the maximum correlation (that is, the maximum correlation coefficient) is smaller than the preset threshold value, the period analysis unit 24 determines that the pulse wave signal does not have periodicity (low).

[0113] in addition, Figure 5 and Figure 6 The example is an example, and the cycle analysis unit 24 may be referred to later. Figure 7 Periodic analysis was performed as described.

[0114] Figure 7 is a graph showing the lag with the largest correlation over time.

[0115] exist Figure 7 In the example, the lag with the largest correlation is shown along the time change. For example, the lag calculated at the current time is included in the range of the mean ± standard deviation of the lag (data) with the largest correlation in the past.

[0116] like Figure 7 As shown, in the case where the lag calculated at the current time is included in the range of the mean ± standard deviation of the lag with the maximum correlation in the past, the period analysis unit 24 can determine that there is periodicity, and in the case where the lag is not included in the range of the mean ± standard deviation of the lag with the maximum correlation in the past, it can be determined that there is no periodicity.

[0117] <Handling of Wearable Heart Rate Monitor>

[0118] Figure 8 It shows Figure 1 Flow chart of the processing of the wearable heart rate meter 11.

[0119] The pulse wave signal, the acceleration signal, and the angular velocity signal are supplied to the signal quality estimation unit 21 and the noise reduction processing unit 22 .

[0120] In step S11 , the signal quality estimation unit 21 estimates the signal quality of the pulse wave signal supplied from the preceding stage, thereby outputting information indicating the signal quality (a signal quality label and an estimated value) to the overall control unit 27 .

[0121] In step S12, the noise reduction processing unit 22 reduces noise superimposed on the pulse wave signal, acceleration signal and angular velocity signal supplied from the previous stage. The noise reduction processing unit 22 outputs the pulse wave signal after the noise reduction processing to the signal quality estimation unit 23, the cycle analysis unit 24 and the heart rate estimation unit 25.

[0122] In step S13, the signal quality estimation unit 23 estimates the signal quality of the pulse wave signal after the noise reduction processing supplied from the noise reduction processing unit 22. The signal quality estimation unit 23 supplies information indicating the signal quality of the pulse wave signal after the noise reduction processing (signal quality label and estimation value) to the overall control unit 27.

[0123] In step S14, cycle analysis unit 24 analyzes the periodicity of the pulse wave signal after noise reduction processing supplied from noise reduction processing unit 22 with reference to information indicating signal quality supplied from overall control unit 27. Cycle analysis unit 24 outputs the analysis result of the periodicity to overall control unit 27.

[0124] In step S15, the heart rate estimation unit 25 estimates the heart rate from the pulse wave signal after the noise reduction processing provided by the noise reduction processing unit 22, with reference to the periodic analysis result provided by the overall control unit 27. In addition, the heart rate estimation unit 25 estimates the reliability of the heart rate at the peak position based on the signal quality estimation results of the signal quality estimation unit 21 and the signal quality estimation unit 23. At this time, as described above, the periodic analysis result provided by the overall control unit 27 may be referenced. The heart rate estimation unit 25 outputs the estimated heart rate and the reliability of the estimated heart rate to the correction processing unit 26.

[0125] In step S16, correction processing unit 26 performs correction processing on the estimated heart rate supplied from heart rate estimation unit 25 based on the reliability of the estimated heart rate supplied from heart rate estimation unit 25. Correction processing unit 26 outputs the correction-processed heart rate and the reliability of the heart rate to a subsequent stage (not shown).

[0126] Here, usually, outlier detection is performed based on statistical information on heart rate time series data, linear prediction, etc. However, due to the influence of the user's autonomic nervous state, etc., the heart rate fluctuates greatly and may be erroneously detected as an outlier.

[0127] On the other hand, in the present technical content, by using the reliability of determining whether the waveform shape of the input pulse wave signal and the pulse wave signal after correction processing originates from the heartbeat, the pulse rate will not be mistakenly detected as an abnormal value, and the abnormal value can be detected, so that the heart rate can be estimated with high accuracy.

[0128] Note that the reliability of the final heart rate output from the correction processing unit 26 is not limited to the good or bad signal quality label in the signal quality estimation results of the signal quality estimation units 21 and 23 .

[0129] <Example of reliability of final heart rate output>

[0130] Fig. 9 AB shows an example of the reliability of the final output heart rate.

[0131] Fig. 9 A shows a case where the logical product of the signal quality labels in the signal quality estimation results of the signal quality estimation units 21 and 23 is good, and the result of the cycle analysis of the cycle analysis unit 24 is high, wherein the reliability of the final output heart rate is 1.0.

[0132] It should be noted that only when the signal quality label in the signal quality estimation result of the signal quality estimation unit 21 is good and the signal quality label in the signal quality estimation result of the signal quality estimation unit 23 is good, the logical product of the signal quality labels in the signal quality estimation results of the signal quality estimation units 21 and 23 is expressed as good.

[0133] In the case where the logical product of the signal quality labels in the signal quality estimation results of the signal quality estimation units 21 and 23 is good, and the result of the cycle analysis of the cycle analysis unit 24 is low, it indicates that the reliability of the final output heart rate is 0.5.

[0134] In the case where the logical product of the signal quality labels in the signal quality estimation results of the signal quality estimation units 21 and 23 is bad and the result of the cycle analysis of the cycle analysis unit 24 is high, it indicates that the reliability of the final output heart rate is 0.5.

[0135] In the case where the logical product of the signal quality flags in the signal quality estimation results of the signal quality estimation units 21 and 23 is bad and the result of the cycle analysis of the cycle analysis unit 24 is low, it indicates that the reliability of the final output heart rate is 0.0.

[0136] Fig. 9 B shows a case where the logical product of the signal quality labels in the signal quality estimation results of the signal quality estimation units 21 and 23 is good, and the result of the cycle analysis of the cycle analysis unit 24 is high, wherein the reliability of the final output heart rate is 1.0.

[0137] When the logical product of the signal quality labels in the signal quality estimation results of the signal quality estimation units 21 and 23 is good and the result of the cycle analysis of the cycle analysis unit 24 is low, the reliability of the final output heart rate is indicated to be the autocorrelation maximum value (maximum correlation coefficient).

[0138] In the case where the logical product of the signal quality labels in the signal quality estimation results of the signal quality estimation units 21 and 23 is bad and the result of the cycle analysis of the cycle analysis unit 24 is high, it indicates that the reliability of the final output heart rate is the autocorrelation maximum value.

[0139] In the case where the logical product of the signal quality flags in the signal quality estimation results of the signal quality estimation units 21 and 23 is bad and the result of the cycle analysis of the cycle analysis unit 24 is low, it indicates that the reliability of the final output heart rate is 0.0.

[0140] As described above, as the reliability of the final output heart rate from the correction processing unit 26, not only the good or bad signal quality label in the signal quality estimation results of the signal quality estimation units 21 and 23 can be output, but also the value based on the results of the signal quality estimation units 21 and 23 and the cycle analysis unit 24 can be output.

[0141] <2. Second Embodiment>

[0142] <Configuration example of a wearable heart rate monitor>

[0143] Fig.10 : is a block diagram showing an exemplary configuration of a wearable heart rate meter according to a second embodiment of the present technology.

[0144] Fig.10 Wearable Heart Rate Monitors 101 and Figure 1 The wearable heart rate meter 11 is different in that a body movement context analysis unit (also referred to herein as a body movement context analyzer) 111 is added, and the signal quality estimation unit 21, the signal quality estimation unit 23, and the overall control unit 27 are replaced by a signal quality estimation unit 112, a signal quality estimation unit 113, and an overall control unit 114. Fig.10 In, with Figure 1 Corresponding components are denoted by the same reference numerals, and description thereof will be omitted.

[0145] The body movement context analysis unit 111 estimates (analyzes) what body movement state the user is in (sleeping, running, walking, sitting, etc.) based on the body movement information about the user.

[0146] Specifically, the body movement context analysis unit 111 estimates the user's ongoing activities and the body movement context indicating the user's body movement state based on sensor information (such as acceleration, angular velocity, atmospheric pressure, and geomagnetism) as body movement information about the user. The estimated body movement context is output to the overall control unit 114.

[0147] It should be noted that although various methods have been proposed as context analysis methods, the present technical content is a framework that does not rely on context analysis processing, so that any context analysis method can be used in the body movement context analysis unit 111.

[0148] The body movement context is also input to the signal quality estimation unit 112. At this time, the signal quality estimation unit 112 uses a signal quality estimation model that is constructed by machine learning using a data set of pulse wave signals of various S / N values ​​generated in advance and the body movement context as input. Other configurations of the signal quality estimation unit 112 are similar to those of the signal quality estimation unit 21. The same configuration can be applied to the signal quality estimation unit 113.

[0149] The overall control unit 114 controls the operation or stop of the processing of the noise reduction processing unit 22, the signal quality estimation unit 113 and the period analysis unit 24 based on the body movement context provided by the body movement context analysis unit 111, the signal quality estimation result of the signal quality estimation unit 112 and the signal quality estimation result after the noise reduction processing of the signal quality estimation unit 113, as indicated by the dotted arrows.

[0150] For example, the body movement context analysis unit 111 estimates whether there is body movement (determined as static or dynamic) and the strength (magnitude) of the body movement as the body movement context based on a threshold value set in advance from the norm value of the acceleration sensor. Whether there is body movement is determined by comparing the magnitude of the body movement with a preset threshold value α (α is a small value close to 0), and when the magnitude of the body movement is less than the predetermined threshold value α, it is determined that there is no body movement.

[0151] In this case, the overall control unit 114 calculates in advance the limit performance of the noise reduction processing relative to the input signal quality and the body movement intensity, for example, based on the body movement intensity estimated by the body movement context analysis unit 111, the signal quality estimation result of the signal quality estimation unit 112, and the signal quality estimation result after noise reduction processing by the signal quality estimation unit 113.

[0152] When it is determined that the body movement intensity estimated by the body movement context analysis unit 111 exceeds the limit performance of the noise reduction processing, the reliability estimated by the heart rate estimation unit 25 remains almost unchanged even if post-noise reduction processing (at least one of noise reduction processing, signal quality estimation after noise reduction processing, or period analysis) is performed.

[0153] Therefore, in the case where it is determined that the body motion intensity estimated by the body motion situation analysis unit 111 exceeds the limit performance of the body motion noise reduction processing, the overall control unit 114 controls to stop the above-mentioned processing after the noise reduction processing, at least the period analysis processing (see Fig.11 ). This can reduce the computational cost and power consumption of the framework as a whole.

[0154] In addition, when the body movement context analysis unit 111 estimates that the body movement is less than the predetermined threshold α, and the signal quality estimation unit 112 estimates that the signal quality is better (higher) than the predetermined threshold β, it can be clearly seen that noise is hardly superimposed on the input pulse wave.

[0155] Therefore, the overall control unit 114 stops the processing of the noise reduction processing unit 12, the signal quality estimation unit 113, and the period analysis unit 24 (see Fig.12 ). This can reduce the computational cost and power consumption of the framework as a whole.

[0156] <First processing of wearable heart rate monitor>

[0157] Fig.11 It is used to illustrate Fig.10 Flowchart of the first process of the wearable heart rate meter 101.

[0158] The pulse wave signal, the acceleration signal, and the angular velocity signal are supplied to the signal quality estimation unit 112 and the noise reduction processing unit 22. In addition, sensor information such as acceleration, angular velocity, atmospheric pressure, and geomagnetism is input to the body movement context analysis unit 111.

[0159] In step S111 , the body movement context analysis unit 111 receives sensor information as body movement information about the user, performs body movement context analysis, and estimates body movement intensity, and outputs the estimated body movement intensity to the overall control unit 27 .

[0160] In step S112, the signal quality estimation unit 112 estimates the signal quality of the pulse wave signal supplied from the preceding stage, thereby outputting information indicating the signal quality (signal quality label and estimation value) to the overall control unit 114. At this time, as described above, the body movement context supplied from the overall control unit 114 may be used as input.

[0161] In step S113, the overall control unit 114 determines whether the body movement intensity provided from the body movement situation analysis unit 111 exceeds the limit performance of the noise reduction process. When it is determined in step S113 that the body movement intensity does not exceed the limit performance of the noise reduction process, the process proceeds to step S114.

[0162] Since the processing of steps S114 to S118 is the same as Figure 8 The processes of steps S12 to S16 are basically similar, so description thereof will be omitted.

[0163] When it is determined in step S113 that the body motion intensity exceeds the limit performance of the noise reduction processing, steps S114 to S116 are skipped and the processing proceeds to step S117. That is, since there is no point in performing the processing, each processing of the noise reduction processing unit 22, the signal quality estimation unit 113 and the cycle analysis unit 24 is stopped.

[0164] In the case where steps S114 to S116 are skipped, in step S117, the heart rate estimation unit 25 estimates the heart rate from the pulse wave signal which is supplied from the noise reduction processing unit 22 and which has not been subjected to the noise reduction processing. In addition, the heart rate estimation unit 25 estimates the reliability of the heart rate at the detected peak position based on the signal quality estimation result of the signal quality estimation unit 112. The heart rate estimation unit 25 outputs the estimated heart rate and the reliability of the estimated heart rate to the correction processing unit 26.

[0165] Then, in step S118, the correction processing unit 26 performs correction processing on the heart rate based on the reliability of the estimated heart rate supplied from the heart rate estimation unit 25. The correction processing unit 26 outputs the correction-processed heart rate and the reliability of the heart rate to a subsequent stage (not shown).

[0166] That is, even if steps S117 and S118 are performed in a case where it is determined that the body motion intensity exceeds the limit performance of the noise reduction processing, a bad signal quality label is output.

[0167] As described above, since the processing of the noise reduction processing unit 22, the signal quality estimation unit 113, and the period analysis unit 24 is skipped according to the body motion context, the calculation cost and power consumption of the framework (processing) as a whole can be reduced.

[0168] <The second process of the wearable heart rate monitor>

[0169] Fig.12 It shows Fig.10 Flow chart of a second process of the wearable heart rate meter 101.

[0170] The pulse wave signal, the acceleration signal, and the angular velocity signal are supplied to the signal quality estimation unit 112 and the noise reduction processing unit 22. In addition, sensor information such as acceleration, angular velocity, atmospheric pressure, and geomagnetism is input to the body movement context analysis unit 111.

[0171] In step S151 , the body movement context analysis unit 111 receives sensor information as body movement information about the user, performs body movement context analysis, and estimates the presence or absence of body movement. The estimated presence or absence of body movement is output to the overall control unit 114 .

[0172] In step S152 , the signal quality estimation unit 112 estimates the signal quality of the pulse wave signal supplied from the preceding stage to output information indicating the signal quality (a signal quality label and an estimated value) to the overall control unit 114 .

[0173] In step S153, the overall control unit 114 determines whether no body movement has occurred or whether the signal quality is high. In the case where the body movement provided from the body movement context analysis unit 111 is equal to or greater than the predetermined threshold α or the signal quality is equal to or less than the predetermined threshold β, it is determined in step S153 that body movement has occurred or the signal quality is poor, and the process proceeds to step S154.

[0174] Since the processing of steps S154 to S158 is the same as Figure 8 The processes of steps S12 to S16 are basically similar, so description thereof will be omitted.

[0175] When the body motion is less than the predetermined threshold value α and the signal quality is better than the predetermined threshold value β in step S153, it is determined that the body motion does not occur and the signal quality is high, steps S154 to S156 are skipped, and the process proceeds to step S157. That is, since the signal quality is high, each process of the noise reduction processing unit 22, the signal quality estimation unit 113, and the period analysis unit 24 is stopped.

[0176] In the case where steps S154 to S156 are skipped, in step S157, the heart rate estimation unit 25 estimates the heart rate from the pulse wave signal which is supplied from the noise reduction processing unit 22 and which has not been subjected to the noise reduction processing. Furthermore, the heart rate estimation unit 25 estimates the reliability of the heart rate at the detected peak position based on the signal quality estimation result of the signal quality estimation unit 112. The heart rate estimation unit 25 outputs the estimated heart rate and the reliability of the estimated heart rate to the correction processing unit 26.

[0177] Then, in step S158, the correction processing unit 26 performs correction processing on the heart rate based on the reliability of the heart rate provided from the heart rate estimation unit 25. The correction processing unit 26 outputs the heart rate after the correction processing and the reliability of the heart rate to the subsequent stage (not shown). In this case, since the signal quality is good, a signal quality label corresponding to good is output or good is output.

[0178] As described above, since the processing of the noise reduction processing unit 22, the signal quality estimation unit 113, and the period analysis unit 24 is skipped according to the body motion context, the calculation cost and power consumption of the framework (processing) as a whole can be reduced.

[0179] Please note that Fig.11 and Fig.12, an example of skipping the processing of the noise reduction processing unit 22, the signal quality estimation unit 113, and the cycle analysis unit 24 is described, but instead of skipping all three processes, at least one of the noise reduction processing unit 22, the signal quality estimation unit 113, and the cycle analysis unit 24 may be skipped.

[0180] In addition, Figure 1 Wearable heart rate monitors 11 and Fig.10 In the wearable heart rate monitor 101, the correction processing unit 26 is not required, and the wearable heart rate monitor 101 can be configured without the correction processing unit 26. Figure 1 Wearable heart rate monitors 11 and Fig.10 Wearable heart rate monitors 101.

[0181] Note that, in the above description, an example of a pulse wave signal is described, but the present technical content can be applied not only to a pulse wave but also to a biometric signal with high periodicity, such as blood flow or continuous blood pressure.

[0182] <3. Third Implementation>

[0183] The third embodiment describes an example in which the beat-by-beat heart rate (ie, heart rate variability, hereinafter referred to as instantaneous heart rate) estimated more accurately in the above-mentioned first and second embodiments and the reliability of the instantaneous heart rate are used to estimate the arousal level.

[0184] <Exemplary Configuration of Apparatus for Learning Arousal Estimation Model>

[0185] Fig.13 : is a block diagram showing an exemplary configuration of the arousal degree estimation model learning device 201 according to the third embodiment of the present technology.

[0186] Fig.13 The device 201 for learning the arousal estimation model shown in the figure is a learning device for learning the arousal estimation model, which is a machine learning model that uses the instantaneous heart rate and reliability level estimated from the pulse wave signal in the above-mentioned first and second embodiments to estimate the arousal, i.e., one of the user's emotional states, from the pulse wave signal.

[0187] The apparatus 201 for learning an arousal estimation model is configured to include a data set storage unit 211 , an instantaneous heart rate estimation unit 212 , a reliability conversion unit 213 , and an arousal estimation model learning unit 214 .

[0188] The data set storage unit 211 stores the arousal level label (Y i = X at high (1) or low (0) i= n samples of the data set of [pulse wave signal, acceleration signal, gyroscope signal].

[0189] The instantaneous heart rate estimation unit 212 is equivalent to a device for estimating the instantaneous heart rate, such as Figure 1 Wearable heart rate monitor 11 or Fig.10 Wearable heart rate monitors 101.

[0190] That is, the instantaneous heart rate estimation unit 212 uses the X provided from the data set storage unit 211. i As input Figure 1 Wearable heart rate monitors in 11 or Fig.10 Wearable Heart Rate Monitors 101 i The instantaneous heart rate estimation unit 212 calculates the instantaneous heart rate IHR i The confidence level of the instantaneous heart rate reliability is output to the arousal estimation model training unit 214 , and the confidence level of the instantaneous heart rate reliability is output to the reliability conversion unit 213 .

[0191] The reliability conversion unit 213 uses the reliability supplied from the instantaneous heart rate estimation unit 212 as an input, and uses a preset LUT (lookup table) or a conversion function (e.g., a linear function or a nonlinear function such as a sigmoid function) to calculate the signal quality r used in the arousal degree estimation model learning unit 214. i The reliability conversion unit 213 calculates the signal quality r using a predefined LUT (lookup table) or a conversion function (eg, a nonlinear function such as a linear function or a sigmoid function). i The reliability conversion unit 213 converts the calculated signal quality r i Output to the arousal level estimation model learning unit 214.

[0192] The arousal level estimation model learning unit 214 uses the instantaneous heart rate IHR provided from the instantaneous heart rate estimation unit 212. i and the signal quality r provided from the reliability conversion unit 213 i To learn an arousal estimation model for estimating arousal level labels (0 or 1).

[0193] As an example of a learning method performed in the arousal estimation model learning unit 214 , a binary classification model learning method using a DNN is described.

[0194] Typically, the logarithmic loss function LBCE for each data used to train a binary classification model does not take into account the signal quality of the data used for training, which is the reason for the decrease in model accuracy.

[0195] Therefore, in the arousal estimation model learning unit 214, the signal quality r is used.i The weighted logarithmic loss function L loss , as shown in Equation (1) below, so that when learning the model, data with higher signal quality contributes more (to the error).

[0196] <Equation 1>

[0197]

[0198] Among them, Y i is the true value of the arousal model, Y with ^ (hat symbol) i is the predicted value (probability value) of the arousal label estimated by the model. Moreover, r i is the signal quality (low 0.0 to high 1.0). The higher the signal quality, the greater the sample contribution during model training.

[0199] In other words, the arousal estimation model learning unit 214 repeats the learning of the arousal estimation model so that the higher the quality, the larger the error assigned, and minimizes the error by changing the model coefficients, thereby making the cost smaller. This can suppress the decrease in model accuracy. The example described here is based on a logarithmic loss function, but is not limited to this.

[0200] The arousal level estimation model learned by the arousal level estimation model learning unit 214 is used for arousal level estimation described next.

[0201] <Exemplary Configuration of Arousal Degree Estimation Device>

[0202] Fig.14 : is a block diagram showing an exemplary configuration of the awakening degree estimation device 251 according to the third embodiment of the present technology.

[0203] exist Fig.14 In the awakening degree estimation device 251 shown in FIG. Fig.13 Arousal level estimation is performed using the arousal level estimation model learned by the arousal level estimation model learning device 201 shown in FIG.

[0204] The awakening degree estimation device 251 is configured to include Fig.13 The instant heart rate estimation unit 212, the reliability conversion unit 213, the arousal degree estimation model storage unit 261 and the arousal degree estimation unit 262.

[0205] The instantaneous heart rate estimation unit 212 uses, for example, X provided by the sensor. i As input Fig.13 Estimate instantaneous heart rate IHR as in i The instantaneous heart rate estimator 212 calculates the instantaneous heart rate IHR iThe heart rate is output to the arousal degree estimator 262 , and the reliability of the instantaneous heart rate is output to the reliability converter 213 .

[0206] like Fig.13 As shown, the reliability conversion unit 213 uses the reliability provided from the instantaneous heart rate estimation unit 212 as an input, and calculates the signal quality r used in the awakening degree estimation unit 262 using a preset LUT (lookup table) and a conversion function. i The reliability conversion unit 213 converts the calculated signal quality r i Output to the awakening degree estimation unit 262.

[0207] The arousal estimation model storage unit 261 stores the Fig.13 The arousal level estimation model learned by the arousal level estimation model learning device 201 shown in FIG.

[0208] The arousal level estimation unit 262 uses the instantaneous heart rate IHR provided by the instantaneous heart rate estimation unit 212 i and the signal quality r provided by the reliability conversion unit 213 i As an input, the arousal level is estimated using the arousal estimation model read from the arousal estimation model storage unit 261 .

[0209] Specifically, the awakening degree estimation unit 262 determines the signal quality r according to the signal quality r. i For example, the wake-up estimation unit 262 performs weighted prediction by comparing the input X i The arousal level is estimated by weighting the m predictions that are close in time, as shown in Equation (2) below.

[0210] <Equation 2>

[0211]

[0212] Where i represents the current time to be predicted, and j represents the time close to time i.

[0213] The awakening estimation unit 262 outputs the estimated awakening level to the subsequent stage. The estimated awakening level is used in the subsequent stage for an application requiring awakening level information.

[0214] As described above, in the third embodiment, the heart rate and the reliability of the heart rate estimated more accurately in the above-mentioned first and second embodiments are used for arousal estimation. This makes it possible to estimate the arousal level more accurately.

[0215] exist Fig.13 Arousal estimation model learning device 201 and Fig.14In the awakening estimation device 251, the reliability conversion unit 213 may be excluded. In this case, the reliability is used instead of the signal quality r in the awakening estimation model learning unit 214 and the awakening estimation unit 262. i .

[0216] In the above description, the instantaneous heart rate estimated by the present technology and its reliability are used to estimate the arousal level, which is one of the emotional states of a person, that is, the concentration and relaxation states, which is the vertical axis direction described in Russell's circumplex model. When the reliability of the input information can be defined, the present technology can also be used to estimate the pleasant and unpleasant states on the horizontal axis described by Russell's circumplex model.

[0217] <4. Others>

[0218] <Overview of Related Art and Effects of the Present Technology>

[0219] As described above, in the related art, even if the noise reduction process is performed as expected, the peak position after the filtering process may be shifted due to the poor S / N of the input signal. In this case, even in the case of analyzing the heart rate variability using the time series data of the peak interval of the pulse wave signal, the analyzed heart rate variability has a large deviation from the analysis result of the heart rate variability obtained by the electrocardiograph as a reference, and the accuracy of the medical care application using the heart rate variability is reduced.

[0220] PTL 1 proposes calculating the noise intensity based on the spectrum analysis of the pulse wave signal and calculating the index based on whether the quality of the pulse wave signal is equal to or higher than a reference value. However, in the technology described in PTL 1, since the quality of each peak of the pulse wave signal cannot be evaluated, the detection accuracy of the heart rate by peak detection is reduced.

[0221] In addition, PTL 2 proposes to separate the pulse wave signal to obtain a plurality of sub-signal segments, and determine whether it is noise based on the self-similarity in the pulse wave signal of the sub-signal segment. However, for example, in strong periodic movements such as jogging and running, body movement is periodically and strongly superimposed on the pulse wave signal as body movement noise. Therefore, the self-similarity increases, and the body movement is erroneously detected as a pulse wave signal due to heartbeat. In addition, even in the case of a pulse wave signal after a self-similarity analysis is applied to a noise reduction process, it is difficult to completely reduce the noise by the noise reduction process, and therefore, the remaining periodic noise is a factor of erroneous determination by the self-similarity analysis.

[0222] In one aspect of the present technology, a first signal quality estimation is performed on an input biometric signal, and a second signal quality estimation is performed on a biometric signal after noise reduction processing. Then, a heart rate is estimated based on the biometric signal after noise reduction processing, and the reliability of the heart rate is estimated based on the result of the first signal quality estimation and the result of the second signal quality estimation.

[0223] Therefore, in one aspect of the present technical content, by using the result of the first signal quality estimation and the result of the second signal quality estimation, it is determined whether the signal waveform of each heartbeat originates from the heartbeat. Therefore, the reliability of the heart rate (peak detection) at the detected peak position can be estimated with high accuracy. That is, in estimating the heart rate from the pulse wave signal, the signal quality of the peak value of the pulse wave signal of each heartbeat can be estimated. With this configuration, the heart rate can be estimated with high accuracy.

[0224] In another aspect of the present technical content, an estimation model for estimating the emotional state of a person is learned using the heart rate estimated by the above-mentioned aspect of the present technical content and the reliability of the heart rate.

[0225] This enables a highly accurate estimation of a person's emotional state.

[0226] <Computer Configuration Example>

[0227] The above-described series of processes can be executed by hardware or software. In the case where the series of processes are executed by software, a program constituting the software is installed from a program recording medium to a computer incorporated in dedicated hardware, a general-purpose personal computer, or the like.

[0228] Fig.15 : is a block diagram showing a configuration example of hardware of a computer that executes the above-described series of processes by a program.

[0229] A central processing unit (CPU) 301 , a read only memory (ROM) 302 , and a random access memory (RAM) 303 are connected to each other via a bus 304 .

[0230] An input / output interface 305 is further connected to the bus 304. An input unit 306 including a microphone, a keyboard, a mouse, etc. and an output unit 307 including a display, a speaker, etc. are connected to the input / output interface 305. In addition, a storage unit 308 including a hard disk, a nonvolatile memory, etc., a communication unit 309 including a network interface, etc., and a drive 310 that drives a removable medium 311 are connected to the input / output interface 305.

[0231] In the computer configured as above, for example, the CPU 301 loads a program stored in the storage unit 308 into the RAM 303 via the input / output interface 305 and the bus 304 and executes the program, thereby performing the above-described series of processing.

[0232] The program executed by the CPU 301 is, for example, provided by being recorded in the removable medium 311 or via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting, and installed in the storage unit 308 .

[0233] Note that the program executed by the computer may be a program that executes processing in time series in the order described in this specification, or may be a program that executes processing in parallel or at necessary timing such as when a call is made.

[0234] Note that in this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all components are in the same housing. Therefore, multiple devices housed in their own housings and connected via a network, as well as a device that houses multiple modules in one housing, are both systems.

[0235] In addition, the effects described in this specification are merely examples and are not limited, and other effects may be provided.

[0236] The implementation of the present technical content is not limited to the above-mentioned embodiment, and various modifications can be made without departing from the gist of the present technical content.

[0237] For example, the present technical content may have a configuration of cloud computing in which one function is shared and cooperatively processed by a plurality of devices via a network.

[0238] In addition, each step described in the above flowchart may be performed by one device or may be shared and performed by a plurality of devices.

[0239] Furthermore, in the case where a plurality of processes are included in one step, the plurality of processes included in one step may be executed by one device or may be shared and executed by a plurality of devices.

[0240] <Configuration combination example>

[0241] Note that the present technical content may have the following configurations. (1)

[0243] a first signal quality estimator that estimates a first signal quality of an input biometric signal;

[0244] a second signal quality estimator that estimates a second signal quality of an output signal from the noise reduction processor; and

[0245] A heart rate estimator estimates the heart rate based on the output signal from the noise reduction processor and estimates the reliability of the estimated heart rate based on a result of the estimated first signal quality and a result of the estimated second signal quality. (2)

[0247] The signal processing device according to (1), wherein

[0248] The heart rate estimator detects a peak position of an output signal from a noise reduction processor and estimates the reliability of an estimated heart rate at the detected peak position. (3)

[0250] The signal processing device according to (1) or (2), wherein the heart rate estimation unit estimates the reliability of the estimated heart rate to be high when the signal quality in the result of the first signal quality estimation and the signal quality in the result of the second signal quality estimation are high. (4)

[0252] The signal processing device according to (2) or (3), further comprising:

[0253] A periodic analysis unit, the periodic analysis unit performing periodic analysis on the biometric signal after the noise reduction process;

[0254] The heart rate estimation unit estimates the reliability of the estimated heart rate based on the result of the first signal quality estimation, the result of the second signal quality estimation, and the result of the periodic analysis of the biometric signal after the noise reduction process. (5)

[0256] The signal processing device according to (4) further comprises:

[0257] A noise reduction processing unit is used to perform noise reduction processing on the biometric signal. (6)

[0259] The signal processing device according to (5) further comprises:

[0260] A body movement state analysis unit analyzes a user's body movement state based on body movement information about the user acquired by a sensor. (7)

[0262] The signal processing device according to (6), wherein

[0263] When it is analyzed that the body movement of the user is greater than the first threshold, at least one of the noise reduction processing unit, the second signal quality estimation unit or the period analysis unit stops processing. (8)

[0265] The signal processing device according to (6), wherein

[0266] When it is analyzed that the signal quality is high in the result of the first signal quality estimation and the user's body movement is less than the second threshold, at least one of the noise reduction processing unit, the second signal quality estimation unit or the period analysis unit stops processing. (9)

[0268] The signal processing device according to any one of (1) to (8), further comprising:

[0269] A correction processing unit corrects the estimated heart rate based on the reliability estimated for the estimated heart rate. (10)

[0271] The signal processing device according to any one of (1) to (9), wherein the biometric signal is a pulse wave signal. (11)

[0273] The signal processing device according to any one of (1) to (10), wherein the signal processing device is provided in a wearable housing. (12)

[0275] A signal processing method performed by a signal processing device, the method comprising:

[0276] performing a first signal quality estimation of an input biometric signal;

[0277] estimating a second signal quality of the biometric signal after noise reduction processing;

[0278] estimating heart rate based on the biometric signal after noise reduction processing; and

[0279] The reliability of the estimated heart rate is estimated based on a result of the first signal quality estimation and a result of the second signal quality estimation. (13)

[0281] A program that causes a computer to:

[0282] a first signal quality estimation unit, which performs a first signal quality estimation on an input biometric signal;

[0283] a second signal quality estimation unit, which performs second signal quality estimation on the biometric signal after noise reduction processing; and

[0284] A heart rate estimation unit estimates the heart rate based on the biometric signal after the noise reduction process, and estimates the reliability of the estimated heart rate based on a result of the first signal quality estimation and a result of the second signal quality estimation. (14)

[0286] A learning device, comprising:

[0287] a first signal quality estimation unit, which performs a first signal quality estimation on an input biometric signal;

[0288] a second signal quality estimation unit, which estimates the second signal quality of the biometric signal after noise reduction processing;

[0289] a heart rate estimation unit, which estimates the heart rate based on the biometric signal after the noise reduction process, and estimates the reliability of the estimated heart rate based on a result of the first signal quality estimation and a result of the second signal quality estimation;

[0290] a correction processing unit that corrects the estimated heart rate based on the reliability estimated for the estimated heart rate; and

[0291] An estimation model learning unit learns an estimation model for estimating an emotional state of a person using the estimated heart rate and the reliability of the estimated heart rate. (15)

[0293] A signal processing device, comprising:

[0294] a first signal quality estimator that estimates a first signal quality of an input biometric signal;

[0295] a second signal quality estimator that estimates a second signal quality of an output signal from the noise reduction processor; and

[0296] A heart rate estimator estimates the heart rate based on the output signal from the noise reduction processor and estimates the reliability of the estimated heart rate based on a result of the estimated first signal quality and a result of the estimated second signal quality. (16)

[0298] The signal processing device according to (15), wherein

[0299] The heart rate estimator detects a peak position of an output signal from a noise reduction processor and estimates the reliability of an estimated heart rate at the detected peak position. (17)

[0301] The signal processing device according to (15) or (16), wherein the heart rate estimator estimates the reliability of the heart rate to be high when the signal quality in the result of the first signal quality estimation and the signal quality in the result of the second signal quality estimation are high. (18)

[0303] The signal processing device according to (16) or (17), further comprising:

[0304] A periodic analysis unit performs periodic analysis on the biometric signal after noise reduction processing, wherein the heart rate estimation unit estimates the reliability of the heart rate based on the result of the first signal quality estimation, the result of the second signal quality estimation and the result of the periodic analysis of the biometric signal after noise reduction processing. (19)

[0306] The signal processing device according to (18), further comprising:

[0307] A noise reduction processing unit is used to perform noise reduction processing on the biometric signal.

[0308] (20) The signal processing device according to (19), further comprising:

[0309] A body movement state analysis unit analyzes a user's body movement state based on body movement information about the user acquired by a sensor. (twenty one)

[0311] The signal processing device according to (20), wherein

[0312] When it is analyzed that the body movement of the user is greater than the first threshold, at least one of the noise reduction processing unit, the second signal quality estimation unit or the period analysis unit stops processing. (twenty two)

[0314] The signal processing device according to (20), wherein

[0315] When it is analyzed that the signal quality in the result of the first signal quality estimation is high and the user's body movement is less than the second threshold, at least one of the noise reduction processing unit, the second signal quality estimation unit or the period analysis unit stops processing. (twenty three)

[0317] The signal processing device according to any one of (15) to (22), further including a correction processing unit that corrects the estimated heart rate based on reliability for the heart rate estimation. (twenty four)

[0319] The signal processing device according to any one of (15) to (23), wherein the biometric signal is a pulse wave signal. (25)

[0321] The signal processing device according to any one of (15) to (24), wherein the signal processing device is provided in a wearable housing. (26)

[0323] A signal processing method performed by a signal processing device, the method comprising:

[0324] performing a first signal quality estimation of an input biometric signal;

[0325] a second signal quality estimation of the biometric signal after noise reduction processing; and

[0326] The heart rate is estimated based on the biometric signal after the noise reduction process, and the reliability of the heart rate is estimated based on the result of the first signal quality estimation and the result of the second signal quality estimation. (27)

[0328] A program that causes a computer to function as

[0329] a first signal quality estimation unit, which performs a first signal quality estimation on an input biometric signal;

[0330] a second signal quality estimation unit, which performs second signal quality estimation on the biometric signal after noise reduction processing; and

[0331] A heart rate estimation unit estimates the heart rate based on the biometric signal after the noise reduction process, and estimates the reliability of the heart rate based on a result of the first signal quality estimation and a result of the second signal quality estimation. (28)

[0333] A learning device, comprising:

[0334] a first signal quality estimation unit, which performs a first signal quality estimation on an input biometric signal;

[0335] a second signal quality estimation unit, which estimates the second signal quality of the biometric signal after noise reduction processing;

[0336] a heart rate estimation unit, which estimates the heart rate based on the biometric signal after noise reduction processing, and estimates the reliability of the heart rate based on a result of the first signal quality estimation and a result of the second signal quality estimation;

[0337] a correction processing unit that corrects the heart rate based on the reliability of the heart rate estimate; and

[0338] An estimation model learning unit that learns an estimation model for estimating an emotional state of a person using the estimated heart rate and the reliability of the estimated heart rate. (29)

[0340] The signal processing method according to (26) further comprises:

[0341] The body movement of the user is determined based on the body movement information about the user acquired by the sensor. (30)

[0343] According to the signal processing method described in (26) and (29),

[0344] When the result of the estimated first signal quality is higher and the result of the estimated second signal quality is higher, the reliability of the estimated heart rate is higher. (31)

[0346] The signal processing method according to (26) and (29) to (30) further comprises:

[0347] Performing periodic analysis on the input biometric signal after the noise of the input biometric signal has been reduced,

[0348] The reliability of the estimated heart rate is based on a result of the first signal quality estimation, a result of the second signal quality estimation, and a result of a period analysis of the input biometric signal after noise of the input biometric signal is reduced. (32)

[0350] According to the signal processing method described in (26) and (29) to (31),

[0351] Reducing the noise of the input biometric signal includes a noise reduction processor that performs noise reduction processing on the input biometric signal. (33)

[0353] The signal processing method according to (26) and (29) to (32) further comprises:

[0354] The estimated heart rate is corrected based on a reliability of the estimated heart rate. (34)

[0356] The signal processing method according to (26) and (29) to (33), wherein the input biometric signal includes a pulse wave signal.

[0357] It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and alterations may occur depending on design requirements and other factors, as long as they fall within the scope of the appended claims or the equivalents thereof.

[0358] Reference numerals list

[0359] 11. Wearable heart rate monitor

[0360] 21 Signal quality estimation unit

[0361] 22 Noise Reduction Processing Units

[0362] 23 Signal quality estimation unit

[0363] 24 Cycle Analysis Unit

[0364] 25 Heart rate estimation unit

[0365] 26 Correction Processing Unit

[0366] 27 Overall control unit

[0367] 31 Analysis window setting unit

[0368] 32 Time feature quantity calculation unit

[0369] 33 Frequency feature quantity calculation unit

[0370] 34 Signal quality estimation processing unit

[0371] 35 Signal quality estimation model storage unit

[0372] 51 Signal Quality Estimation Model Learning Unit

[0373] 61 Signal Quality Estimation Model Learning Unit

[0374] 62 Dataset storage unit

[0375] 71 Signal quality label generation unit

[0376] 81 Noise Reduction Processing Unit

[0377] 82 Heart rate estimation unit

[0378] 83 Arithmetic unit

[0379] 84 Comparison determination unit

[0380] 101 Wearable Heart Rate Monitor

[0381] 111 Body Movement Situation Analysis Unit

[0382] 112 Signal quality estimation unit

[0383] 113 Signal quality estimation unit

[0384] 114 Overall control unit

[0385] 201 Device for learning arousal estimation model

[0386] 211 Dataset Memory

[0387] 212 Instantaneous heart rate estimator

[0388] 213 Reliability Converter

[0389] 214 Arousal estimation model learning unit

[0390] 251 Arousal estimation device

[0391] 261 Arousal estimation model memory

[0392] 262 Arousal Estimator

Claims

1. A signal processing device, comprising: a first signal quality estimator that estimates a first signal quality of an input biometric signal; a second signal quality estimator that estimates a second signal quality of an output signal from the noise reduction processor; as well as A heart rate estimator estimates a heart rate based on the output signal from the noise reduction processor and estimates reliability of the estimated heart rate based on a result of the estimated first signal quality and a result of the estimated second signal quality.

2. The signal processing device according to claim 1, The heart rate estimator detects a peak position of the output signal from the noise reduction processor and estimates the reliability of the estimated heart rate at the detected peak position.

3. The signal processing device according to claim 1, Wherein when the result of the estimated first signal quality is high and the result of the estimated second signal quality is high, the heart rate estimator estimates the reliability of the estimated heart rate to be high.

4. The signal processing device according to claim 2, further comprising: a period analyzer that periodically performs analysis on the output signal from the noise reduction processor; The heart rate estimator estimates the reliability of the estimated heart rate based on the result of estimating the first signal quality, the result of estimating the second signal quality, and the result of the periodic analysis of the output signal from the noise reduction processor. 5 . The signal processing device according to claim 4 , wherein the noise reduction processor performs noise reduction processing on the input biometric signal to obtain the output signal.

6. The signal processing device according to claim 5, further comprising: A body movement state analyzer determines body movement of a user based on body movement information about the user acquired by a sensor.

7. The signal processing device according to claim 6, When the body movement of the user is greater than a first threshold, at least one of the noise reduction processor, the second signal quality estimator or the period analyzer stops processing.

8. The signal processing device according to claim 7, Wherein when the result of the estimated first signal quality is high and the body movement of the user is less than a second threshold, at least one of the noise reduction processor, the second signal quality estimator or the period analyzer stops the processing.

9. The signal processing device according to claim 1, further comprising: A correction processor corrects the estimated heart rate based on the reliability of the estimated heart rate.

10. The signal processing device according to claim 1, The input biometric signal includes a pulse wave signal.

11. The signal processing device according to claim 1, The signal processing device is arranged in the wearable shell.

12. A signal processing method comprising the following steps: estimating a first signal quality of an input biometric signal; reducing noise of the input biometric signal; estimating a second signal quality of the input biometric signal after reducing the noise of the input biometric signal; as well as After reducing the noise of the input biometric signal, a heart rate is estimated based on the input biometric signal, and reliability of the estimated heart rate is estimated based on a result of estimating the first signal quality and a result of estimating the second signal quality.

13. A program executed by a processor, the program causing the processor to: estimating a first signal quality of an input biometric signal; estimating a second signal quality of the input biometric signal after noise reduction processing; estimating a heart rate based on the input biometric signal after the noise reduction processing, and Reliability of the estimated heart rate is estimated based on a result of estimating the first signal quality and a result of estimating the second signal quality.

14. A learning device comprising: a first signal quality estimator that estimates a first signal quality of an input biometric signal; a second signal quality estimator that estimates a second signal quality of an output signal from the noise reduction processor; a heart rate estimator that estimates a heart rate based on the output signal from the noise reduction processor and estimates reliability of the estimated heart rate based on a result of the estimation of the first signal quality and a result of the estimation of the second signal quality; a correction processor that corrects the estimated heart rate based on the estimated reliability of the heart rate; as well as A trained estimation model is provided to estimate an emotional state of a person using the estimated heart rate and the reliability of the estimated heart rate.

15. The signal processing method according to claim 12, further comprising: The body movement of the user is determined based on body movement information about the user acquired by the sensor.

16. The signal processing method according to claim 12, Wherein when the result of the estimated first signal quality is higher and the result of the estimated second signal quality is higher, the reliability of the estimated heart rate is higher.

17. The signal processing method according to claim 12, further comprising: After reducing the noise of the input biometric signal, periodically performing analysis on the input biometric signal, The reliability of the estimated heart rate is based on the result of estimating the first signal quality, the result of estimating the second signal quality, and the result of periodic analysis of the input biometric signal after reducing the noise of the input biometric signal.

18. The signal processing method according to claim 12, The step of reducing the noise of the input biometric signal includes a noise reduction processor performing noise reduction processing on the input biometric signal.

19. The signal processing method according to claim 12, further comprising: The estimated heart rate is corrected based on the reliability of the estimated heart rate.

20. The signal processing method according to claim 12, The input biometric signal includes a pulse wave signal.

Images