Pulse wave detection method

Abstract

A pulse wave detection method according to the present disclosure is for detecting a pulse wave of a living body in a non-contact manner. The pulse wave detection method includes: an acquisition step for acquiring a skin surface image of the living body as luminance values of RGB pigments by using an imaging device; a calculation step for calculating a blood flow rate equivalent value from the luminance values; and an extraction step for extracting vital information from the blood flow rate equivalent value. In the calculation step, a calculation value obtained by subtracting a logarithmic value of a luminance value of a G pigment from an average value of a logarithmic value of a luminance value of an R pigment and a logarithmic value of a luminance value of a B pigment is used for the calculation of the blood flow rate equivalent value.

Description

Pulse wave detection method

[0001] The present disclosure relates to a pulse wave detection method.

[0002] In living organisms, changes in skin color occur due to fluctuations in blood vessel volume and oxygen saturation concentration caused by heartbeat. For this reason, for example, by photographing a person's face over time with a camera, it is possible to obtain blood flow, which is information about the pulse wave, from changes in color (see International Publication No. 2020 / 090348).

[0003] To obtain accurate blood flow, signal processing is required to remove the influence of melanin contained in the skin from the captured image and grasp the pulse waveform. For example, independent component analysis can be used to extract components related to blood flow, but this requires a large amount of calculation. Therefore, the invention described in the above international publication extracts a vector related to blood flow using a vector orthogonal to both the melanin absorption vector and shading vector, thereby reducing the amount of calculation.

[0004] International Publication No. 2020 / 090348

[0005] For example, when considering an embedded system that detects driver fatigue (drowsiness) from the pulse wave of a car driver and controls driving accordingly, it is essential to obtain pulse wave information in real time, and since pulse wave detection itself is not the goal, there are also restrictions on the hardware that can be used for pulse wave detection.

[0006] On the other hand, since pulse waves are determined by changes in blood flow over time, it is necessary to continuously measure the blood flow rate (or an equivalent amount that changes in conjunction with the blood flow rate) from moment to moment, so when considering application to the embedded systems mentioned above, even the method of extracting vectors related to blood flow rate still requires a large amount of calculations, and therefore further reduction in the amount of calculations is required.

[0007] The present disclosure has been made in light of the above circumstances, and aims to provide a pulse wave detection method that can calculate a blood flow equivalent value required for detecting vital information based on a pulse wave with a small amount of calculation while maintaining accuracy.

[0008] A pulse wave detection method according to one aspect of the present disclosure is a pulse wave detection method for non-contact detection of the pulse wave of a living body, and includes an acquisition step of acquiring an image of the skin surface of the living body as luminance values ​​of RGB pigments using an imaging device, a calculation step of calculating a blood flow equivalent value from the luminance values, and an extraction step of extracting vital information from the blood flow equivalent value, wherein the calculation step calculates the blood flow equivalent value using a calculated value obtained by subtracting the logarithmic value of the G pigment luminance value from the average value of the logarithmic value of the R pigment luminance value and the logarithmic value of the B pigment luminance value.

[0009] The pulse wave detection method of the present disclosure can calculate a blood flow equivalent value required for detecting vital information based on a pulse wave with a small amount of calculation while maintaining accuracy.

[0010] Fig. 1 is a flow diagram showing a pulse wave detection method according to an embodiment of the present disclosure. Fig. 2 is a schematic diagram showing the configuration of an image capture device used in the pulse wave detection method of Fig. 1. Fig. 3 is a graph showing an example of luminance values ​​acquired by the image capture device of Fig. 2.

[0011] [Description of Embodiments of the Present Disclosure] First, embodiments of the present disclosure will be listed and described.

[0012] (1) A pulse wave detection method according to one aspect of the present disclosure is a pulse wave detection method for contactlessly detecting the pulse wave of a living body, and includes an acquisition step of acquiring an image of the skin surface of the living body as luminance values ​​of RGB pigments using an imaging device, a calculation step of calculating a blood flow equivalent value from the luminance values, and an extraction step of extracting vital information from the blood flow equivalent value, wherein the calculation step calculates the blood flow equivalent value by subtracting the logarithmic value of the G pigment luminance value from the average value of the logarithmic value of the R pigment luminance value and the logarithmic value of the B pigment luminance value.

[0013] The inventors of the present invention discovered that simple calculations on the luminance values ​​of the R pigment, the B pigment, and the G pigment can eliminate most of the components based on melanin and allow the components based on blood flow to become dominant. Therefore, in this pulse wave detection method, a calculated value is used to calculate the blood flow equivalent value by subtracting the logarithmic value of the G pigment luminance value from the average value of the logarithmic value of the R pigment luminance value and the logarithmic value of the B pigment luminance value. This pulse wave detection method can accurately calculate the blood flow equivalent value by eliminating the influence of melanin and the like with only a few additions and subtractions.

[0014] Here, "blood flow equivalent value" refers to a quantity that changes in conjunction with blood flow, such as not only the blood flow itself but also a quantity that is proportional to the blood flow. In the case of a quantity that is proportional to the blood flow, the proportionality coefficient does not necessarily need to be known. If such a blood flow equivalent value is obtained, the pulse wave can be detected. Note that, although the following explanation may be given simply using "blood flow rate," the same effect can be achieved even with a blood flow equivalent value. Furthermore, "logarithmic value" includes not only an accurate logarithmic calculation value, but also a calculation value using a function that calculates a logarithmic value approximately.

[0015] (2) In the pulse wave detection method of (1) above, the luminance value of each dye used in the calculation step may be corrected based on the spectral sensitivity of the imaging device. The luminance values ​​of each dye obtained from the imaging device may be modulated due to differences in spectral sensitivity. Therefore, by correcting the luminance values ​​of each dye obtained from the imaging device based on the spectral sensitivity of the imaging device, the relative relationship between the luminance values ​​of each dye is corrected, thereby improving the accuracy of the calculated blood flow equivalent value.

[0016] (3) In the pulse wave detection method of (2) above, when the spectral sensitivity of R pigment is wr, the spectral sensitivity of G pigment is wg, and the spectral sensitivity of B pigment is wb, the correction in the calculation step may be performed according to the following formula 1. By using the following formula 1 in this way, the accuracy of the blood flow equivalent value can be further improved. Corrected R pigment luminance value = R pigment luminance value before correction × wr Corrected B pigment luminance value = B pigment luminance value before correction × wb ... 1 Corrected G pigment luminance value = G pigment luminance value before correction × 2 × wg

[0017] (4) In any of the pulse wave detection methods (1) to (3), the calculated value calculated from the luminance values ​​of the RGB dyes obtained by capturing a white or black image with the imaging device may be used as a reference value, and the blood flow equivalent value may be expressed as a relative amount relative to the reference value. In this pulse wave detection method, the calculated value calculated from the luminance values ​​of the RGB dyes obtained by capturing a white or black image with the imaging device should essentially be zero, but is not necessarily zero due to various error factors. Therefore, by regarding the calculated value obtained from a white or black image as the reference value, i.e., zero, and expressing the blood flow equivalent value as a relative amount relative to the reference value, the influence of the various error factors can be reduced. This further improves the accuracy of the calculated blood flow equivalent value.

[0018] (5) In any one of the pulse wave detection methods (1) to (4), the living body may be a human body, and the skin may be facial skin. The pulse wave detection method has high accuracy in calculating the blood flow equivalent value, particularly when targeting the face of a human body.

[0019] (6) In any of the pulse wave detection methods (1) to (5), the vital sign information may be a heart rate, a respiratory rate, or a blood pressure correlation coefficient. This pulse wave detection method can be suitably used to extract a heart rate, a respiratory rate, or a blood pressure correlation coefficient. Here, the "blood pressure correlation coefficient" refers to a parameter value correlated with blood pressure.

[0020] [Details of the embodiment of the present invention] A pulse wave detection method according to one embodiment of the present invention will be described below with reference to the accompanying drawings as appropriate.

[0021] As shown in FIG. 1, a pulse wave detection method according to an embodiment of the present disclosure includes an acquisition step S1, a calculation step S2, and an extraction step S3.

[0022] <Acquisition Step> In the acquisition step S1, an image of the skin surface of the living body X is acquired as luminance values ​​of RGB pigments by the imaging device 1 as shown in FIG.

[0023] The imaging device 1 includes an LED 1a that irradiates the surface of the living body X with irradiated light L1 and a photodetector 1b that detects reflected light L2 from the living body X. As shown in FIG. 2 , the irradiated light L1 penetrates a certain distance from the surface of the living body X into the interior, so the photodetector 1b also captures reflected light L2 from blood vessels Y near the surface of the living body X. Because blood flow through blood vessels Y increases and decreases with the heartbeat, and blood vessels Y repeatedly contract as shown in FIG. 2 , the luminance value of reflected light L2 increases and decreases corresponding to the contraction of blood vessels Y, i.e., the heartbeat, as shown in FIG. 3 . This pulse wave detection method calculates the blood flow rate (a value equivalent to blood flow rate) from changes in the luminance value of the reflected light L2 and detects the pulse wave of the living body X from the temporal change in blood flow rate in a non-contact manner. A human body is preferred as the living body X. This pulse wave detection method is suitable for detecting the pulse wave of a human body.

[0024] <Calculation Step> In the calculation step S2, the blood flow rate is calculated from the brightness value. Since the blood flow rate changes from moment to moment, it is calculated as a function of time. Since the blood flow rate is generally calculated by digital processing, it may be calculated discretely with respect to time.

[0025] Specifically, the luminance value of the reflected light L2 contains not only light absorption due to the blood vessels Y but also absorption due to melanin contained mainly in the skin of the living body X and absorption due to shadows caused by unevenness of the skin, and therefore, in order to calculate the blood flow rate, it is necessary to extract from the luminance value the component due to light absorption due to the blood vessels Y. In the pulse wave detection method, in the calculation step S2, the blood flow rate is calculated by subtracting the logarithmic value of the G pigment luminance value from the average value of the logarithmic value of the R pigment luminance value and the logarithmic value of the B pigment luminance value.

[0026] The reason why the above calculated value can accurately approximate the blood flow rate will be explained below. For simplicity, the luminance value measured by the imaging device 1 will be explained as consisting of light absorption (absorbance) due to blood vessel Y and absorbance due to melanin. In reality, these are dominant. Also, "R pigment," "G pigment," and "B pigment" may be abbreviated as simply "R," "G," and "B."

[0027] The absorbance due to blood vessel Y is dominated by absorption by oxyhemoglobin in arteries and by deoxyhemoglobin in veins. It is known that absorption by oxyhemoglobin is large mainly for R and G, while absorption by deoxyhemoglobin is large for G and B. The blood flow rate at a location in blood vessel Y increases and decreases in accordance with the pulse wave as described above, and this blood flow rate is represented as a time function f(t).

[0028] The absorbance of blood vessel Y in R is determined by the ratio of oxyhemoglobin contained in the blood flow. If the imaging position is fixed, the ratio of oxyhemoglobin is considered to be constant. Therefore, there is a coefficient Hor that takes into account the absorption sensitivity of oxyhemoglobin to R. The absorbance of blood vessel Y in R is expressed as Hor ｆ（ｔ） It is thought that it can be approximated by

[0029] Similarly, the absorbance due to blood vessel Y at B is determined by the proportion of reduced hemoglobin contained in the blood flow, and the absorbance due to blood vessel Y at B is expressed as Hrb using the coefficient Hrb. ｆ（ｔ） It is thought that it can be approximated by

[0030] The absorbance due to blood vessel Y in G is determined by the ratio of oxygenated hemoglobin and reduced hemoglobin, and the absorbance due to blood vessel Y in G is calculated as (Hog + Hrg) using the coefficients Hog and Hrg. ｆ（ｔ） It is thought that it can be approximated by

[0031] When the imaging position is fixed, the absorbance due to melanin does not change over time. Therefore, if the absorption due to melanin in R, G, and B is Mr, Mg, and Mb, the absorptions Ar, Ag, and Ab in each pigment are expressed as the product of the absorbance due to blood vessel Y and the absorbance due to melanin, as shown in the following formula 2: Ar=Hor ｆ（ｔ） ×Mr Ag=(Hog+Hrg) ｆ（ｔ） ×Mg...2 Ab=Hrb ｆ（ｔ） ×MB

[0032] Taking the logarithm of the above equation 2, the following equation 3 is obtained: log(Ar) = f(t) × log(Hor) + log(Mr) log(Ag) = f(t) × (log(Hog + Hrg)) + log(Mg) ... 3 log(Ab) = f(t) × log(Hrb) + log(Mb)

[0033] Here, the inventors have noted that the absorbance of melanin changes linearly with wavelength in logarithmic notation. The wavelength of R (red) is 700 nm, the wavelength of G (green) is 546 nm, and the wavelength of B (blue) is 436 nm, with the G wavelength being approximately midway between R and B. In this case, the absorbance based on melanin is calculated by averaging the values ​​at the R wavelength and the B wavelength, and is approximately equal to the value at the G wavelength, and can be expressed as log(Mr) + log(Mb) = 2 × log(Mg) ... 4.

[0034] Since the luminance value is determined in accordance with the absorbance, the calculated value Z obtained by subtracting the logarithm of the luminance value of the G pigment from the average value of the logarithm of the luminance value of the R pigment and the logarithm of the luminance value of the B pigment is expressed by the following equation 5 from the above equations 3 and 4. That is, it is possible to obtain a result that contains almost no components based on melanin, and is dominated by components based on blood flow. Z = 1 / 2 × (log(Ar) + log(Ab)) - log(Ag) = {1 / 2 × log(Hor) + 1 / 2 × log(Hrb) - log(Hog + Hrg)} × f(t) + 1 / 2 × (log(Mr) + log(Mb) - 2 × log(Mg)) = {1 / 2 × log(Hor) + 1 / 2 × log(Hrb) - log(Hog + Hrg)} × f(t) ...5

[0035] Here, we have explained that the brightness value measured by the imaging device 1 is composed of light absorption (absorbance) due to blood vessels Y and absorbance due to melanin, but if the amount changes linearly with wavelength in logarithmic notation, then that component will similarly be hardly included.

[0036] The luminance value of each dye used in the calculation step S2 may be corrected by the spectral sensitivity of the imaging device 1. The luminance value of each dye obtained from the imaging device 1 may be modulated due to differences in spectral sensitivity. Therefore, by correcting the luminance value of each dye obtained from the imaging device 1 by the spectral sensitivity of the imaging device 1, the relative relationship of the luminance values ​​between the dyes is corrected, thereby improving the accuracy of the calculated blood flow rate. Note that the spectral sensitivity of the imaging device 1 is often known depending on the device, but if it is unknown, an image with a known luminance value can be captured with the imaging device 1, and the spectral sensitivity can be determined to match the known luminance value.

[0037] Specifically, when the spectral sensitivity of the R dye is wr, the spectral sensitivity of the G dye is wg, and the spectral sensitivity of the B dye is wb, the correction in the calculation step is preferably performed according to the following formula 1. By using the following formula 1 in this way, the accuracy of the blood flow rate can be further improved. Corrected R dye luminance value = R dye luminance value before correction × wr Corrected B dye luminance value = B dye luminance value before correction × wb ...1 Corrected G dye luminance value = G dye luminance value before correction × 2 × wg

[0038] The calculated value obtained from the luminance values ​​of the RGB dyes obtained by photographing a white or black object with the imaging device 1 may be used as a reference value, and the blood flow rate may be expressed as a relative amount from the reference value. The calculated value Z obtained from the luminance values ​​of the RGB dyes obtained by photographing a white or black object with the imaging device 1 should essentially be 0, but this is not necessarily 0 due to various error factors such as shading. Therefore, by regarding the calculated value obtained from a white or black object as the reference value, i.e., 0, and expressing the blood flow rate as a relative amount from the reference value, the influence of the various error factors can be reduced. Specifically, for example, if the reference value obtained from the luminance values ​​of the RGB dyes obtained by photographing a white object is 5, and the calculated amount Z calculated using Equation 5 is 15, the blood flow rate will be 10 (=15-5) based on the reference value 5. Expressing the blood flow rate as a relative amount from the reference value in this manner further improves the accuracy of the calculated blood flow rate.

[0039] Since black is less dependent on the surrounding environment, using black as the reference value effectively serves as an absolute adjustment amount. Furthermore, white can change subtly depending on the surrounding environment. This allows for adjustment that takes into account the influence of the surrounding environment. Either white or black alone may be used to calculate the reference value, but both can also be used. By determining the reference value using both white and black, the accuracy of the calculated blood flow rate can be further improved.

[0040] The skin captured by the imaging device 1 is preferably skin with a high concentration of capillaries, such as the face, hands, and neck, and more preferably facial skin. Among facial skin, areas with a high capillary density are even more preferable. In the above formula 5, since Hor > Hog and Hrb > Hrg, detecting a pulse wave using both oxyhemoglobin and deoxyhemoglobin provides higher sensitivity. Since capillaries are present in a moderate balance between arteries containing a large amount of oxyhemoglobin and veins containing a large amount of deoxyhemoglobin, information on both oxyhemoglobin and deoxyhemoglobin can be used, improving the accuracy of the calculated blood flow rate. Furthermore, because the face has a high capillary density, it is easy to capture changes in brightness based on the pulse wave.

[0041] <Extraction Step> In the extraction step S3, vital information is extracted from the blood flow rate.

[0042] The vital information may include a heart rate, a respiratory rate, and a blood pressure correlation coefficient. The pulse wave detection method can be suitably used to extract the heart rate, the respiratory rate, or the blood pressure correlation coefficient.

[0043] The heart rate can be calculated from, for example, the repetition period of the time function of the blood flow calculated in the calculation step S2.

[0044] The respiratory rate can be calculated from the movement of the blood vessel Y. The movement of the blood vessel Y is superimposed not only on the movement associated with changes in blood flow but also on the movement of the entire body. Body movement due to breathing is one example of this, and is superimposed even when the body is stationary. For this reason, by applying a band-pass filter to the movement of the blood vessel Y, the movement of the blood vessel Y due to breathing can be extracted and the respiratory rate can be calculated.

[0045] Blood pressure correlation coefficients can be extracted by constructing a machine learning model based on Lasso regression. For example, the area between the forehead and chin can be divided into a grid, and the pulse wave characteristics of each divided area can be linked to the measurements of a cuff-type blood pressure monitor to create training data.

[0046] Furthermore, the blood pressure correlation coefficient varies from person to person and is dependent on factors such as gender, age, body temperature, stiffness of blood vessels Y, and the pulsation strength of the heart itself. If the skin captured by the imaging device 1 is facial skin, facial image information is used, making it possible to estimate gender, age, and body temperature through additional learning. Furthermore, the stiffness of blood vessels Y and the pulsation strength of the heart itself can be calibrated if measurements from an individual's cuff-type blood pressure monitor have been acquired.

[0047] <Advantages> In this pulse wave detection method, the blood flow rate is calculated by subtracting the logarithm of the luminance value of the G pigment from the average value of the logarithm of the luminance value of the R pigment and the logarithm of the luminance value of the B pigment. Therefore, this pulse wave detection method can accurately calculate the blood flow rate by removing the influence of melanin and the like with only a few additions and subtractions.

[0048] [Other Embodiments] The above-described embodiments do not limit the configuration of the present disclosure. Therefore, the above-described embodiments may include omissions, substitutions, or additions of components based on the description in this specification and common general technical knowledge, and all of these should be construed as belonging to the scope of the present disclosure.

[0049] In the above embodiment, the calculation step uses a calculated value obtained by subtracting the logarithm of the G dye luminance value from the average value of the logarithm of the R dye luminance value and the logarithm of the B dye luminance value to calculate the blood flow rate, but the result is essentially the same even if a value obtained by multiplying this calculated value by a constant is used. For example, the value obtained by subtracting twice the logarithm of the G dye luminance value from the sum of the logarithm of the R dye luminance value and the logarithm of the B dye luminance value is twice the calculated value, but the same effect is achieved as when the calculated value is used, and it can be said that the calculated value is used.

[0050] In the above embodiment, the imaging device has been described as having an LED that irradiates light onto the surface of the living body and a photodetector that detects reflected light from the living body, but the imaging device is not limited to this configuration and may also use, for example, a camera.

[0051] As described above, the pulse wave detection method of the present disclosure can calculate the blood flow equivalent value required for detecting vital information based on pulse waves with a small amount of calculation while maintaining accuracy. Therefore, the pulse wave detection method can be easily applied to embedded systems.

[0052] 1 Imaging device 1a LED 1b Photodetector L1 Irradiated light L2 Reflected light X Living body Y Blood vessel

Claims

1. A pulse wave detection method for non-contact detection of a living body's pulse wave, comprising: an acquisition step of acquiring an image of the living body's skin surface as RGB pigment luminance values ​​using an imaging device; a calculation step of calculating a blood flow equivalent value from the luminance values; and an extraction step of extracting vital information from the blood flow equivalent value, wherein in the calculation step, the blood flow equivalent value is calculated by subtracting the logarithm of the G pigment luminance value from the average value of the R pigment luminance value and the B pigment luminance value.

2. A method for detecting a pulse wave according to claim 1, wherein the luminance value of each dye used in said calculation step is corrected based on the spectral sensitivity of said imaging device.

3. The pulse wave detection method according to claim 2, wherein the correction in the calculation step is performed according to the following formula 1, where wr is the spectral sensitivity of R pigment, wg is the spectral sensitivity of G pigment, and wb is the spectral sensitivity of B pigment. Corrected R pigment luminance value = R pigment luminance value before correction x wr Corrected B pigment luminance value = B pigment luminance value before correction x wb ... 1 Corrected G pigment luminance value = G pigment luminance value before correction x 2 x wg 4. A pulse wave detection method according to any one of claims 1 to 3, wherein the calculated value found from the luminance values ​​of RGB pigments obtained by photographing white or black with the imaging device is set as a reference value, and the value equivalent to blood flow is expressed as a relative amount from the reference value.

5. A pulse wave detection method according to any one of claims 1 to 4, wherein the living body is a human body and the skin is facial skin.

6. A pulse wave detection method according to any one of claims 1 to 5, wherein the vital information is a heart rate, a respiratory rate or a blood pressure correlation function.

Images