Pulse wave detection method

Abstract

To provide a pulse wave detection method capable of calculating a blood flow rate corresponding value, which is necessary for detecting vital information based on a pulse wave, by a small calculation amount while maintaining accuracy.SOLUTION: A pulse wave detection method for detecting a pulse wave of a living body in a non-contact manner includes: an acquisition process for acquiring a skin surface image of a living body by an imaging device as a luminance value of RGB pigments; a calculation process for calculating a blood flow rate corresponding value from the luminance value; and an extraction process for extracting vital information from the blood flow rate corresponding value. In the calculation process, an arithmetic value obtained by subtracting a logarithmic value of a luminance value of a G pigment from an average of a logarithmic value of a luminance value of an R pigment and a logarithmic value of a luminance value of a B pigment is used for calculating the blood flow rate corresponding value.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a pulse wave detection method. [Background technology]

[0002] In living organisms, changes in skin color occur due to fluctuations in blood vessel volume and oxygen saturation concentration caused by heartbeat. For this reason, for example, by photographing a person's face over time with a camera, it is possible to obtain blood flow, which is information about the pulse wave, from changes in color (see International Publication No. WO 2020 / 090348).

[0003] To obtain an accurate blood flow rate, signal processing is required to remove the effects of melanin contained in the skin from the captured image and grasp the pulse wave shape. For example, independent component analysis can be used to extract the components related to the blood flow rate, but this requires a large amount of calculation. Therefore, in the invention described in the above international publication, a vector related to the blood flow rate is extracted using a vector perpendicular to both the absorption vector and the shadow vector of melanin, thereby reducing the amount of calculation. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2020 / 090348 Summary of the Invention [Problem to be solved by the invention]

[0005] For example, when considering an embedded system that detects a driver's fatigue (drowsiness) from their pulse wave and controls driving accordingly, it is essential to obtain pulse wave information in real time, and since pulse wave detection itself is not the objective, there are also restrictions on the hardware that can be used for pulse wave detection.

[0006] On the other hand, since the pulse wave is determined by the time change of blood flow, it is necessary to continuously measure the blood flow rate (or the equivalent amount that changes in conjunction with the blood flow rate) from moment to moment, so when considering application to the above-mentioned embedded system, even the above-mentioned method of extracting the vector related to the blood flow rate still requires a large amount of calculations. Therefore, a further reduction in the amount of calculations is required.

[0007] The present disclosure has been made in light of the above circumstances, and has an object to provide a pulse wave detection method that can calculate a blood flow equivalent value required for detecting vital information based on a pulse wave with a small amount of calculation while maintaining accuracy. [Means for solving the problem]

[0008] A pulse wave detection method according to one aspect of the present disclosure is a pulse wave detection method for non-contact detection of a living body's pulse wave, and includes an acquisition step of acquiring an image of the skin surface of the living body as luminance values ​​of RGB pigments by an imaging device, a calculation step of calculating a blood flow equivalent value from the luminance values, and an extraction step of extracting vital information from the blood flow equivalent value, in which the blood flow equivalent value is calculated using a calculated value obtained by subtracting the logarithm of the G pigment luminance value from the average value of the logarithm of the R pigment luminance value and the logarithm of the B pigment luminance value. Effect of the Invention

[0009] The pulse wave detection method of the present disclosure can calculate a blood flow equivalent value required for detecting vital information based on a pulse wave with a small amount of calculation while maintaining accuracy. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a flow diagram illustrating a pulse wave detection method according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a schematic diagram showing the configuration of an imaging device used in the pulse wave detection method of FIG. [Diagram 3] FIG. 3 is a graph showing an example of luminance values ​​acquired by the imaging device of FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] [Description of the embodiments of the present disclosure] First, the embodiments of the present disclosure will be listed and described.

[0012] (1) A pulse wave detection method according to one aspect of the present disclosure is a pulse wave detection method for non-contact detection of a pulse wave of a living body, the method including: an acquisition step of acquiring an image of a skin surface of the living body as luminance values ​​of RGB pigments by an imaging device; a calculation step of calculating a blood flow equivalent value from the luminance values; and an extraction step of extracting vital information from the blood flow equivalent value, wherein the blood flow equivalent value is calculated using a calculated value obtained by subtracting the logarithm of the G pigment luminance value from the average value of the logarithm of the R pigment luminance value and the logarithm of the B pigment luminance value.

[0013] The inventors of the present invention have found that simple calculations of the R, B and G pigment luminance values ​​result in almost no components based on melanin, and the components based on blood flow are predominant. Therefore, in this pulse wave detection method, a calculated value obtained by subtracting the logarithmic value of the G pigment luminance value from the average value of the logarithmic value of the R pigment luminance value and the logarithmic value of the B pigment luminance value is used to calculate the blood flow equivalent value. Therefore, this pulse wave detection method can accurately calculate the blood flow equivalent value by removing the influence of melanin and the like with only a few additions and subtractions.

[0014] Here, "blood flow equivalent value" refers to an amount that changes in conjunction with blood flow, and is, for example, not only the blood flow itself but also an amount that is proportional to the blood flow. In the case of an amount that is proportional to the blood flow, the proportionality coefficient does not necessarily need to be known. If such a blood flow equivalent value is obtained, the pulse wave can be detected. Note that, in the following explanation, the "blood flow rate" may be used as a representative example, but the same effect can be obtained even with a blood flow equivalent value. Furthermore, "logarithmic value" includes not only an accurate logarithmic calculation value, but also a calculation value using a function formula that approximately calculates a logarithmic value.

[0015] (2) In the pulse wave detection method of (1) above, the luminance value of each dye used in the calculation step may be corrected by the spectral sensitivity of the imaging device. The luminance value of each dye obtained from the imaging device may be modulated due to differences in spectral sensitivity. Therefore, by correcting the luminance value of each dye obtained from the imaging device by the spectral sensitivity of the imaging device, the relative relationship between the luminance values ​​of each dye is corrected, thereby improving the accuracy of the calculated blood flow equivalent value.

[0016] (3) In the pulse wave detection method of (2) above, when the spectral sensitivity of R dye is wr, the spectral sensitivity of G dye is wg, and the spectral sensitivity of B dye is wb, it is preferable that the correction in the calculation step is performed according to the following formula 1. By using the following formula 1 in this way, the accuracy of the blood flow equivalent value can be further improved. Brightness value of R dye after correction = Brightness value of R dye before correction × wr Brightness value of B pigment after correction = Brightness value of B pigment before correction × wb 1 Brightness value of G pigment after correction = Brightness value of G pigment before correction × 2 × wg

[0017] (4) In any of the pulse wave detection methods (1) to (3), the calculated value calculated from the luminance values ​​of RGB dyes obtained by photographing white or black with the imaging device may be set as a reference value, and the blood flow equivalent value may be expressed as a relative amount from the reference value. In this pulse wave detection method, the calculated value calculated from the luminance values ​​of RGB dyes obtained by photographing white or black with the imaging device should essentially be 0, but is not necessarily 0 due to various error factors. For this reason, by regarding the calculated value obtained from white or black as the reference value, i.e., 0, and expressing the blood flow equivalent value as a relative amount from the reference value, the influence of the various error factors can be reduced. Therefore, the accuracy of the calculated blood flow equivalent value can be further improved.

[0018] (5) In any of the pulse wave detection methods (1) to (4) above, the living body may be a human body and the skin may be facial skin. The pulse wave detection method has high accuracy in calculating the blood flow equivalent value, particularly when the target is the face of a human body.

[0019] (6) In any of the pulse wave detection methods (1) to (5) above, the vital sign information may be a heart rate, a respiratory rate, or a blood pressure correlation coefficient. This pulse wave detection method can be suitably used to extract a heart rate, a respiratory rate, or a blood pressure correlation coefficient. Here, the "blood pressure correlation coefficient" refers to a parameter value that is correlated with blood pressure.

[0020] [Details of the embodiment of the present invention] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A pulse wave detection method according to an embodiment of the present invention will be described below with reference to the accompanying drawings.

[0021] As shown in FIG. 1, a pulse wave detection method according to an embodiment of the present disclosure includes an acquisition step S1, a calculation step S2, and an extraction step S3.

[0022] <Acquisition process> In the acquisition step S1, an image of the skin surface of a living body X is acquired as luminance values ​​of RGB pigments by an image capturing device 1 as shown in FIG.

[0023] The imaging device 1 has an LED 1a for irradiating the surface of the living body X with irradiation light L1, and a photodetector 1b for detecting reflected light L2 reflected from the living body X. As shown in FIG. 2, the irradiation light L1 penetrates a certain distance from the surface of the living body X to the inside, so that the photodetector 1b also captures reflected light L2 from a blood vessel Y near the surface of the living body X. The blood flow through the blood vessel Y increases and decreases according to the pulsation of the heart, and the blood vessel Y repeatedly contracts as shown in FIG. 2, so that the luminance value of the reflected light L2 increases and decreases corresponding to the contraction of the blood vessel Y, that is, the pulsation of the heart, as shown in FIG. 3. The pulse wave detection method calculates the blood flow rate (a value equivalent to the blood flow rate) from the change in the luminance value of the reflected light L2, and detects the pulse wave of the living body X from the time change in the blood flow rate in a non-contact manner. The living body X is preferably a human body. The pulse wave detection method can be suitably used for detecting the pulse wave of the human body.

[0024] <Calculation process> In the calculation step S2, the blood flow is calculated from the brightness value. Since the blood flow changes from moment to moment, it is calculated as a function of time. Generally, the blood flow is calculated by digital processing, so it may be calculated discretely with respect to time.

[0025] Specifically, since the luminance value of the reflected light L2 contains not only light absorption due to blood vessel Y but also absorption due to melanin contained mainly in the skin of the living body X and absorption due to shadows caused by unevenness of the skin, in order to calculate the blood flow rate, it is necessary to extract from the luminance value the component due to light absorption due to blood vessel Y. In the pulse wave detection method, in the calculation step S2, the blood flow rate is calculated by subtracting the logarithm of the luminance value of the G pigment from the average value of the logarithm of the luminance value of the R pigment and the logarithm of the luminance value of the B pigment.

[0026] The reason why the above calculated value can accurately approximate the blood flow volume will be described below. For simplicity, the luminance value measured by the imaging device 1 will be described as being composed of light absorption (absorbance) due to blood vessel Y and absorbance due to melanin. In reality, these are predominant. Also, "R pigment", "G pigment", and "B pigment" may be abbreviated to simply "R", "G", and "B".

[0027] The absorbance due to blood vessel Y is dominated by absorption by oxyhemoglobin in arteries, and by reduced hemoglobin in veins. It is known that absorption by oxyhemoglobin is large mainly for R and G, while absorption by reduced hemoglobin is large for G and B. The blood flow rate at a location where blood vessel Y is located increases and decreases according to the pulse wave as described above, and this blood flow rate is expressed as a function of time f(t).

[0028] The absorbance of blood vessel Y at R is determined by the ratio of oxygenated hemoglobin in the bloodstream. If the imaging position is fixed, the ratio of oxygenated hemoglobin is considered to be constant, so there is a coefficient Hor that takes into account the absorption sensitivity of oxygenated hemoglobin to R. The absorbance of blood vessel Y at R is expressed as Hor f(t) It is thought that it can be approximated by

[0029] Similarly, the absorbance due to blood vessel Y at B is determined by the proportion of reduced hemoglobin in the bloodstream, and the absorbance due to blood vessel Y at B is expressed as Hrb using the coefficient Hrb. f(t) It is thought that it can be approximated by

[0030] The absorbance due to blood vessel Y in G is determined by the ratio of oxygenated hemoglobin and reduced hemoglobin, and the absorbance due to blood vessel Y in G is calculated by (Hog + Hrg) using the coefficients Hog and Hrg. f(t) It is thought that it can be approximated by

[0031] When the imaging position is fixed, the absorbance due to melanin does not vary with time. Therefore, if the absorption due to melanin in R, G, and B is Mr, Mg, and Mb, the absorption in each pigment Ar, Ag, and Ab is expressed as the product of the absorbance due to blood vessel Y and the absorbance due to melanin, as shown in Equation 2 below. Ar=Hor f(t) ×Mr Ag = (Hog + Hrg) f(t) ×Mg 2 Ab=Hrb f(t) ×Mb

[0032] Taking the logarithm of the above equation 2 gives the following equation 3. log(Ar)=f(t)×log(Hor)+log(Mr) log(Ag)=f(t)×(log(Hog+Hrg))+log(Mg)...3 log(Ab)=f(t)×log(Hrb)+log(Mb)

[0033] Here, the inventors have noted that the absorbance of melanin changes linearly with respect to the wavelength in logarithmic notation. The wavelength of R (red) is 700 nm, the wavelength of G (green) is 546 nm, and the wavelength of B (blue) is 436 nm, with the wavelength of G being approximately in the middle between R and B. In this case, the average value of the absorbance at the wavelength of R and the absorbance at the wavelength of B is approximately equal to the value at the wavelength of G, log(Mr)+log(Mb)=2×log(Mg) ···4 This can be expressed as:

[0034] Since the luminance value is determined according to the absorbance, the calculated value Z obtained by subtracting the logarithm of the G pigment luminance value from the average value of the logarithm of the R pigment luminance value and the logarithm of the B pigment luminance value is given by the following formula 5 from the above formulas 3 and 4. In other words, it is possible to obtain a value that contains almost no components based on melanin, and the components based on blood flow are predominant. Z=1 / 2×(log(Ar)+log(Ab))-log(Ag) = {1 / 2 × log(Hor) + 1 / 2 × log(Hrb) -log(Hog+Hrg)}×f(t) +1 / 2×(log(Mr)+log(Mb)-2×log(Mg)) = {1 / 2 × log(Hor) + 1 / 2 × log(Hrb) -log(Hog+Hrg)}×f(t) 5

[0035] Here, the luminance value measured by the imaging device 1 has been described as being composed of light absorption (absorbance) due to blood vessels Y and absorbance due to melanin, but if the amount changes linearly with respect to wavelength in logarithmic notation, then similarly that component will be hardly included.

[0036] The luminance value of each dye used in the calculation step S2 may be corrected by the spectral sensitivity of the imaging device 1. The luminance value of each dye obtained from the imaging device 1 may be modulated due to differences in spectral sensitivity. For this reason, by correcting the luminance value of each dye obtained from the imaging device 1 by the spectral sensitivity of the imaging device 1, the relative relationship of the luminance values ​​between the dyes is corrected, thereby improving the accuracy of the calculated blood flow rate. Note that the spectral sensitivity of the imaging device 1 is often known depending on the device, but if it is unknown, an image with a known luminance value can be captured by the imaging device 1, and the spectral sensitivity can be determined to match the known luminance value.

[0037] Specifically, when the spectral sensitivity of the R dye is wr, the spectral sensitivity of the G dye is wg, and the spectral sensitivity of the B dye is wb, the correction in the calculation step is preferably performed according to the following formula 1. By using the following formula 1 in this way, the accuracy of the blood flow can be further improved. Brightness value of R dye after correction = Brightness value of R dye before correction × wr Brightness value of B pigment after correction = Brightness value of B pigment before correction × wb 1 Brightness value of G pigment after correction = Brightness value of G pigment before correction × 2 × wg

[0038] The above-mentioned calculated value obtained from the brightness values ​​of the RGB dyes obtained by photographing white or black with the imaging device 1 may be set as a reference value, and the above-mentioned blood flow rate may be expressed as a relative amount from the reference value. The above-mentioned calculated value obtained from the brightness values ​​of the RGB dyes obtained by photographing white or black with the imaging device 1 should be 0, but it is not necessarily 0 due to various error factors such as shading. Therefore, by regarding the calculated value obtained from white or black as the reference value, i.e., 0, and expressing the above-mentioned blood flow rate as a relative amount from the reference value, the influence of the above-mentioned various error factors can be reduced. To give a specific example, for example, when the above-mentioned reference value obtained from the brightness values ​​of the RGB dyes obtained by photographing white is 5, if the calculation amount Z calculated by the above formula 5 is calculated as 15, the blood flow rate will be 10 (=15-5) based on the above-mentioned reference value 5. By expressing the above-mentioned blood flow rate as a relative amount from the above-mentioned reference value in this way, the accuracy of the calculated blood flow rate can be further improved.

[0039] Since black is less dependent on the surrounding environment, when black is used as the reference value, it effectively serves as an absolute adjustment amount. Furthermore, white may change subtly depending on the surrounding environment. This allows adjustment that takes into account the influence of the surrounding environment. Either white or black may be used to calculate the reference value, but both may also be used. By determining the reference value using both white and black, the accuracy of the calculated blood flow rate can be further improved.

[0040] The skin captured by the imaging device 1 is preferably skin on the face, hands, neck, etc., where capillaries are concentrated, and more preferably skin on the face. Of the skin on the face, a part with a high capillary density is even more preferable. In the above formula 5, since Hor>Hog and Hrb>Hrg are generally satisfied, it is more sensitive to detect the pulse wave using both oxyhemoglobin and reduced hemoglobin. Since the capillaries include arteries containing a large amount of oxyhemoglobin and veins containing a large amount of reduced hemoglobin in a moderate balance, it is possible to use information on both oxyhemoglobin and reduced hemoglobin, and the accuracy of the calculated blood flow rate can be improved. In addition, since the face has a high density of capillaries, it is easy to capture the change in brightness value based on the pulse wave.

[0041] <Extraction process> In the extraction step S3, vital information is extracted from the blood flow rate.

[0042] The vital information may include a heart rate, a respiratory rate, and a blood pressure correlation factor. The pulse wave detection method may be suitably used to extract the heart rate, the respiratory rate, or the blood pressure correlation factor.

[0043] The heart rate can be calculated, for example, from the repetition period of the time function of the blood flow calculated in the calculation step S2.

[0044] The respiratory rate can be calculated from the movement of blood vessel Y. The movement of blood vessel Y is superimposed not only on the movement associated with changes in blood flow, but also on the movement of the entire body. Body movement due to breathing is one example of this, and is superimposed even when the body is stationary. For this reason, the movement of blood vessel Y due to breathing can be extracted by applying a bandpass filter to the movement of blood vessel Y, and the respiratory rate can be calculated.

[0045] Blood pressure correlation coefficients can be extracted by constructing a machine learning model based on Lasso regression. For example, the area between the forehead and chin of the face can be divided into a grid, and the pulse wave characteristics of the divided areas can be linked to the measurements of a cuff-type blood pressure monitor to create training data.

[0046] Furthermore, the blood pressure correlation coefficient varies from person to person and also depends on gender, age, body temperature, the hardness of blood vessels Y, the pulsation strength of the heart itself, etc. If the skin captured by the imaging device 1 is facial skin, facial image information is used, so that gender, age, and body temperature can be estimated by additional learning. Furthermore, the hardness of blood vessels Y and the pulsation strength of the heart itself can be calibrated if the measured values ​​of the individual's cuff-type blood pressure monitor are acquired.

[0047] <Advantages> In this pulse wave detection method, the blood flow rate is calculated by subtracting the logarithm of the G pigment luminance value from the average value of the logarithm of the R pigment luminance value and the logarithm of the B pigment luminance value. Therefore, this pulse wave detection method can accurately calculate the blood flow rate by removing the influence of melanin, etc., with only a few additions and subtractions.

[0048] [Other embodiments] The above-mentioned embodiment does not limit the configuration of the present disclosure. Therefore, the above-mentioned embodiment may omit, replace, or add components of each part of the above-mentioned embodiment based on the description in this specification and common technical knowledge, and it should be construed that all of them belong to the scope of the present disclosure.

[0049] In the above embodiment, the calculation step uses a calculated value obtained by subtracting the logarithm of the G dye luminance value from the average value of the logarithm of the R dye luminance value and the logarithm of the B dye luminance value to calculate the blood flow rate, but the calculation result is substantially the same even if a value obtained by multiplying this calculated value by a constant is used. For example, the value obtained by subtracting twice the logarithm of the G dye luminance value from the sum of the logarithm of the R dye luminance value and the logarithm of the B dye luminance value is twice the above calculated value, but the same effect is obtained as when the above calculated value is used, and it can be said that the above calculated value is used.

[0050] In the above embodiment, the imaging device has an LED that irradiates light onto the surface of the living body and a photodetector that detects light reflected from the living body, but the imaging device is not limited to this configuration and may, for example, be a camera. [Industrial Applicability]

[0051] As described above, the pulse wave detection method of the present disclosure can calculate a blood flow equivalent value required for detecting vital information based on a pulse wave with a small amount of calculation while maintaining accuracy. Therefore, the pulse wave detection method can be easily applied to embedded systems. [Explanation of symbols]

[0052] 1. Imaging device 1a LED 1b Photodetector L1 irradiation light L2 reflected light X Living organism Y blood vessel

Claims

1. A pulse wave detection method for non-contact detection of a pulse wave of a living body, comprising: an acquisition step of acquiring an image of a skin surface of a living body as luminance values ​​of RGB dyes by an imaging device; A calculation step of calculating a blood flow equivalent value from the luminance value; an extraction step of extracting vital information from the blood flow equivalent value; Equipped with In the calculation step, the blood flow equivalent value is calculated by subtracting the logarithm of the G dye luminance value from the average value of the logarithm of the R dye luminance value and the logarithm of the B dye luminance value.

2. 2. The method according to claim 1, wherein the luminance value of each dye used in said calculation step is corrected based on the spectral sensitivity of said imaging device.

3. 3. The method for detecting a pulse wave according to claim 2, wherein the correction in the calculation step is performed in accordance with the following formula 1, where wr is the spectral sensitivity of R dye, wg is the spectral sensitivity of G dye, and wb is the spectral sensitivity of B dye. Brightness value of R pigment after correction = Brightness value of R pigment before correction × wr Brightness value of B pigment after correction = Brightness value of B pigment before correction × wb ... 1 Luminance value of G pigment after correction = Luminance value of G pigment before correction × 2 × wg

4. 4. The pulse wave detection method according to claim 1, wherein the calculated value obtained from luminance values ​​of RGB dyes obtained by photographing white or black with the imaging device is set as a reference value, and the value equivalent to blood flow is expressed as a relative amount from the reference value.

5. 4. The pulse wave detecting method according to claim 1, wherein the living body is a human body, and the skin is facial skin.

6. 4. The pulse wave detection method according to claim 1, wherein the vital information is a heart rate, a respiratory rate, or a blood pressure correlation function.

Images