Pulse wave estimation device, pulse wave estimation method, condition estimation system, and condition estimation method

Abstract

The present invention provides a pulse wave estimation device (MSD) including: an image acquisition unit (GSY) that acquires an image (Im) of a person (HK) captured under near-infrared light; a skin area detection unit (HRK) that detects from the image (Im) a skin area (S), which is the location of the skin of the person (HK); a measurement area setting unit (KRS) that sets, in the skin area (S), a plurality of measurement areas (R), which are areas for measuring the luminance of the skin of the person (HK); a point identification unit (TTO) that identifies a first point (P1) and a second point (P2) which are in one of the plurality of measurement areas (R) and along a direction (HO) in which a pulse wave (M) of the person (HK) propagates in the image (Im); a luminance signal generation unit (KSS) that, out of a first distribution (A) and a second distribution (B) with mutually different correspondence relationships between a coordinate position and the size of a weighting specified by the coordinate position, superimposes the luminance that the first point (P1) has and the luminance that the second point (P2) has with the first distribution (A) to thereby generate a first luminance signal (KI1) indicating the luminance at the first point (P1) after being weighted by the first distribution (A) and a second luminance signal (KI2) indicating the luminance at the second point (P2) after being weighted by the first distribution (A), and superimposes the luminance that the first point (P1) has and the luminance that the second point (P2) has with the second distribution (B) to thereby generate a third luminance signal (KI3) indicating the luminance at the first point (P1) after being weighted by the second distribution (B) and a fourth luminance signal (KI4) indicating the luminance at the second point (P2) after being weighted by the second distribution (B); a pulse wave source signal generation unit (MMS) that generates a first pulse wave source signal (MM1) by combining the first luminance signal (KI1) with the second luminance signal (KI2) and generates a second pulse wave source signal (MM2) by combining the third luminance signal (KI3) with the fourth luminance signal (KI4); a pulse wave signal generation unit (MSS) that generates a pulse wave signal (MY) indicating the pulse wave (M) of the person (HK) by subtracting one of the first pulse wave source signal (MM1) or the second pulse wave source signal (MM2) from the other; and a pulse wave estimation unit (MSU) that estimates the pulse wave (M) of the person (HK) on the basis of the pulse wave signal (MY).

Description

[Technical Field]

[0001] The present disclosure relates to a pulse wave estimation device, a pulse wave estimation method, a condition estimation system, and a condition estimation method. [Background technology]

[0002] The pulse wave detection device described in Patent Document 1 subtracts, for example, the luminance of the red component, which has a small pulse wave component and fluctuates with the movement of the user's head, from the luminance of the green component, which has a large pulse wave component and fluctuates with the movement of the user's head, in an image of the user captured in the visible light band by an RGB camera. By this subtraction, the pulse wave detection device described above removes errors in estimating the user's pulse wave caused by the movement of the user's head, i.e., noise, thereby improving the accuracy of estimating the pulse wave. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2016-006027 Summary of the Invention [Problem to be solved by the invention]

[0004] However, when using the pulse wave detection device described above to monitor the pulse rate of a user sleeping at night, for example, it is necessary to use a near-infrared camera capable of capturing images in the near-infrared wavelength band instead of the RGB camera capable of capturing images in the visible light band. The pulse wave detection device has a problem in that, when the near-infrared camera is used, it is not possible to estimate the pulse wave by subtracting the luminance of the green component from the luminance of the red component.

[0005] An object of the present disclosure is to provide a pulse wave estimation device, a pulse wave estimation method, a condition estimation system, and a condition estimation method that are capable of estimating a subject's pulse wave even using an image captured under near-infrared light. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, a pulse wave estimation device according to the present disclosure includes an image acquisition unit that acquires an image of a person taken under near-infrared light, a skin area detection unit that detects a skin area, which is the location of the person's skin, from the image, a measurement area setting unit that sets a plurality of measurement areas in the skin area, which are areas for measuring the luminance of the person's skin, a point identification unit that identifies a first point and a second point within one of the plurality of measurement areas and along a direction in which the person's pulse wave propagates in the image, and a first distribution and a second distribution that have mutually different correspondence relationships between positions on coordinates and magnitudes of weighting identified by the positions on coordinates, whereby a first luminance signal indicating the luminance at the first point after being weighted by the first distribution and a second luminance signal indicating the luminance at the second point after being weighted by the first distribution are superimposed on the first distribution. a luminance signal generating unit that generates a third luminance signal indicating the luminance at the first point after being weighted by the second distribution and a fourth luminance signal indicating the luminance at the second point after being weighted by the second distribution by superimposing the luminance at the first point and the luminance at the second point on the second distribution; a pulse wave source signal generating unit that generates a first pulse wave source signal by combining the first luminance signal and the second luminance signal and generates the second pulse wave source signal by combining the third luminance signal and the fourth luminance signal; a pulse wave signal generating unit that generates a pulse wave signal indicating the person's pulse wave by subtracting one of the first pulse wave source signal and the second pulse wave source signal from the other; and a pulse wave estimating unit that estimates the person's pulse wave based on the pulse wave signal. [Effects of the Invention]

[0007] According to the pulse wave estimation device according to the present disclosure, it is possible to improve the accuracy of estimating the pulse wave of a subject even when using an image captured under near-infrared light. [Brief explanation of the drawings]

[0008] [Figure 1]2 is a functional block diagram of the pulse wave estimation device MSD of the first embodiment. FIG. [Figure 2] Fig. 2A shows an image Im of embodiment 1. Fig. 2B shows a measurement region R of embodiment 1. Fig. 2C shows a skin region S of embodiment 1. [Figure 3] 1 shows the basic operation up to generation of the first luminance signal KI1 to the fourth luminance signal KI4 in the first embodiment. [Figure 4] 1 shows the basic operation up to the pulse wave signal MY in the first embodiment. [Figure 5] 1 shows detailed operations for generating the first pulse wave original signal MM1 to the second pulse wave original signal MM2 in the first embodiment. [Figure 6] 10 shows detailed operations up to generation of pulse wave signal MY in the first embodiment. [Figure 7] 3 shows the configurations of weighting functions A and B according to the first embodiment. [Figure 8] 10 shows a luminance signal of a comparative example. [Figure 9] 1 shows the hardware configuration of a pulse wave estimation device MSD according to a first embodiment. [Figure 10] 1 shows a hardware configuration based on software realization of a pulse wave estimation device MSD according to a first embodiment. [Figure 11] 4 is a flowchart showing the operation of the pulse wave estimation device MSD of the first embodiment. [Figure 12] 10 shows the configuration of a pulse wave estimation device MSD according to a modification of the first embodiment. [Figure 13] FIG. 10 is a functional block diagram of a pulse wave estimation device MSD according to a second embodiment. [Figure 14] 10 is a flowchart showing the operation of the pulse wave estimation device MSD of the second embodiment. [Figure 15] 10 shows the operation of pulse wave estimation device MSD of embodiment 2 up to generation of first luminance signal KI1(R1) to fourth luminance signal KI4(R1) and first luminance signal KI1(R1) to fourth luminance signal KI4(R2). [Figure 16] 10 shows the operation up to generation of an average pulse wave signal HMY of a pulse wave estimation device MSD according to the second embodiment. [Figure 17]FIG. 11 is a functional block diagram of a pulse wave source signal generating unit MMS according to a third embodiment. [Figure 18] 10 is a flowchart showing the operation of the pulse wave estimation device MSD of the third embodiment. [Figure 19] FIG. 10 is a functional block diagram of a measurement region setting unit KRS, a point identification unit TTO, and a luminance signal generation unit KSS according to a fourth embodiment. [Figure 20] 10 shows a measurement region R, a first point P1 to a fourth point P4, and a first luminance signal KI1 to a fourth luminance signal KI4 of the fourth embodiment. [Figure 21] 10 is a flowchart showing the operation of the pulse wave estimation device MSD of the fourth embodiment. [Figure 22] FIG. 10 is a functional block diagram of a state estimation system JSS according to a fifth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] An embodiment of a pulse wave estimation device according to the present disclosure will be described.

[0010] Embodiment 1. First Embodiment The pulse wave estimation device MSD of the first embodiment will be described.

[0011] In the following, for ease of explanation and understanding, multiple names may be collectively referred to by one reference symbol. For example, multiple names "measurement region R1," "measurement region R2," "measurement region R3," "measurement region R4," etc. may be collectively referred to by one reference symbol "R."

[0012] <Functions of the First Embodiment> FIG. 1 is a functional block diagram of the pulse wave estimation device MSD of the first embodiment.

[0013] FIG. 2 shows an image Im, a measurement region R, and a skin region S according to the first embodiment.

[0014] FIG. 3 shows the basic operation up to generation of the first luminance signal KI1 to the fourth luminance signal KI4 in the first embodiment.

[0015] FIG. 4 shows the basic operation up to the pulse wave signal MY in the first embodiment.

[0016] FIG. 5 shows detailed operations for generating the first pulse wave original signal MM1 to the second pulse wave original signal MM2 in the first embodiment.

[0017] FIG. 6 shows detailed operations up to generation of pulse wave signal MY in the first embodiment.

[0018] The functions of the pulse wave estimation device MSD of the first embodiment will be described below with reference to FIGS.

[0019] As shown in FIG. 1, the pulse wave estimation device MSD of embodiment 1 includes an image acquisition unit GSY, a skin area detection unit HRK, a measurement area setting unit KRS, a point identification unit TTO, a luminance signal generation unit KSS, a pulse wave original signal generation unit MMS, a pulse wave signal generation unit MSS, and a pulse wave estimation unit MSU.

[0020] The image acquisition unit GSY corresponds to the "image acquisition unit", the skin area detection unit HRK corresponds to the "skin area detection unit", the measurement area setting unit KRS corresponds to the "measurement area setting unit", the point identification unit TTO corresponds to the "point identification unit TTO", the luminance signal generation unit KSS corresponds to the "luminance signal generation unit", the pulse wave source signal generation unit MMS corresponds to the "pulse wave source signal generation unit", the pulse wave signal generation unit MSS corresponds to the "pulse wave signal generation unit", and the pulse wave estimation unit MSU corresponds to the "pulse wave estimation unit".

[0021] The image acquisition unit GSY acquires an image Im (also shown in FIG. 2) of at least one human subject HK under near-infrared light (for example, using a near-infrared camera). Since the image Im is captured under near-infrared light, it is a single-color image (for example, a monochrome image, or an image of only one color among multiple colors). Here, "near-infrared light" does not mean only near-infrared light in a narrow sense, but also means, for example, a single color of visible light (for example, green light) in a broad sense.

[0022] The skin region detection unit HRK detects the position of the skin of the subject HK in the image Im, for example, a skin region S (also shown in FIG. 2B) which is the region where the skin of the subject HK is located (for example, the facial skin of the subject HK), by setting landmarks LM1, LM2, LM3, ... in the image Im using a conventionally known method. The skin region detection unit HRK generates skin region information SJ indicating the skin region S of the subject HK.

[0023] Specifically, the skin region information SJ indicates, for example, whether or not the skin region S of the subject HK is present in the image Im, and the position and size of the skin region S of the subject HK in the image Im.

[0024] The measurement region setting unit KRS sets a plurality of measurement regions R1, R2, R3, R4, ... (also shown in FIG. 2C) in the skin region S of the subject HK, which are regions for measuring the brightness of the skin of the subject HK, to be used to estimate the pulse wave M of the subject HK. The measurement region setting unit KRS generates measurement region information RJ indicating the measurement regions R.

[0025] The plurality of measurement regions R1, R2, R3, R4, . . . are, for example, quadrilaterals (shown in FIG. 2C).

[0026] Specifically, the measurement region information RJ indicates, for example, the position and size of each measurement region R in the image Im, and indicates, for example, the coordinate positions of the four vertices of the measurement region R, which is a quadrangle.

[0027] As shown in FIG. 3, the point identification unit TTO identifies a first point P1 and a second point P2 within the measurement region R (e.g., within the measurement region R56) and along the direction HO in which the pulse wave M of the subject HK propagates in the image Im.

[0028] Here, "identifying a first point P1 and a second point P2 along the direction HO in which the pulse wave M of the subject HK propagates in the image Im" should be interpreted broadly and includes at least the following: (1) (1A) Along the direction HO in which the pulse wave M of the subject HK propagates in the image Im, or (1B) along the direction HO in which the pulse wave M of the subject HK will propagate in the image Im, for example, as shown in the lower left of Figure 3, identify at least one of the first points P1, P3, P5, P7, and P9 and one of the second points P2, P4, P6, P8, and P10 (identify two or more points). (2) In order to achieve the above-mentioned (1), specify an area where the first point P1, the second point P2, and two or more points among points P3 to P10 are likely to exist (for example, an area where the first point P1, the second point P2, point P5, and point P6 are likely to exist). Here, as suggested in the lower left of Figure 3, for example, the imaginary line (not shown; the same applies below) passing through the first point P1 and the second point P2, the imaginary line passing through the points P3 and P4, the imaginary line passing through the points P5 and P6, the imaginary line passing through the points P7 and P8, and the imaginary line passing through the points P9 and P10 are each along the above-mentioned direction HO, i.e., parallel to the direction HO.

[0029] As shown in Figures 3 and 5, the luminance signal generation unit KSS generates a first luminance signal KI1, a second luminance signal KI2, a third luminance signal KI3, and a fourth luminance signal KI4 using the luminance at the first point P1 and the luminance at the second point P2, as well as weighting functions A and B.

[0030] Weighting function A and weighting function B correspond to the "first distribution" and the "second distribution".

[0031] More specifically, the luminance signal generation unit KSS convolves the luminance of a first point P1 in the measurement region R and the luminance of a second point P2 in the measurement region R with the weighting function A. As a result, as shown in FIG. 3, the luminance signal generation unit KSS generates a first luminance signal KI1 indicating the luminance at the first point P1 after being weighted by the weighting function A, and a second luminance signal KI2 indicating the luminance at the second point P2 after being weighted by the weighting function A.

[0032] The luminance signal generation unit KSS also convolves the luminance of the first point P1 in the measurement region R and the luminance of the second point P2 in the measurement region R with weighting function B instead of the above-mentioned weighting function A. As a result, the luminance signal generation unit KSS generates a third luminance signal KI3 indicating the luminance at the first point P1 after being weighted by weighting function B, and a fourth luminance signal KI4 indicating the luminance at the second point P2 after being weighted by weighting function B, as shown in FIG.

[0033] FIG. 7 shows the configurations of weighting functions A and B according to the first embodiment.

[0034] In FIG. 7, the horizontal axis indicates the position on the coordinate system, i.e., the distance from the center of gravity O, while the vertical axis indicates the magnitude of the weighting. As shown in FIG. 7, weighting function A and weighting function B have spatially different weightings. Specifically, weighting function A has a small weight spread, which is the degree of spread of the weighting (vertical axis) relative to the distance (horizontal axis), while weighting function B has a wide weight spread. More specifically, comparing weighting function A and weighting function B, weighting function A has weighting values ​​concentrated within a shorter distance range than weighting function B; in other words, weighting function B has weighting values ​​spread over a longer distance range than weighting function A. In other words, weighting function A has weighting values ​​that are substantially 0 at relatively short distances, while weighting function B has weighting values ​​that are substantially 0 at relatively long distances.

[0035] For example, a two-dimensional Gaussian function can be used as weighting function A and weighting function B. Weighting function A and weighting function B using a two-dimensional Gaussian function can be expressed using, for example, the coordinate position, the weighting coefficient at the coordinate position, the coordinate position of the center of gravity, and the spread of the Gaussian function as parameters.

[0036] Returning to Figure 1, the explanation will continue.

[0037] As shown in Figures 4 and 5, the pulse wave source signal generator MMS generates a first pulse wave source signal MM1 by combining the first luminance signal KI1 and the second luminance signal KI2. As shown in Figures 4 and 5, the pulse wave source signal generator MMS also generates a second pulse wave source signal MM2 by combining the third luminance signal KI3 and the fourth luminance signal KI4.

[0038] As shown in FIG. 5, the first pulse wave source signal MM1 has a characteristic in which the peaks of the pulse wave are emphasized, while the second pulse wave source signal MM2 has a characteristic in which the peaks of the pulse wave are not emphasized.

[0039] As shown in FIGS. 4 and 6, the pulse wave signal generator MSS generates a pulse wave signal MY indicating the pulse wave M of the subject HK by subtracting the second pulse wave source signal MM2 from the first pulse wave source signal MM1.

[0040] FIG. 8 shows a luminance signal of a comparative example.

[0041] The pulse wave detection device described in Patent Document 1, which is a comparative example, subtracts a luminance signal (red component) KIR from a luminance signal (green component) KIG, or subtracts a luminance signal (blue component) KIB from a luminance signal (green component) KIG, as shown in Figure 8. By performing this subtraction, the pulse wave detection device improves the accuracy of estimating the pulse wave M of the subject HK by removing noise, which is an error in estimating the pulse wave M of the subject HK due to movement of the subject HK's head.

[0042] The above-mentioned first pulse wave original signal MM1 corresponds to the luminance signal (green component) KIG of the comparative example, while the above-mentioned second pulse wave original signal MM2 corresponds to the luminance signal (red component) KIR or the luminance signal (blue component) KIB of the comparative example.

[0043] Returning to Figure 1, the explanation will continue.

[0044] The pulse wave estimation unit MSU estimates the pulse wave M of the subject HK based on the pulse wave signal MY (shown in FIGS. 4 and 6). The pulse wave estimation unit MSU generates pulse wave information MJ indicating the pulse wave M.

[0045] Pulse wave information MJ represents, for example, the pulse rate (beats per minute). As the pulse wave rate, which is pulse wave information MJ, pulse wave estimation unit MSU uses, for example, the peak frequency in a frequency power spectrum obtained by performing a Fourier transform on pulse wave signal MY.

[0046] Configuration of First Embodiment FIG. 9 shows the hardware configuration of the pulse wave estimation device MSD of the first embodiment.

[0047] As shown in FIG. 9, the pulse wave estimation device MSD includes a processing circuit SH, and may further include an input circuit NY and an output circuit SY as required.

[0048] The processing circuit SH is dedicated hardware that realizes the functions of the image acquisition unit GSY to the pulse wave estimation unit MSU (shown in FIG. 1) of the pulse wave estimation device MSD.

[0049] The processing circuit SH may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof.

[0050] The input circuit NY and the output circuit SY exchange inputs and outputs related to the operation of the processing circuit SH with, for example, the outside of the pulse wave estimation device MSD.

[0051] FIG. 10 shows a hardware configuration based on software realization of the pulse wave estimation device MSD of the first embodiment.

[0052] As shown in FIG. 10, the pulse wave estimation device MSD includes a processor PR and a memory circuit KI, and may further include an input circuit NY and an output circuit SY as required.

[0053] The processor PR is a CPU (also called a central processing unit, processing device, arithmetic unit, microprocessor, microcomputer, or DSP (Digital Signal Processing)) that executes programs. The processor PR realizes the functions of the image acquisition unit GSY to the pulse wave estimation unit MSU (shown in FIG. 1) of the pulse wave estimation device MSD.

[0054] The processor PR realizes the above-mentioned functions by software, firmware, or a combination of software and firmware. The software and firmware are written as programs and stored in the memory circuit KI.

[0055] The processor PR realizes the above-mentioned functions by reading and executing the above-mentioned programs from the memory circuit KI. The above-mentioned programs can also be said to cause a computer to execute the procedures and methods of the image acquisition unit GSY through the pulse wave estimation unit MSU.

[0056] Here, the memory circuit KI is, for example, a non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), as well as a magnetic disk, flexible disk, optical disk, compact disk, mini disk, DVD (Digital Versatile Disc), etc.

[0057] Of the functions of the image acquisition unit GSY to the pulse wave estimation unit MSU (shown in Figure 1), some of the functions may be realized by a processing circuit SH (shown in Figure 9), while other functions may be realized by a processor PR (shown in Figure 10).

[0058] As described above, the functions of image acquisition unit GSY through pulse wave estimation unit MSU of pulse wave estimation device MSD can be realized by hardware, software, firmware, or a combination of these.

[0059] The input circuit NY and the output circuit SY exchange inputs and outputs related to the operation of the processor PR with, for example, the outside of the pulse wave estimation device MSD.

[0060] Operation of the First Embodiment The operation of the pulse wave estimation device MSD of the first embodiment will be described.

[0061] 11 is a flowchart showing the operation of pulse wave estimation device MSD of embodiment 1. The operation of pulse wave estimation device MSD of embodiment 1 will be described with reference to the flowchart of FIG.

[0062] Step ST11: The image acquisition unit GSY (shown in FIG. 1) acquires an image Im (shown in FIGS. 1, 2A, etc.) of the subject HK (shown in FIG. 1).

[0063] Step ST12: The skin region detection unit HRK (shown in FIG. 1) detects the skin region S (shown in FIGS. 1, 2B, etc.) of the subject HK in the image Im of the subject HK. The skin region detection unit HRK generates skin region information SJ indicating the skin region S of the subject HK.

[0064] Step ST13: The measurement region setting unit KRS (shown in FIG. 1) sets a plurality of measurement regions R (shown in FIGS. 1, 2C, etc.) in the skin region S of the subject HK.

[0065] Step ST14: The point identification unit TTO (shown in Figure 1) identifies a first point P1 and a second point P2 (shown in Figures 3, 5, etc.) within the measurement region R, for example, within the measurement region R56 (shown in Figure 3), and along the direction HO (shown in Figure 3) in which the pulse wave M of the subject HK propagates.

[0066] Step ST15: The luminance signal generation unit KSS (shown in Figure 1) generates a first luminance signal KI1 to a fourth luminance signal KI4 (shown in Figures 3, 4, etc.) using the luminance at a first point P1 in the measurement area R and the luminance at a second point P2 in the measurement area R (shown in Figure 3), as well as weighting functions A and B (shown in Figure 3, etc.).

[0067] Step ST16: The pulse wave signal generator MSS (shown in FIG. 1) generates a first pulse wave original signal MM1 (shown in FIGS. 4 and 5) by combining the first luminance signal KI1 and the second luminance signal KI2. The pulse wave original signal generator MMS also generates a second pulse wave original signal MM2 (shown in FIGS. 4 and 5) by combining the third luminance signal KI3 and the fourth luminance signal KI4.

[0068] Step ST17: The pulse wave signal generating unit MSS (shown in Figure 1) generates a pulse wave signal MY (shown in Figures 4 and 6) indicating the pulse wave M of the subject HK by performing subtraction between the first pulse wave source signal MM1 and the second pulse wave source signal MM2, more specifically, by subtracting the second pulse wave source signal MM2 from the first pulse wave source signal MM1.

[0069] Step ST18: The pulse wave estimation unit MSU (shown in FIG. 1) estimates the pulse wave M (shown in FIG. 1) of the subject HK based on the pulse wave signal MY. The pulse wave estimation unit MSU generates pulse wave information MJ (shown in FIG. 1) indicating the pulse wave M.

[0070] Effects of the First Embodiment As described above, in the pulse wave estimation device MSD of embodiment 1, the luminance signal generation unit KSS generates the first luminance signal KI1 to the fourth luminance signal KI4 based on the luminance at a first point P1 within the measurement region R, the luminance at a second point P2 within the measurement region R, weighting function A, and weighting function B. The pulse wave source signal generation unit MMS generates the first pulse wave source signal MM1 and the second pulse wave source signal MM2 using the first luminance signals KI1 to KI4. The pulse wave signal generation unit MSS generates the pulse wave signal MY using the first pulse wave source signal MM1 and the second pulse wave source signal MM2. The pulse wave estimation unit MSU estimates the pulse wave M based on the pulse wave signal MY. This improves the accuracy of estimating the pulse wave M of the subject HK, even when using an image Im of the subject HK captured under near-infrared light.

[0071] <Modification of the First Embodiment> A modification of the first embodiment will be described.

[0072] FIG. 12 shows the configuration of a pulse wave estimation device MSD according to a modification of the first embodiment.

[0073] The first pulse wave original signal MM1 and the second pulse wave original signal MM2 described in the first embodiment contain various noise components NS (not shown), for example, noise components NS caused by defects in some of the elements that make up the image acquisition unit GSY. In order to remove the noise components NS from the first pulse wave original signal MM1 and the second pulse wave original signal MM2, it is desirable to apply a filter process FS (not shown) to the first pulse wave original signal MM1 and the second pulse wave original signal MM2.

[0074] The pulse wave estimation device MSD of the modified example of the first embodiment further includes a noise removal unit NJO to perform the above-mentioned filtering process FS.

[0075] As shown in FIG. 12, the noise elimination unit NJO is disposed between the pulse wave original signal generation unit MMS and the pulse wave signal generation unit MSS.

[0076] The noise removal unit NJO performs the above-mentioned filtering process FS on the first pulse wave original signal MM1 and the second pulse wave original signal MM2, for example, by using a low-pass filter, a high-pass filter, or a band-pass filter.

[0077] In the following description, it is assumed that the noise removal unit NJO performs band-pass filtering as the filtering process FS.

[0078] The bandpass filter is, for example, a Butterworth filter, and it is desirable that the lower cutoff frequency of the bandpass filter is, for example, 0.5 Hz, and the higher cutoff frequency is, for example, 5.0 Hz.

[0079] The noise removal unit NJO removes noise components NS contained in the first pulse wave original signal MM1 and the second pulse wave original signal MM2 from the first pulse wave original signal MM1 and the second pulse wave original signal MM2, and outputs the first pulse wave original signal MM1 from which the noise components NS have been removed and the second pulse wave original signal MM2 from which the noise components NS have been removed to the pulse wave signal generation unit MSS.

[0080] In the pulse wave estimation device MSD of this variation of embodiment 1, the noise removal unit NJO applies filter processing FS to the first pulse wave source signal MM1 and the second pulse wave source signal MM2. This removes noise components NS contained in the first pulse wave source signal MM1 and the second pulse wave source signal MM2. As a result, the noise components NS are removed from the pulse wave signal MY generated by the pulse wave source signal generation unit MMS, and the accuracy of the estimation of the pulse wave M of the subject HK by the pulse wave estimation unit MSU can be improved compared to embodiment 1.

[0081] Embodiment 2. Second Embodiment A pulse wave estimation device MSD according to a second embodiment will be described.

[0082] Unlike the pulse wave estimation device MSD of embodiment 1 (shown in FIG. 1) which estimates the pulse wave M of subject HK based on the same single measurement area R, for example, measurement area R56 (shown in FIG. 6), the pulse wave estimation device MSD of embodiment 2 estimates the pulse wave M of subject HK based on two mutually different measurement areas R, for example, measurement areas R1 and R2 (shown in FIG. 2C).

[0083] <Functions of the second embodiment> FIG. 13 is a functional block diagram of the pulse wave estimation device MSD of the second embodiment.

[0084] As is clear from a comparison of Fig. 13 with Fig. 1 (functional block diagram of pulse wave estimation device MSD of embodiment 1), pulse wave estimation device MSD of embodiment 2 basically has the same functions as pulse wave estimation device MSD of embodiment 1. However, pulse wave estimation device MSD of embodiment 2 differs from pulse wave estimation device MSD of embodiment 1 in that it further includes an averaging unit HKK.

[0085] The averaging unit HKK corresponds to the "averaging unit."

[0086] The averaging unit HKK averages the pulse wave signal MY(R1) for the measurement region R1 and the pulse wave signal MY(R2) for the measurement region R2.

[0087] <Configuration of Second Embodiment> The hardware configuration of the pulse wave estimation device MSD of embodiment 2 and the hardware configuration based on software implementation are similar to the hardware configuration of the pulse wave estimation device MSD of embodiment 1 (shown in FIG. 9) and the hardware configuration based on software implementation (shown in FIG. 10).

[0088] <Operation of the Second Embodiment> FIG. 14 is a flowchart showing the operation of the pulse wave estimation device MSD of the second embodiment.

[0089] FIG. 15 shows the operation of the pulse wave estimation device MSD of embodiment 2 up to generation of the first luminance signal KI1(R1) to the fourth luminance signal KI4(R1) and the first luminance signal KI1(R1) to the fourth luminance signal KI4(R2).

[0090] FIG. 16 shows the operation of pulse wave estimation device MSD of embodiment 2 up to generation of average pulse wave signal HMY.

[0091] The operation of the pulse wave estimation device MSD of the second embodiment will be described with reference to FIGS.

[0092] Step ST21: The image acquisition unit GSY (shown in FIG. 13) acquires an image Im (shown in FIG. 15) of the subject HK (shown in FIG. 13) in the same manner as in step ST11 of the first embodiment.

[0093] Step ST22: The skin region detection unit HRK (shown in FIG. 13) detects the skin region S (shown in FIG. 15) of the subject HK in the image Im of the subject HK, similar to step ST12 in embodiment 1. The skin region detection unit HRK generates skin region information SJ indicating the skin region S of the subject HK.

[0094] Step ST23: The measurement region setting unit KRS (shown in FIG. 13) sets a plurality of measurement regions R (shown in FIG. 15) in the skin region S of the subject HK, similarly to step ST13 in the first embodiment.

[0095] Step ST24: The point identification unit TTO (shown in Figure 13) differs from step ST14 in embodiment 1, and as shown in Figure 15, identifies a first point P1 and a second point P2 in the measurement area R1 and along the direction HO in which the pulse wave M of the subject HK propagates (for example, shown in Figure 3 in embodiment 1), and on the other hand, identifies a first point P1 and a second point P2 in a measurement area R2 different from the measurement area R1 and along the direction HO in which the pulse wave M of the subject HK propagates.

[0096] Step ST25: The luminance signal generation unit KSS (shown in FIG. 13) differs from step ST15 in embodiment 1, and as shown in FIG. 15, generates a first luminance signal KI1(R1) to a fourth luminance signal KI4(R1) using the luminance at a first point P1 in the measurement area R1, the luminance at a second point P2 in the measurement area R1, and weighting functions A and B, and generates a first luminance signal KI1(R2) to a fourth luminance signal KI4(R2) using the luminance at a first point P1 in the measurement area R2, the luminance at a second point P2 in the measurement area R2, and weighting functions A and B.

[0097] Step ST26: The pulse wave signal generator MSS (shown in FIG. 13) differs from step ST16 in the first embodiment in that, for measurement region R1, it generates a first pulse wave original signal MM1(R1) by combining the first luminance signal KI1(R1) and the second luminance signal KI2(R1), and generates a second pulse wave original signal MM2(R1) by combining the third luminance signal KI3(R1) and the fourth luminance signal KI4(R1), as shown in FIG. 16. Meanwhile, for measurement region R2, the pulse wave signal generator MSS generates a first pulse wave original signal MM1(R2) by combining the first luminance signal KI1(R2) and the second luminance signal KI2(R2), and generates a second pulse wave original signal MM2(R2) by combining the third luminance signal KI3(R2) and the fourth luminance signal KI4(R2).

[0098] Step ST27: The pulse wave signal generation unit MSS (shown in FIG. 13) differs from step ST17 in embodiment 1 in that, for measurement region R1, it subtracts the second pulse wave original signal MM2(R1) from the first pulse wave original signal MM1(R1), as shown in FIG. 16, to generate a pulse wave signal MY(R1) that indicates the pulse wave M of subject HK. On the other hand, for measurement region R2, the pulse wave signal generation unit MSS subtracts the second pulse wave original signal MM2(R2) from the first pulse wave original signal MM1(R2), to generate a pulse wave signal MY(R2) that indicates the pulse wave M of subject HK.

[0099] Step ST28: The averaging unit HKK (shown in FIG. 13) averages the pulse wave signal MY(R1) for measurement region R1 and the pulse wave signal MY(R2) for measurement region R2, as shown in FIG. 16. As a result, the averaging unit HKK generates an averaged pulse wave signal HMY, which is a single averaged signal, as shown in FIG.

[0100] Step ST29: The pulse wave estimation unit MSU (shown in FIG. 13) estimates the pulse wave M of the subject HK based on the average pulse wave signal HMY. The pulse wave estimation unit MSU generates pulse wave information MJ indicating the pulse wave M.

[0101] Effects of the Second Embodiment As described above, in the pulse wave estimation device MSD of embodiment 2, the image acquisition unit GSY through the pulse wave signal generation unit MSS cooperate to generate pulse wave signals MY(R1) and MY(R2) for two mutually distinct measurement regions R, i.e., measurement regions R1 and R2. The averaging unit HKK averages the pulse wave signals MY(R1) and MY(R2) to generate a single averaged signal, the average pulse wave signal HMY. The pulse wave estimation unit MSU estimates the pulse wave M of the subject HK based on the average pulse wave signal HMY, instead of the pulse wave signal MY of embodiment 1. As a result, even if the measurement region R1 is partially obstructed due to the presence of an obstacle between the subject HK and the pulse wave estimation device MSD, i.e., even if a portion of the measurement region R1 is not captured in the image Im of the subject HK, the pulse wave M of the subject HK can be estimated with the same accuracy as in embodiment 1 by also using the measurement region R2.

[0102] Embodiment 3. Third Embodiment A pulse wave estimation device MSD according to a third embodiment will be described.

[0103] Unlike the pulse wave estimation device MSD of embodiment 1, which subtracts the second pulse wave source signal MM2 from the first pulse wave source signal MM1 to estimate the pulse wave M of subject HK, the pulse wave estimation device MSD of embodiment 3 subtracts the result of multiplying the second pulse wave source signal MM2 by a coefficient (multiplication coefficient c, described below) from the first pulse wave source signal MM1.

[0104] <Functions of the Third Embodiment> The pulse wave estimation device MSD of embodiment 3 basically has the same functions (shown in FIG. 1) as the pulse wave estimation device MSD of embodiment 1. However, the pulse wave estimation device MSD of embodiment 3 differs from the pulse wave estimation device MSD of embodiment 1 in that the pulse wave source signal generation unit MMS of embodiment 3 has functions different from those of the pulse wave source signal generation unit MMS of embodiment 1.

[0105] FIG. 17 is a functional block diagram of the pulse wave source signal generating unit MMS of the third embodiment.

[0106] As shown in FIG. 17, the pulse wave original signal generating unit MMS of the third embodiment has a coefficient calculation function KE, a multiplication function JO, and a subtraction function GE.

[0107] The pulse wave source signal generating unit MMS of the third embodiment also handles first pulse wave source signal information W, second pulse wave source signal information M, and coefficient information C.

[0108] The relationship between each piece of information and each signal is as follows: (A) First pulse wave source signal information W indicates the first pulse wave source signal MM1. (B) Second pulse wave source signal information M indicates a second pulse wave source signal MM2. (C) Coefficient information C indicates a multiplication coefficient c.

[0109] The multiplication coefficient c corresponds to the "coefficient."

[0110] The pulse wave signal generating unit MSS generates a pulse wave signal MY based on the first pulse wave source signal information W and the second pulse wave source signal information M, in other words, generates a pulse wave signal MY based on the first pulse wave source signal MM1 and the second pulse wave source signal MM2.

[0111] More specifically, the pulse wave signal generating unit MSS performs the following. (1) As the coefficient calculation function KE, coefficient information C indicating a multiplication coefficient c, which is a coefficient for multiplying the second pulse wave source signal information M, is calculated based on the first pulse wave source signal information W and the second pulse wave source signal information M. (2) As the multiplication function JO, the second pulse wave original signal MM2 is multiplied by the multiplication coefficient c in (1) above. (3) The result of the multiplication in (2) above is subtracted from the first pulse wave original signal MM1.

[0112] Pulse wave signal generator MSS performs steps (1) to (3) to generate pulse wave signal MY.

[0113] The coefficient information C described above indicates the amount, ie, magnitude, of the second pulse wave source signal information M to be subtracted.

[0114] More specifically, the value of the multiplication coefficient c is adjusted so that the components necessary to estimate the pulse wave M of the subject HK remain in the pulse wave signal MY obtained by subtracting the result obtained by multiplying the second pulse wave original signal MM2 by the multiplication coefficient c from the first pulse wave original signal MM1.

[0115] When the value of the multiplication coefficient c is large, the result of multiplying the second pulse wave original signal MM2 by the multiplication coefficient c is large; conversely, when the multiplication coefficient c is small, the result of multiplying the second pulse wave original signal MM2 by the multiplication coefficient c is small.

[0116] The pulse wave signal generating unit MSS calculates coefficient information C so as to minimize the difference between the first pulse wave source signal information W and the result obtained by multiplying the second pulse wave source signal information M by coefficient information C. This allows the pulse wave signal generating unit MSS to adjust the amount of second pulse wave source signal information M included in the first pulse wave source signal information W using coefficient information C.

[0117] <Configuration of the Third Embodiment> The hardware configuration of the pulse wave estimation device MSD of embodiment 3 and the hardware configuration based on software implementation are similar to the hardware configuration of the pulse wave estimation device MSD of embodiment 1 (shown in FIG. 9) and the hardware configuration based on software implementation (shown in FIG. 10).

[0118] <Operation of the Third Embodiment> The operation of the pulse wave estimation device MSD of embodiment 3 is basically the same as the operation of the pulse wave estimation device MSD of embodiment 1 (shown in FIG. 11). However, the pulse wave estimation device MSD of embodiment 3 differs from the pulse wave estimation device MSD of embodiment 1 in the operation of the pulse wave signal generation unit MSS. The following description will focus on the operation of the pulse wave signal generation unit MSS of embodiment 3.

[0119] 18 is a flowchart showing the operation of the pulse wave estimation device MSD of embodiment 3. The operation of the pulse wave estimation device MSD of embodiment 3 will be described with reference to the flowchart of FIG.

[0120] Step ST17-1: The pulse wave signal generation unit MSS (shown in FIG. 17) receives the first pulse wave source signal MM1 and the second pulse wave source signal MM2 (shown in FIGS. 17, 1, and 4) from the preceding pulse wave source signal generation unit MMS (shown in FIG. 1). The pulse wave signal generation unit MSS, as a coefficient calculation function KE, calculates a multiplication coefficient c based on the first pulse wave source signal information W and the second pulse wave source signal information M.

[0121] Step ST17-2: The pulse wave signal generator MSS uses the multiplication function JO to multiply the second pulse wave original signal MM2 by the multiplication coefficient c calculated in step ST17-1 above.

[0122] Step ST17-3: The pulse wave signal generation unit MSS, using its subtraction function GE, subtracts the result of the multiplication performed in step ST17-2 from the first pulse wave original signal MM1. The pulse wave signal generation unit MSS calculates a pulse wave signal MY through this subtraction. The pulse wave signal generation unit MSS outputs the calculated pulse wave signal MY to the pulse wave estimation unit MSU in the subsequent stage.

[0123] Step ST17-3 and subsequent steps: Pulse wave estimation unit MSU estimates pulse wave M of subject HK based on pulse wave signal MY received from pulse wave signal generation unit MSS and generated through steps ST17-1 to ST17-3 described above.

[0124] Effect of the Third Embodiment As described above, in the pulse wave estimation device MSD of embodiment 3, the pulse wave signal generation unit MSS calculates the pulse wave signal MY by subtracting the result obtained by multiplying the second pulse wave source signal MM2 by the multiplication coefficient c from the first pulse wave source signal MM1. This prevents signal components useful for estimating the pulse wave M of the subject HK, which are contained in the first pulse wave source signal MM1 and the second pulse wave source signal MM2, from being unnecessarily attenuated. As a result, the pulse wave estimation device MSD of embodiment 3 can estimate the pulse wave M of the subject HK with higher accuracy than the pulse wave estimation device MSD of embodiment 1, which simply subtracts the second pulse wave source signal MM2 from the first pulse wave source signal MM1.

[0125] Embodiment 4. Fourth Embodiment A pulse wave estimation device MSD according to a fourth embodiment will be described.

[0126] The pulse wave estimation device MSD of embodiment 4 differs from the pulse wave estimation device MSD of embodiment 1 (shown in FIG. 1) in that it uses weighting functions such as weighting function A and weighting function B (shown in FIG. 3, for example) to generate a fourth luminance signal KI4 from a first luminance signal KI1, and does not use the weighting functions.

[0127] <Functions of the fourth embodiment> The pulse wave estimation device MSD of embodiment 4 basically has the same functions as the pulse wave estimation device MSD of embodiment 1. However, the pulse wave estimation device MSD of embodiment 4 differs from the pulse wave estimation device MSD of embodiment 1 in that the measurement region setting unit KRS, point identification unit TTO, and luminance signal generation unit KSS of embodiment 4 differ in function from the measurement region setting unit KRS, point identification unit TTO, and luminance signal generation unit KSS of embodiment 1.

[0128] FIG. 19 is a functional block diagram of the measurement region setting unit KRS, the point identification unit TTO, and the luminance signal generation unit KSS according to the fourth embodiment.

[0129] FIG. 20 shows the measurement region R, the first point P1 to the fourth point P4, and the first luminance signal KI1 to the fourth luminance signal KI4 of the fourth embodiment.

[0130] As shown in FIGS. 19 and 20, the measurement region setting unit KRS has a function of setting a first measurement region (for example, measurement region R34) and a function of setting a second measurement region (for example, measurement region R56).

[0131] Here, as shown in FIG. 20, measurement region R34 and measurement region R56 share a common center of gravity O, which is the reference position on the coordinate system of measurement region R34, and a common center of gravity O, which is the reference position on the coordinate system of measurement region R56. Measurement region R34 and measurement region R56 have a relationship in which measurement region R56 is larger than measurement region R34. Alternatively, measurement region R56 may have a relationship in which measurement region R56 is smaller than measurement region R34. Measurement region R34 and measurement region R56 share a corresponding relationship between a position on the coordinate system and a magnitude of weighting specified by the position on the coordinate system, and share, for example, one weight function (e.g., weight function A or weight function B (shown in FIG. 7)).

[0132] The point identification unit TTO identifies a first point P1 and a second point P2 in the first measurement region, for example, in the measurement region R34, along the direction HO in which the pulse wave M of the subject HK propagates, as shown in Fig. 20. The point identification unit TTO also sets a third point P3 and a fourth point P4 in the second measurement region, for example, in the measurement region R56, along the direction HO in which the pulse wave M of the subject HK propagates, as shown in Fig. 20.

[0133] 20, since the measurement region R56 is wider than the measurement region R34, the distance between the third point P3 and the fourth point P4 is longer than the distance between the first point P1 and the second point P2. In contrast to the above, when the measurement region R56 is narrower than the measurement region R34, the distance between the third point P3 and the fourth point P4 is shorter than the distance between the first point P1 and the second point P2.

[0134] The luminance signal generating unit KSS outputs a first luminance signal KI1 indicating the luminance of a first point P1 in a first measurement region, for example, measurement region R34, and a second luminance signal KI2 indicating the luminance of a second point P2 in measurement region R34, as shown in Fig. 20. The pulse wave signal generating unit MSS also outputs a third luminance signal KI3 indicating the luminance of a third point P3 in a second measurement region, for example, measurement region R56, and a fourth luminance signal KI4 indicating the luminance of a fourth point P4 in measurement region R56, as shown in Fig. 20.

[0135] <Configuration of Fourth Embodiment> The hardware configuration of the pulse wave estimation device MSD of embodiment 4 and the hardware configuration based on software implementation are similar to the hardware configuration of the pulse wave estimation device MSD of embodiment 1 (shown in FIG. 9) and the hardware configuration based on software implementation (shown in FIG. 10).

[0136] <Operation of the Fourth Embodiment> 21 is a flowchart showing the operation of the pulse wave estimation device MSD of embodiment 4. The operation of the pulse wave estimation device MSD of embodiment 4 will be described with reference to the flowchart of FIG.

[0137] Before step ST43: The pulse wave estimation device MSD of the fourth embodiment performs steps ST11 and ST12 (shown in FIG. 11).

[0138] Step ST43: The measurement area setting unit KRS (shown in FIG. 19) sets a first measurement area, i.e., a measurement area R34, in the skin area S, as shown in FIG. 20. The measurement area setting unit KRS also sets a second measurement area, i.e., a measurement area R56, in the skin area S, as shown in FIG.

[0139] Step ST44: The point identification unit TTO (shown in FIG. 19) sets the above-mentioned first point P1 and second point P2 in the first measurement region, i.e., measurement region R34, as shown in FIG. 20. The point identification unit TTO also sets the given first point P1 and second point P2 in the second measurement region, i.e., measurement region R56, as shown in FIG.

[0140] Step ST45: The luminance signal generation unit KSS (shown in FIG. 19) outputs a first luminance signal KI1 indicating the luminance of a first point P in the first measurement region, i.e., measurement region R34, and a second luminance signal KI2 indicating the luminance of a second point P in the measurement region R34, as shown in FIG. 20. The luminance signal generation unit KSS also outputs a third luminance signal KI3 indicating the luminance of a third point P3 in the second measurement region, i.e., measurement region R56, and a fourth luminance signal KI4 indicating the luminance of a fourth point P4 in the measurement region R56, as shown in FIG.

[0141] After step ST45: The pulse wave estimation device MSD of the fourth embodiment performs steps ST16 to ST18 (shown in FIG. 11).

[0142] Effects of the Fourth Embodiment As described above, the pulse wave estimation device MSD of embodiment 4 generates the first luminance signal KI1 to the fourth luminance signal KI4 in the same manner as the pulse wave estimation device MSD of embodiment 1, without using the weighting functions A and B of the pulse wave estimation device MSD of embodiment 1. As a result, the pulse wave estimation device MSD of embodiment 4 can improve the accuracy of estimating the pulse wave M of the subject HK while using the image Im of the subject HK captured under near-infrared light, just like the pulse wave estimation device MSD of embodiment 1.

[0143] Embodiment 5. Fifth Embodiment A state estimation system JSS according to the fifth embodiment will be described. <Functions of the fifth embodiment> FIG. 22 is a functional block diagram of the state estimation system JSS according to the fifth embodiment.

[0144] As shown in FIG. 22, the state estimation system JSS of the fifth embodiment includes a pulse wave estimation device MSD and a state estimation device JSD.

[0145] The pulse wave estimation device MSD is one of the pulse wave estimation devices MSD of the above-mentioned embodiment 1 to embodiment 4. Just as the pulse wave estimation devices MSD of embodiments 1 to 4 output a pulse wave M estimated for a subject HK (for example, as shown in FIG. 1), the pulse wave estimation device MSD outputs a pulse wave M estimated for a person HT (for example, a driver of a vehicle), more precisely, pulse wave information MJ of the person HT (for example, the pulse rate of the person HT).

[0146] The state estimation device JSD estimates the state of the person HT based on the pulse wave information MJ of the person HT, and more specifically, estimates, for example, the level of wakefulness KT of the driver of the vehicle. The level of wakefulness KT is expressed, for example, in two stages (1: drowsy state, 2: wakeful state).

[0147] A vehicle driver is generally in an alert state immediately after starting to drive the vehicle. Therefore, the state estimation device JSD estimates which of the two states the vehicle driver is in, for example, based on the pulse rate from the time the vehicle driver starts driving until 10 minutes have passed. More specifically, when it is determined that the pulse rate of the vehicle driver is significantly lower than the standard, the state estimation device JSD estimates that the level of alertness KT of the vehicle driver is "1: drowsy state."

[0148] The state estimation device JSD may emit, for example, a warning sound together with the degree of wakefulness KT being the above-mentioned "1: sleepy state."

[0149] The person HT may be a driver of a vehicle, or may be, for example, a person at home (for example, a person sitting in a living room) or a person working in an office (for example, a person traveling in an elevator).

[0150] Effect of the Fifth Embodiment As described above, the state estimation system JSS of the fifth embodiment includes one of the pulse wave estimation devices MSD of the first to fourth embodiments, and is therefore able to estimate the state of the person HT.

[0151] The above-described embodiments may be combined with each other without departing from the spirit of the present disclosure, and components in each embodiment may be appropriately deleted or modified, or other components may be added. [Industrial Applicability]

[0152] The pulse wave estimation device and the like according to the present disclosure can be used to improve the accuracy of estimating a subject's pulse wave even when using an image captured under near-infrared light. [Explanation of symbols]

[0153] A weighting function, B weighting function, GSY image acquisition unit, HK subject, HRK skin area detection unit, Im image, KI1 first luminance signal, KI2 second luminance signal, KI3 third luminance signal, KI4 fourth luminance signal, KRS measurement area setting unit, KSS luminance signal generation unit, M pulse wave, MM1 first pulse wave source signal, MM2 second pulse wave source signal, MMS pulse wave source signal generation unit, MSD pulse wave estimation device, MSS pulse wave signal generation unit, MSU pulse wave estimation unit, MY pulse wave signal, P1 first point, P2 second point, R measurement area, S skin area, TTO point identification unit.

Claims

1. an image acquisition unit that acquires an image of a person under near-infrared light; a skin area detection unit that detects a skin area, which is the location of the person's skin, from the image; a measurement area setting unit that sets a plurality of measurement areas in the skin area, the measurement areas being areas for measuring the luminance of the human skin; a point specifying unit that specifies a first point and a second point within one of the plurality of measurement regions and along a direction in which the pulse wave of the person propagates in the image; a first distribution and a second distribution having mutually different correspondences between positions on a coordinate system and magnitudes of weightings specified by the positions on the coordinate system, wherein the luminance of the first point and the luminance of the second point are superimposed on the first distribution to generate a first luminance signal indicating the luminance at the first point after being weighted by the first distribution, and a second luminance signal indicating the luminance at the second point after being weighted by the first distribution; and a luminance signal generating unit that generates a third luminance signal indicating the luminance at the first point after being weighted by the second distribution and a fourth luminance signal indicating the luminance at the second point after being weighted by the second distribution by superimposing the luminance at the first point and the luminance at the second point on the second distribution; generating a first pulse wave source signal by combining the first luminance signal and the second luminance signal; and a pulse wave source signal generating unit that generates a second pulse wave source signal by combining the third luminance signal and the fourth luminance signal; a pulse wave signal generating unit that generates a pulse wave signal indicative of the person's pulse wave by subtracting one of the first pulse wave source signal and the second pulse wave source signal from the other; a pulse wave estimation unit that estimates a pulse wave of the person based on the pulse wave signal; A pulse wave estimation device comprising:

2. an averaging unit that averages the pulse wave signals generated in the plurality of measurement regions to generate a single averaged pulse wave signal; the pulse wave estimation unit estimates a pulse wave of the person based on the averaged pulse wave signal. The pulse wave estimation device according to claim 1 .

3. The pulse wave signal generating unit generates the pulse wave signal. calculating a coefficient for adjusting the second pulse wave source signal so that a signal component for estimating the person's pulse wave remains in the pulse wave signal obtained by subtracting the second pulse wave source signal from the first pulse wave source signal; multiplying the second pulse wave original signal by the coefficient; and and subtracting the result of the multiplication from the first pulse wave original signal. The pulse wave estimation device according to claim 1 .

4. the measurement area setting unit sets, as the plurality of measurement areas, the one measurement area and another measurement area that has a common reference position on a coordinate system with the one measurement area and is wider or narrower than the one measurement area, and the one measurement area and the other measurement area share a correspondence relationship between a position on the coordinate system and a magnitude of a weight specified by the position on the coordinate system; the point specifying unit specifies a third point and a fourth point in the other measurement area along a direction in which the pulse wave of the person propagates, the third point and the fourth point being longer or shorter than a distance between the third point and the fourth point; the luminance signal generating unit outputs a first luminance signal indicating the luminance of the first point in the one measurement area, a second luminance signal indicating the luminance of the second point in the one measurement area, a third luminance signal indicating the luminance of the third point in the other measurement area, and a fourth luminance signal indicating the luminance of the fourth point in the other measurement area, instead of generating the first luminance signal, the second luminance signal, the third luminance signal, and the fourth luminance signal using the first distribution and the second distribution. The pulse wave estimation device according to claim 1 .

5. A pulse wave estimation device according to any one of claims 1 to 4, a state estimation device that estimates a state of the person based on the pulse wave output from the pulse wave estimation device; A state estimation system including:

6. The image acquisition unit acquires an image of a person under near-infrared light, a skin area detection unit that detects a skin area, which is a position of the person's skin, from the image; the measurement area setting unit sets a plurality of measurement areas in the skin area, which are areas for measuring the luminance of the person's skin; the point specifying unit specifies a first point and a second point within one of the plurality of measurement areas and along a direction in which the pulse wave of the person propagates in the image; the luminance signal generation unit generates a first luminance signal indicating the luminance at the first point after being weighted by the first distribution and a second luminance signal indicating the luminance at the second point after being weighted by the first distribution by superimposing the luminance at the first point and the luminance at the second point on the first distribution, the first distribution and the second distribution having mutually different correspondence relationships between positions on coordinates and magnitudes of weighting specified by the positions on coordinates; and generating a third luminance signal indicating the luminance at the first point after being weighted by the second distribution and a fourth luminance signal indicating the luminance at the second point after being weighted by the second distribution by superimposing the luminance at the first point and the luminance at the second point on the second distribution; the pulse wave source signal generating unit generates a first pulse wave source signal by combining the first luminance signal and the second luminance signal; and generating a second pulse wave source signal by combining the third luminance signal and the fourth luminance signal; the pulse wave signal generation unit generates a pulse wave signal indicative of the person's pulse wave by subtracting one of the first pulse wave source signal and the second pulse wave source signal from the other; the pulse wave estimation unit estimates a pulse wave of the person based on the pulse wave signal. Pulse wave estimation method.

7. the averaging unit averages the pulse wave signals generated in the plurality of measurement regions to generate a single averaged pulse wave signal; the pulse wave estimation unit estimates a pulse wave of the person based on the averaged pulse wave signal. The pulse wave estimation method according to claim 6.

8. The pulse wave signal generating unit generates the pulse wave signal. calculating a coefficient for adjusting the second pulse wave source signal so that a signal component for estimating the person's pulse wave remains in the pulse wave signal obtained by subtracting the second pulse wave source signal from the first pulse wave source signal; multiplying the second pulse wave original signal by the coefficient; and and subtracting the result of the multiplication from the first pulse wave original signal. The pulse wave estimation method according to claim 6.

9. the measurement area setting unit sets, as the plurality of measurement areas, the one measurement area and another measurement area that has a common reference position on a coordinate system with the one measurement area and is wider or narrower than the one measurement area, and the one measurement area and the other measurement area share a correspondence relationship between a position on the coordinate system and a magnitude of a weight specified by the position on the coordinate system; the point specifying unit specifies a third point and a fourth point in the other measurement area along a direction in which the pulse wave of the person propagates, the third point and the fourth point being longer or shorter than a distance between the third point and the fourth point; the luminance signal generating unit outputs a first luminance signal indicating the luminance of the first point in the one measurement area, a second luminance signal indicating the luminance of the second point in the one measurement area, a third luminance signal indicating the luminance of the third point in the other measurement area, and a fourth luminance signal indicating the luminance of the fourth point in the other measurement area, instead of generating the first luminance signal, the second luminance signal, the third luminance signal, and the fourth luminance signal using the first distribution and the second distribution. The pulse wave estimation method according to claim 6.

10. A pulse wave estimation method according to any one of claims 6 to 9; a state estimation method for estimating a state of the person based on the pulse wave output from the pulse wave estimation method; A state estimation method including:

Images