Pulse wave estimation device, pulse wave estimation method, state estimation system, and state estimation method

Abstract

The present invention provides a pulse wave estimation device (MSD) including: an image acquisition unit (GSY) that acquires an image (Im) of a person (HK) captured under near-infrared light; a skin area detection unit (HRK) that detects from the image (Im) a skin area (S), which is the location of the skin of the person (HK); a measurement area setting unit (KRS) that sets, in the skin area (S), a plurality of measurement areas (R), which are areas for measuring the luminance of the skin of the person (HK); a point identification unit (TTO) that identifies a first point (P1) and a second point (P2) which are in one of the plurality of measurement areas (R) and along a direction (HO) in which a pulse wave (M) of the person (HK) propagates in the image (Im); a luminance signal generation unit (KSS) that, out of a first distribution (A) and a second distribution (B) with mutually different correspondence relationships between a coordinate position and the size of a weighting specified by the coordinate position, superimposes the luminance that the first point (P1) has and the luminance that the second point (P2) has with the first distribution (A) to thereby generate a first luminance signal (KI1) indicating the luminance at the first point (P1) after being weighted by the first distribution (A) and a second luminance signal (KI2) indicating the luminance at the second point (P2) after being weighted by the first distribution (A), and superimposes the luminance that the first point (P1) has and the luminance that the second point (P2) has with the second distribution (B) to thereby generate a third luminance signal (KI3) indicating the luminance at the first point (P1) after being weighted by the second distribution (B) and a fourth luminance signal (KI4) indicating the luminance at the second point (P2) after being weighted by the second distribution (B); a pulse wave source signal generation unit (MMS) that generates a first pulse wave source signal (MM1) by combining the first luminance signal (KI1) with the second luminance signal (KI2) and generates a second pulse wave source signal (MM2) by combining the third luminance signal (KI3) with the fourth luminance signal (KI4); a pulse wave signal generation unit (MSS) that generates a pulse wave signal (MY) indicating the pulse wave (M) of the person (HK) by subtracting one of the first pulse wave source signal (MM1) or the second pulse wave source signal (MM2) from the other; and a pulse wave estimation unit (MSU) that estimates the pulse wave (M) of the person (HK) on the basis of the pulse wave signal (MY).

Description

Pulse wave estimation device, pulse wave estimation method, condition estimation system, and condition estimation method

[0001] The present disclosure relates to a pulse wave estimation device, a pulse wave estimation method, a condition estimation system, and a condition estimation method.

[0002] The pulse wave detection device described in Patent Document 1 subtracts, for example, the luminance of the red component, which has a small pulse wave component and fluctuates with the movement of the user's head, from the luminance of the green component, which has a large pulse wave component and fluctuates with the movement of the user's head, in an image of a user captured in the visible light band by an RGB camera. By this subtraction, the pulse wave detection device described above removes errors in estimating the user's pulse wave caused by the movement of the user's head, i.e., noise, thereby improving the accuracy of estimating the pulse wave.

[0003] International Publication No. 2016-006027

[0004] However, when using the pulse wave detection device described above to monitor the pulse rate of a user sleeping at night, for example, it is necessary to use a near-infrared camera capable of capturing images in the near-infrared wavelength band instead of the RGB camera capable of capturing images in the visible light band. The pulse wave detection device has a problem in that when the near-infrared camera is used, it is not possible to estimate the pulse wave by subtracting the luminance of the green component from the luminance of the red component.

[0005] An object of the present disclosure is to provide a pulse wave estimation device, a pulse wave estimation method, a condition estimation system, and a condition estimation method that are capable of estimating a subject's pulse wave even using an image captured under near-infrared light.

[0006] In order to solve the above-mentioned problems, a pulse wave estimation device according to the present disclosure includes an image acquisition unit that acquires an image of a person taken under near-infrared light, a skin region detection unit that detects a skin region, which is the location of the person's skin, from the image, a measurement region setting unit that sets a plurality of measurement regions in the skin region, which are regions for measuring the luminance of the person's skin, a point identification unit that identifies a first point and a second point within one of the plurality of measurement regions and along a direction in which the person's pulse wave propagates in the image, and a first luminance signal indicating the luminance at the first point after being weighted by the first distribution and a second luminance signal indicating the luminance at the second point after being weighted by the first distribution by superimposing the first distribution on the luminance at the first point and the luminance at the second point, which are a first distribution and a second distribution that have mutually different correspondence relationships between positions on the coordinates and magnitudes of weighting specified by the positions on the coordinates. a luminance signal generating unit that generates a third luminance signal indicating the luminance at the first point after being weighted by the second distribution and a fourth luminance signal indicating the luminance at the second point after being weighted by the second distribution by superimposing the luminance of the first point and the luminance of the second point on the second distribution; a pulse wave source signal generating unit that generates a first pulse wave source signal by combining the first luminance signal and the second luminance signal and generates the second pulse wave source signal by combining the third luminance signal and the fourth luminance signal; a pulse wave signal generating unit that generates a pulse wave signal indicating the person's pulse wave by subtracting one of the first pulse wave source signal and the second pulse wave source signal from the other; and a pulse wave estimating unit that estimates the person's pulse wave based on the pulse wave signal.

[0007] According to the pulse wave estimation device according to the present disclosure, it is possible to improve the accuracy of estimating the pulse wave of a subject even when using an image captured under near-infrared light.

[0008] 2A is a functional block diagram of the pulse wave estimation device MSD of embodiment 1. FIG. 2A shows an image Im of embodiment 1. FIG. 2B shows a measurement region R of embodiment 1. FIG. 2C shows a skin region S of embodiment 1. FIG. 2B shows a basic operation up to generation of the first luminance signal KI1 to the fourth luminance signal KI4 of embodiment 1. FIG. 2C shows a basic operation up to generation of the pulse wave signal MY of embodiment 1. FIG. 2C shows a detailed operation up to generation of the first pulse wave source signal MM1 to the second pulse wave source signal MM2 of embodiment 1. FIG. 2C shows a detailed operation up to generation of the pulse wave signal MY of embodiment 1. FIG. 2C shows the configurations of weighting functions A and B of embodiment 1. FIG. 2D shows a luminance signal of a comparative example. FIG. 2D shows a hardware configuration of the pulse wave estimation device MSD of embodiment 1. FIG. 2E shows a hardware configuration based on software realization of the pulse wave estimation device MSD of embodiment 1. FIG. 2F is a flowchart showing the operation of the pulse wave estimation device MSD of embodiment 1. FIG. 2F shows the configuration of a pulse wave estimation device MSD of a modified embodiment of embodiment 1. FIG. 2F is a functional block diagram of the pulse wave estimation device MSD of embodiment 2. 1 is a flowchart showing the operation of the pulse wave estimation device MSD of embodiment 2. FIG. 1 shows the operation of the pulse wave estimation device MSD of embodiment 2 up to the generation of the first luminance signal KI1(R1) to the fourth luminance signal KI4(R1) and the first luminance signal KI1(R1) to the fourth luminance signal KI4(R2). FIG. 2 shows the operation of the pulse wave estimation device MSD of embodiment 2 up to the generation of the average pulse wave signal HMY. FIG. 3 is a functional block diagram of a pulse wave source signal generator MMS of embodiment 3. FIG. 4 is a flowchart showing the operation of the pulse wave estimation device MSD of embodiment 3. FIG. 5 is a functional block diagram of a measurement region setting unit KRS, a point identification unit TTO, and a luminance signal generator KSS of embodiment 4. FIG. 6 shows the measurement region R, the first point P1 to the fourth point P4, and the first luminance signal KI1 to the fourth luminance signal KI4 of embodiment 4. FIG. 7 is a flowchart showing the operation of the pulse wave estimation device MSD of embodiment 4. FIG. 8 is a functional block diagram of a state estimation system JSS of embodiment 5.

[0009] An embodiment of a pulse wave estimation device according to the present disclosure will be described.

[0010] First Embodiment A pulse wave estimation device MSD according to a first embodiment will be described.

[0011] In the following, for ease of explanation and understanding, multiple names may be collectively referred to by one reference symbol. For example, multiple names "measurement region R1," "measurement region R2," "measurement region R3," "measurement region R4," etc. may be collectively referred to by one reference symbol "R."

[0012] <Functions of First Embodiment> FIG. 1 is a functional block diagram of a pulse wave estimation device MSD of the first embodiment.

[0013] FIG. 2 shows an image Im, a measurement region R, and a skin region S according to the first embodiment.

[0014] FIG. 3 shows the basic operation of generating the first luminance signal KI1 to the fourth luminance signal KI4 in the first embodiment.

[0015] FIG. 4 shows the basic operation up to the pulse wave signal MY in the first embodiment.

[0016] FIG. 5 shows the detailed operation of generating the first pulse wave original signal MM1 to the second pulse wave original signal MM2 in the first embodiment.

[0017] FIG. 6 shows detailed operations up to generation of the pulse wave signal MY in the first embodiment.

[0018] The functions of the pulse wave estimation device MSD of the first embodiment will be described below with reference to FIGS.

[0019] As shown in FIG. 1, the pulse wave estimation device MSD of embodiment 1 includes an image acquisition unit GSY, a skin area detection unit HRK, a measurement area setting unit KRS, a point identification unit TTO, a luminance signal generation unit KSS, a pulse wave original signal generation unit MMS, a pulse wave signal generation unit MSS, and a pulse wave estimation unit MSU.

[0020] The image acquisition unit GSY corresponds to the "image acquisition unit", the skin area detection unit HRK corresponds to the "skin area detection unit", the measurement area setting unit KRS corresponds to the "measurement area setting unit", the point identification unit TTO corresponds to the "point identification unit TTO", the luminance signal generation unit KSS corresponds to the "luminance signal generation unit", the pulse wave original signal generation unit MMS corresponds to the "pulse wave original signal generation unit", the pulse wave signal generation unit MSS corresponds to the "pulse wave signal generation unit", and the pulse wave estimation unit MSU corresponds to the "pulse wave estimation unit".

[0021] The image acquisition unit GSY acquires an image Im (also shown in FIG. 2 ) of at least one human subject HK under near-infrared light (for example, using a near-infrared camera). Since the image Im is captured under near-infrared light, it is a single-color image (for example, a monochrome image, or an image of only one color among multiple colors). Here, "near-infrared light" does not narrowly mean only near-infrared light, but also broadly mean, for example, a single color of visible light (for example, green light).

[0022] The skin region detection unit HRK detects the position of the skin of the subject HK in the image Im, for example, a skin region S (also shown in FIG. 2B ), which is the region where the skin of the face of the subject HK is located, by setting landmarks LM1, LM2, LM3, ... in the image Im using a conventionally known method. The skin region detection unit HRK generates skin region information SJ indicating the skin region S of the subject HK.

[0023] Specifically, the skin region information SJ indicates, for example, whether or not the skin region S of the subject HK is present in the image Im, and the position and size of the skin region S of the subject HK in the image Im.

[0024] The measurement region setting unit KRS sets a plurality of measurement regions R1, R2, R3, R4, ... (also shown in FIG. 2C ) in the skin region S of the subject HK, which are regions for measuring the brightness of the skin of the subject HK, to be used to estimate the pulse wave M of the subject HK. The measurement region setting unit KRS generates measurement region information RJ indicating the measurement regions R.

[0025] The plurality of measurement regions R1, R2, R3, R4, . . . are, for example, quadrilaterals (shown in FIG. 2C).

[0026] Specifically, the measurement region information RJ indicates, for example, the position and size of each measurement region R in the image Im, and indicates, for example, the coordinate positions of the four vertices of the measurement region R, which is a quadrangle.

[0027] As shown in Figure 3, the point identification unit TTO identifies a first point P1 and a second point P2 within the measurement region R (e.g., within the measurement region R56) and along the direction HO in which the pulse wave M of the subject HK propagates in the image Im.

[0028] Here, "identifying a first point P1 and a second point P2 along the direction HO in which the pulse wave M of the subject HK propagates in the image Im" should be interpreted broadly and includes at least the following: (1) (1A) identifying at least one of the first points P1, P3, P5, P7, and P9 and one of the second points P2, P4, P6, P8, and P10 along the direction HO in which the pulse wave M of the subject HK will propagate in the image Im, as shown in the lower left of FIG. 3 (identifying two or more points). (2) In an attempt to realize the specification of (1) above, a region where the first point P1, the second point P2, and two or more of points P3 to P10 are likely to exist (e.g., a region where the first point P1, the second point P2, point P5, and point P6 are likely to exist) is specified. Here, as suggested in the lower left of Figure 3, for example, an imaginary line (not shown; the same applies hereinafter) passing through the first point P1 and the second point P2, an imaginary line passing through points P3 and P4, an imaginary line passing through points P5 and P6, an imaginary line passing through points P7 and P8, and an imaginary line passing through points P9 and P10 are each along the above-mentioned direction HO, i.e., parallel to the direction HO.

[0029] As shown in Figures 3 and 5, the luminance signal generation unit KSS generates a first luminance signal KI1, a second luminance signal KI2, a third luminance signal KI3, and a fourth luminance signal KI4 using the luminance at the first point P1, the luminance at the second point P2, and weighting functions A and B.

[0030] Weighting function A and weighting function B correspond to the "first distribution" and the "second distribution".

[0031] More specifically, the luminance signal generation unit KSS convolves the luminance of a first point P1 in the measurement region R and the luminance of a second point P2 in the measurement region R with the weighting function A. As a result, the luminance signal generation unit KSS generates a first luminance signal KI1 indicating the luminance at the first point P1 after being weighted by the weighting function A, and a second luminance signal KI2 indicating the luminance at the second point P2 after being weighted by the weighting function A, as shown in FIG.

[0032] The luminance signal generation unit KSS also convolves the luminance of the first point P1 in the measurement region R and the luminance of the second point P2 in the measurement region R with weighting function B instead of the above-described weighting function A. As a result, the luminance signal generation unit KSS generates a third luminance signal KI3 indicating the luminance at the first point P1 after being weighted by weighting function B, and a fourth luminance signal KI4 indicating the luminance at the second point P2 after being weighted by weighting function B, as shown in FIG.

[0033] FIG. 7 shows the configurations of weighting functions A and B according to the first embodiment.

[0034] In FIG. 7 , the horizontal axis represents the position on the coordinate system, i.e., the distance from the center of gravity O, while the vertical axis represents the magnitude of the weighting. As shown in FIG. 7 , weighting function A and weighting function B have spatially different weightings. Specifically, weighting function A has a small weight spread, which is the degree of spread of the weighting (vertical axis) relative to the distance (horizontal axis), while weighting function B has a wide weight spread. More specifically, comparing weighting function A and weighting function B, weighting function A concentrates the weighting values ​​within a shorter distance range than weighting function B; in other words, weighting function B spreads the weighting values ​​within a longer distance range than weighting function A. In other words, weighting function A provides a weighting value that is substantially zero at a relatively short distance, while weighting function B provides a weighting value that is substantially zero at a relatively long distance.

[0035] For example, a two-dimensional Gaussian function can be used as weight function A and weight function B. Weight function A and weight function B using a two-dimensional Gaussian function can be expressed using, for example, a coordinate position, a weighting coefficient at the coordinate position, a coordinate position of the center of gravity, and the spread of the Gaussian function as parameters.

[0036] Returning to FIG. 1, the explanation will continue.

[0037] As shown in Figures 4 and 5, the pulse wave source signal generating unit MMS generates a first pulse wave source signal MM1 by combining the first luminance signal KI1 and the second luminance signal KI2. As shown in Figures 4 and 5, the pulse wave source signal generating unit MMS also generates a second pulse wave source signal MM2 by combining the third luminance signal KI3 and the fourth luminance signal KI4.

[0038] As shown in FIG. 5, the first pulse wave source signal MM1 has a characteristic in which the peaks of the pulse wave are emphasized, while the second pulse wave source signal MM2 has a characteristic in which the peaks of the pulse wave are not emphasized.

[0039] As shown in FIGS. 4 and 6, the pulse wave signal generator MSS generates a pulse wave signal MY indicating the pulse wave M of the subject HK by subtracting the second pulse wave source signal MM2 from the first pulse wave source signal MM1.

[0040] FIG. 8 shows a luminance signal of a comparative example.

[0041] In the pulse wave detection device described in Patent Document 1, which is a comparative example, a luminance signal (red component) KIR is subtracted from a luminance signal (green component) KIG, or a luminance signal (blue component) KIB is subtracted from a luminance signal (green component) KIG, as shown in Figure 8. The above pulse wave detection device improves the accuracy of estimating the pulse wave M of the subject HK by removing noise, which is an error in estimating the pulse wave M of the subject HK due to movement of the subject HK's head, through the subtraction.

[0042] The above-mentioned first pulse wave source signal MM1 corresponds to the luminance signal (green component) KIG of the comparative example, while the above-mentioned second pulse wave source signal MM2 corresponds to the luminance signal (red component) KIR or the luminance signal (blue component) KIB of the comparative example.

[0043] Returning to FIG. 1, the explanation will continue.

[0044] The pulse wave estimation unit MSU estimates the pulse wave M of the subject HK based on the pulse wave signal MY (shown in FIGS. 4 and 6). The pulse wave estimation unit MSU generates pulse wave information MJ indicating the pulse wave M.

[0045] The pulse wave information MJ indicates, for example, the pulse rate (the number of beats per minute). The pulse wave estimation unit MSU uses, as the pulse wave rate, for example, the peak frequency in the frequency power spectrum obtained by performing a Fourier transform on the pulse wave signal MY.

[0046] <Configuration of First Embodiment> FIG. 9 shows the hardware configuration of a pulse wave estimation device MSD of the first embodiment.

[0047] As shown in FIG. 9, the pulse wave estimation device MSD includes a processing circuit SH, and may further include an input circuit NY and an output circuit SY as required.

[0048] The processing circuit SH is dedicated hardware that realizes the functions of the image acquisition unit GSY to the pulse wave estimation unit MSU (shown in FIG. 1) of the pulse wave estimation device MSD.

[0049] The processing circuit SH may be, for example, a single circuit, a complex circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof.

[0050] The input circuit NY and the output circuit SY exchange inputs and outputs related to the operation of the processing circuit SH with, for example, the outside of the pulse wave estimation device MSD.

[0051] FIG. 10 shows a hardware configuration based on software realization of the pulse wave estimation device MSD of the first embodiment.

[0052] As shown in FIG. 10, the pulse wave estimation device MSD includes a processor PR and a memory circuit KI, and may further include an input circuit NY and an output circuit SY as required.

[0053] The processor PR is a CPU (also called a central processing unit, processing device, arithmetic unit, microprocessor, microcomputer, or DSP (Digital Signal Processing)) that executes programs. The processor PR realizes the functions of the image acquisition unit GSY to the pulse wave estimation unit MSU (shown in FIG. 1) of the pulse wave estimation device MSD.

[0054] The processor PR realizes the above-mentioned functions by software, firmware, or a combination of software and firmware. The software and firmware are written as programs and stored in the memory circuit KI.

[0055] The processor PR realizes the above-mentioned functions by reading and executing the above-mentioned programs from the memory circuit KI. The above-mentioned programs can also be said to cause the computer to execute the procedures and methods of the image acquisition unit GSY through the pulse wave estimation unit MSU.

[0056] Here, the memory circuit KI is, for example, a non-volatile or volatile semiconductor memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), etc., as well as a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, a DVD (Digital Versatile Disc), etc.

[0057] Of the functions of the image acquisition unit GSY to the pulse wave estimation unit MSU (shown in FIG. 1), some of the functions may be realized by a processing circuit SH (shown in FIG. 9), while other functions may be realized by a processor PR (shown in FIG. 10).

[0058] As described above, the functions of the image acquisition unit GSY to the pulse wave estimation unit MSU of the pulse wave estimation device MSD can be realized by hardware, software, firmware, or a combination of these.

[0059] The input circuit NY and the output circuit SY exchange inputs and outputs related to the operation of the processor PR with, for example, the outside of the pulse wave estimation device MSD.

[0060] <Operation of First Embodiment> The operation of the pulse wave estimation device MSD of the first embodiment will be described.

[0061] 11 is a flowchart showing the operation of the pulse wave estimation device MSD of embodiment 1. The operation of the pulse wave estimation device MSD of embodiment 1 will be described with reference to the flowchart of FIG.

[0062] Step ST11: The image acquisition unit GSY (shown in FIG. 1) acquires an image Im (shown in FIGS. 1, 2A, etc.) of the subject HK (shown in FIG. 1).

[0063] Step ST12: The skin region detection unit HRK (shown in FIG. 1) detects the skin region S (shown in FIGS. 1, 2B, etc.) of the subject HK in the image Im of the subject HK. The skin region detection unit HRK generates skin region information SJ indicating the skin region S of the subject HK.

[0064] Step ST13: The measurement region setting unit KRS (shown in FIG. 1) sets a plurality of measurement regions R (shown in FIGS. 1, 2C, etc.) in the skin region S of the subject HK.

[0065] Step ST14: The point identification unit TTO (shown in Figure 1) identifies a first point P1 and a second point P2 (shown in Figures 3, 5, etc.) within the measurement region R, for example, within the measurement region R56 (shown in Figure 3), and along the direction HO (shown in Figure 3) in which the pulse wave M of the subject HK propagates.

[0066] Step ST15: The luminance signal generation unit KSS (shown in Figure 1) generates the first luminance signal KI1 to the fourth luminance signal KI4 (shown in Figures 3, 4, etc.) using the luminance at the first point P1 in the measurement area R and the luminance at the second point P2 in the measurement area R (shown in Figure 3), as well as weighting function A and weighting function B (shown in Figure 3, etc.).

[0067] Step ST16: The pulse wave signal generator MSS (shown in FIG. 1) generates a first pulse wave source signal MM1 (shown in FIGS. 4 and 5) by combining the first luminance signal KI1 and the second luminance signal KI2. The pulse wave source signal generator MMS also generates a second pulse wave source signal MM2 (shown in FIGS. 4 and 5) by combining the third luminance signal KI3 and the fourth luminance signal KI4.

[0068] Step ST17: The pulse wave signal generating unit MSS (shown in FIG. 1) generates a pulse wave signal MY (shown in FIGS. 4 and 6) indicating the pulse wave M of the subject HK by performing subtraction between the first pulse wave source signal MM1 and the second pulse wave source signal MM2, more specifically, by subtracting the second pulse wave source signal MM2 from the first pulse wave source signal MM1.

[0069] Step ST18: The pulse wave estimation unit MSU (shown in FIG. 1) estimates the pulse wave M (shown in FIG. 1) of the subject HK based on the pulse wave signal MY. The pulse wave estimation unit MSU generates pulse wave information MJ (shown in FIG. 1) indicating the pulse wave M.

[0070] Effect of First Embodiment As described above, in the pulse wave estimation device MSD of the first embodiment, the luminance signal generation unit KSS generates a first luminance signal KI1 to a fourth luminance signal KI4 based on the luminance at a first point P1 within the measurement region R, the luminance at a second point P2 within the measurement region R, weighting function A, and weighting function B. The pulse wave source signal generation unit MMS generates a first pulse wave source signal MM1 and a second pulse wave source signal MM2 using the first luminance signals KI1 to KI4. The pulse wave signal generation unit MSS generates a pulse wave signal MY using the first pulse wave source signal MM1 and the second pulse wave source signal MM2. The pulse wave estimation unit MSU estimates the pulse wave M based on the pulse wave signal MY. This improves the accuracy of estimating the pulse wave M of the subject HK, even when using an image Im of the subject HK captured under near-infrared light.

[0071] <Modification of First Embodiment> A modification of the first embodiment will be described.

[0072] FIG. 12 shows the configuration of a pulse wave estimation device MSD according to a modification of the first embodiment.

[0073] The first pulse wave source signal MM1 and the second pulse wave source signal MM2 described in the first embodiment contain various noise components NS (not shown), for example, noise components NS caused by defects in some of the elements that make up the image acquisition unit GSY. In order to remove the noise components NS from the first pulse wave source signal MM1 and the second pulse wave source signal MM2, it is desirable to subject the first pulse wave source signal MM1 and the second pulse wave source signal MM2 to filtering processing FS (not shown).

[0074] The pulse wave estimation device MSD of the modified example of the first embodiment further includes a noise removal unit NJO to perform the above-mentioned filtering process FS.

[0075] As shown in FIG. 12, the noise elimination unit NJO is disposed between the pulse wave original signal generation unit MMS and the pulse wave signal generation unit MSS.

[0076] The noise removal unit NJO performs the above-mentioned filtering process FS on the first pulse wave source signal MM1 and the second pulse wave source signal MM2, for example, by using a low-pass filter, a high-pass filter, or a band-pass filter.

[0077] In the following description, it is assumed that the noise removal unit NJO performs band-pass filtering as the filtering process FS.

[0078] The bandpass filter is, for example, a Butterworth filter, and it is desirable that the lower cutoff frequency of the bandpass filter is, for example, 0.5 Hz and the upper cutoff frequency is, for example, 5.0 Hz.

[0079] The noise removal unit NJO removes the noise components NS contained in the first pulse wave source signal MM1 and the second pulse wave source signal MM2 from the first pulse wave source signal MM1 and the second pulse wave source signal MM2, and outputs the first pulse wave source signal MM1 from which the noise components NS have been removed and the second pulse wave source signal MM2 from which the noise components NS have been removed to the pulse wave signal generation unit MSS.

[0080] In the pulse wave estimation device MSD of this variation of embodiment 1, the noise removal unit NJO applies filter processing FS to the first pulse wave source signal MM1 and the second pulse wave source signal MM2. This removes noise components NS contained in the first pulse wave source signal MM1 and the second pulse wave source signal MM2. As a result, the noise components NS are removed from the pulse wave signal MY generated by the pulse wave source signal generation unit MMS, and the accuracy of the estimation of the pulse wave M of the subject HK by the pulse wave estimation unit MSU can be improved compared to embodiment 1.

[0081] Second Embodiment A pulse wave estimation device MSD according to a second embodiment will be described.

[0082] Unlike the pulse wave estimation device MSD of embodiment 1 (shown in FIG. 1) which estimates the pulse wave M of subject HK based on a single measurement region R, for example, measurement region R56 (shown in FIG. 6), the pulse wave estimation device MSD of embodiment 2 estimates the pulse wave M of subject HK based on two mutually different measurement regions R, for example, measurement regions R1 and R2 (shown in FIG. 2C).

[0083] <Functions of Second Embodiment> FIG. 13 is a functional block diagram of a pulse wave estimation device MSD of the second embodiment.

[0084] As is clear from a comparison of Fig. 13 with Fig. 1 (the functional block diagram of the pulse wave estimation device MSD of embodiment 1), the pulse wave estimation device MSD of embodiment 2 basically has the same functions as the pulse wave estimation device MSD of embodiment 1. However, the pulse wave estimation device MSD of embodiment 2 differs from the pulse wave estimation device MSD of embodiment 1 in that it further includes an averaging unit HKK.

[0085] The averaging unit HKK corresponds to the "averaging unit."

[0086] The averaging unit HKK averages the pulse wave signal MY(R1) for the measurement region R1 and the pulse wave signal MY(R2) for the measurement region R2.

[0087] <Configuration of Embodiment 2> The hardware configuration of the pulse wave estimation device MSD of embodiment 2 and the hardware configuration based on software realization are similar to the hardware configuration of the pulse wave estimation device MSD of embodiment 1 (shown in FIG. 9) and the hardware configuration based on software realization (shown in FIG. 10).

[0088] <Operation of Second Embodiment> FIG. 14 is a flowchart showing the operation of the pulse wave estimation device MSD of the second embodiment.

[0089] FIG. 15 shows the operation of the pulse wave estimation device MSD of embodiment 2 up to the generation of the first luminance signal KI1(R1) to the fourth luminance signal KI4(R1) and the first luminance signal KI1(R1) to the fourth luminance signal KI4(R2).

[0090] FIG. 16 shows the operation up to generation of the average pulse wave signal HMY of the pulse wave estimation device MSD of the second embodiment.

[0091] The operation of the pulse wave estimation device MSD of the second embodiment will be described with reference to FIGS.

[0092] Step ST21: The image acquisition unit GSY (shown in FIG. 13) acquires an image Im (shown in FIG. 15) of the subject HK (shown in FIG. 13) in the same manner as in step ST11 of the first embodiment.

[0093] Step ST22: The skin region detection unit HRK (shown in FIG. 13) detects the skin region S (shown in FIG. 15) of the subject HK in the image Im of the subject HK, similar to step ST12 in embodiment 1. The skin region detection unit HRK generates skin region information SJ indicating the skin region S of the subject HK.

[0094] Step ST23: The measurement region setting unit KRS (shown in FIG. 13) sets a plurality of measurement regions R (shown in FIG. 15) in the skin region S of the subject HK, similarly to step ST13 in the first embodiment.

[0095] Step ST24: The point identification unit TTO (shown in FIG. 13) differs from step ST14 in embodiment 1, and as shown in FIG. 15, identifies a first point P1 and a second point P2 in the measurement region R1 and along the direction HO in which the pulse wave M of the subject HK propagates (for example, as shown in FIG. 3 in embodiment 1), and on the other hand, identifies a first point P1 and a second point P2 in a measurement region R2 different from the measurement region R1 and along the direction HO in which the pulse wave M of the subject HK propagates.

[0096] Step ST25: The luminance signal generation unit KSS (shown in FIG. 13) differs from step ST15 in embodiment 1, and as shown in FIG. 15, generates a first luminance signal KI1(R1) to a fourth luminance signal KI4(R1) using the luminance at a first point P1 in the measurement region R1, the luminance at a second point P2 in the measurement region R1, and weighting functions A and B, and generates a first luminance signal KI1(R2) to a fourth luminance signal KI4(R2) using the luminance at a first point P1 in the measurement region R2, the luminance at a second point P2 in the measurement region R2, and weighting functions A and B.

[0097] Step ST26: The pulse wave signal generating unit MSS (shown in FIG. 13) differs from step ST16 in embodiment 1 in that, as shown in FIG. 16, for measurement region R1, it generates a first pulse wave source signal MM1(R1) by combining the first luminance signal KI1(R1) and the second luminance signal KI2(R1), and generates a second pulse wave source signal MM2(R1) by combining the third luminance signal KI3(R1) and the fourth luminance signal KI4(R1). On the other hand, the pulse wave signal generating unit MSS generates a first pulse wave original signal MM1 (R2) by combining the first luminance signal KI1 (R2) and the second luminance signal KI2 (R2) for the measurement region R2, and generates a second pulse wave original signal MM2 (R2) by combining the third luminance signal KI3 (R2) and the fourth luminance signal KI4 (R2).

[0098] Step ST27: The pulse wave signal generation unit MSS (shown in FIG. 13) differs from step ST17 in embodiment 1 in that, for measurement region R1, it subtracts the second pulse wave source signal MM2(R1) from the first pulse wave source signal MM1(R1), as shown in FIG. 16, to generate a pulse wave signal MY(R1) that may indicate the pulse wave M of subject HK. On the other hand, for measurement region R2, the pulse wave signal generation unit MSS subtracts the second pulse wave source signal MM2(R2) from the first pulse wave source signal MM1(R2), to generate a pulse wave signal MY(R2) that may indicate the pulse wave M of subject HK.

[0099] Step ST28: The averaging unit HKK (shown in FIG. 13) averages the pulse wave signal MY(R1) for measurement region R1 and the pulse wave signal MY(R2) for measurement region R2, as shown in FIG. 16. As a result, the averaging unit HKK generates a single averaged signal, the averaged pulse wave signal HMY, as shown in FIG.

[0100] Step ST29: The pulse wave estimation unit MSU (shown in FIG. 13) estimates the pulse wave M of the subject HK based on the average pulse wave signal HMY. The pulse wave estimation unit MSU generates pulse wave information MJ indicating the pulse wave M.

[0101] Effect of Second Embodiment As described above, in the pulse wave estimation device MSD of the second embodiment, the image acquisition unit GSY to the pulse wave signal generation unit MSS cooperate to generate pulse wave signals MY(R1) and MY(R2) for two mutually different measurement regions R, i.e., measurement regions R1 and R2, and the averaging unit HKK averages the pulse wave signals MY(R1) and MY(R2) to generate a single averaged signal, an average pulse wave signal HMY. The pulse wave estimation unit MSU estimates the pulse wave M of the subject HK based on the average pulse wave signal HMY, instead of the pulse wave signal MY of the first embodiment. As a result, even if the measurement area R1 is partially blocked due to the presence of an obstacle between the subject HK and the pulse wave estimation device MSD, i.e., even if a portion of the measurement area R1 is not captured in the image Im of the subject HK, by using the measurement area R2 in combination, it is possible to estimate the pulse wave M of the subject HK with the same accuracy as in embodiment 1.

[0102] Third Embodiment A pulse wave estimation device MSD according to a third embodiment will be described.

[0103] Unlike the pulse wave estimation device MSD of embodiment 1, which subtracts the second pulse wave source signal MM2 from the first pulse wave source signal MM1 to estimate the pulse wave M of the subject HK, the pulse wave estimation device MSD of embodiment 3 subtracts the result of multiplying the second pulse wave source signal MM2 by a coefficient (multiplication coefficient c, described below) from the first pulse wave source signal MM1.

[0104] <Functions of Embodiment 3> The pulse wave estimation device MSD of Embodiment 3 basically has the same functions (shown in FIG. 1 ) as the pulse wave estimation device MSD of Embodiment 1. However, the pulse wave estimation device MSD of Embodiment 3 differs from the pulse wave estimation device MSD of Embodiment 1 in that the pulse wave source signal generation unit MMS of Embodiment 3 has functions that differ from those of the pulse wave source signal generation unit MMS of Embodiment 1.

[0105] FIG. 17 is a functional block diagram of the pulse wave source signal generating unit MMS according to the third embodiment.

[0106] As shown in FIG. 17, the pulse wave source signal generating unit MMS of the third embodiment has a coefficient calculation function KE, a multiplication function JO, and a subtraction function GE.

[0107] The pulse wave source signal generating unit MMS of the third embodiment also handles first pulse wave source signal information W, second pulse wave source signal information M, and coefficient information C.

[0108] The relationship between each piece of information and each signal is as follows: (A) First pulse wave source signal information W indicates the first pulse wave source signal MM1. (B) Second pulse wave source signal information M indicates the second pulse wave source signal MM2. (C) Coefficient information C indicates the multiplication coefficient c.

[0109] The multiplication coefficient c corresponds to the "coefficient."

[0110] The pulse wave signal generating unit MSS generates a pulse wave signal MY based on the first pulse wave source signal information W and the second pulse wave source signal information M, in other words, generates a pulse wave signal MY based on the first pulse wave source signal MM1 and the second pulse wave source signal MM2.

[0111] More specifically, the pulse wave signal generating unit MSS performs the following functions: (1) As a coefficient calculation function KE, it calculates coefficient information C indicating a multiplication coefficient c, which is a coefficient for multiplying the second pulse wave original signal information M, based on the first pulse wave original signal information W and the second pulse wave original signal information M; (2) As a multiplication function JO, it multiplies the second pulse wave original signal MM2 by the multiplication coefficient c in (1) above; and (3) It subtracts the result of the multiplication in (2) above from the first pulse wave original signal MM1.

[0112] The pulse wave signal generator MSS performs steps (1) to (3) to generate the pulse wave signal MY.

[0113] The coefficient information C described above indicates the amount, or magnitude, of the second pulse wave original signal information M to be subtracted.

[0114] More specifically, the value of the multiplication coefficient c is adjusted so that the components necessary for estimating the pulse wave M of the subject HK remain in the pulse wave signal MY obtained by subtracting the result obtained by multiplying the second pulse wave original signal MM2 by the multiplication coefficient c from the first pulse wave original signal MM1.

[0115] When the value of the multiplication coefficient c is large, the result of multiplying the second pulse wave original signal MM2 by the multiplication coefficient c is large; conversely, when the multiplication coefficient c is small, the result of multiplying the second pulse wave original signal MM2 by the multiplication coefficient c is small.

[0116] The pulse wave signal generating unit MSS calculates coefficient information C so as to minimize the difference between the first pulse wave source signal information W and the result obtained by multiplying the second pulse wave source signal information M by coefficient information C. In this way, the pulse wave signal generating unit MSS can adjust the amount of second pulse wave source signal information M included in the first pulse wave source signal information W using coefficient information C.

[0117] <Configuration of Embodiment 3> The hardware configuration of the pulse wave estimation device MSD of embodiment 3 and the hardware configuration based on software realization are similar to the hardware configuration of the pulse wave estimation device MSD of embodiment 1 (shown in FIG. 9) and the hardware configuration based on software realization (shown in FIG. 10).

[0118] <Operation of Embodiment 3> The operation of the pulse wave estimation device MSD of embodiment 3 is basically the same as the operation of the pulse wave estimation device MSD of embodiment 1 (shown in FIG. 11 ). However, the pulse wave estimation device MSD of embodiment 3 differs from the pulse wave estimation device MSD of embodiment 1 in the operation of the pulse wave signal generation unit MSS. The following description will focus on the operation of the pulse wave signal generation unit MSS of embodiment 3.

[0119] 18 is a flowchart showing the operation of the pulse wave estimation device MSD of embodiment 3. The operation of the pulse wave estimation device MSD of embodiment 3 will be described with reference to the flowchart of FIG.

[0120] Step ST17-1: The pulse wave signal generator MSS (shown in FIG. 17) receives the first pulse wave source signal MM1 and the second pulse wave source signal MM2 (shown in FIGS. 17, 1, and 4) from the preceding pulse wave source signal generator MMS (shown in FIG. 1). The pulse wave signal generator MSS, as a coefficient calculation function KE, calculates a multiplication coefficient c based on the first pulse wave source signal information W and the second pulse wave source signal information M.

[0121] Step ST17-2: The pulse wave signal generating unit MSS uses the multiplication function JO to multiply the second pulse wave original signal MM2 by the multiplication coefficient c calculated in step ST17-1 above.

[0122] Step ST17-3: The pulse wave signal generation unit MSS, using its subtraction function GE, subtracts the result of the multiplication performed in step ST17-2 from the first pulse wave original signal MM1. The pulse wave signal generation unit MSS calculates a pulse wave signal MY through this subtraction. The pulse wave signal generation unit MSS outputs the calculated pulse wave signal MY to the pulse wave estimation unit MSU in the subsequent stage.

[0123] Step ST17-3 and subsequent steps: Pulse wave estimation unit MSU estimates pulse wave M of subject HK based on pulse wave signal MY received from pulse wave signal generation unit MSS and generated through steps ST17-1 to ST17-3 described above.

[0124] Effect of Embodiment 3 As described above, in the pulse wave estimation device MSD of Embodiment 3, the pulse wave signal generator MSS calculates the pulse wave signal MY by subtracting the result obtained by multiplying the second pulse wave source signal MM2 by the multiplication coefficient c from the first pulse wave source signal MM1. This prevents signal components useful for estimating the pulse wave M of the subject HK, which are contained in the first pulse wave source signal MM1 and the second pulse wave source signal MM2, from being unnecessarily attenuated. As a result, the pulse wave estimation device MSD of Embodiment 3 can estimate the pulse wave M of the subject HK with higher accuracy than the pulse wave estimation device MSD of Embodiment 1, which simply subtracts the second pulse wave source signal MM2 from the first pulse wave source signal MM1.

[0125] Fourth Embodiment A pulse wave estimation device MSD according to a fourth embodiment will be described.

[0126] Unlike the pulse wave estimation device MSD of embodiment 1 (shown in FIG. 1 ), which uses weighting functions such as weighting function A and weighting function B (shown in FIG. 3 , for example) to generate a fourth luminance signal KI4 from a first luminance signal KI1, the pulse wave estimation device MSD of embodiment 4 does not use the weighting functions.

[0127] <Functions of Embodiment 4> The pulse wave estimation device MSD of Embodiment 4 basically has the same functions as the pulse wave estimation device MSD of Embodiment 1. On the other hand, the pulse wave estimation device MSD of Embodiment 4 differs from the pulse wave estimation device MSD of Embodiment 1 in that the measurement region setting unit KRS, point identification unit TTO, and luminance signal generation unit KSS of Embodiment 4 differ in function from the measurement region setting unit KRS, point identification unit TTO, and luminance signal generation unit KSS of Embodiment 1.

[0128] FIG. 19 is a functional block diagram of a measurement region setting unit KRS, a point specifying unit TTO, and a luminance signal generating unit KSS according to the fourth embodiment.

[0129] FIG. 20 shows the measurement region R, the first point P1 to the fourth point P4, and the first luminance signal KI1 to the fourth luminance signal KI4 of the fourth embodiment.

[0130] As shown in Figures 19 and 20, the measurement area setting unit KRS has the function of setting a first measurement area (e.g., measurement area R34) and the function of setting a second measurement area (e.g., measurement area R56).

[0131] Here, as shown in FIG. 20 , the measurement region R34 and the measurement region R56 share a common center of gravity O, which is the reference position on the coordinate system of the measurement region R34, and a common center of gravity O, which is the reference position on the coordinate system of the measurement region R56. The measurement region R34 and the measurement region R56 have a relationship in which the measurement region R56 is larger than the measurement region R34. Alternatively, the measurement region R56 may have a relationship in which it is smaller than the measurement region R34. The measurement region R34 and the measurement region R56 share a corresponding relationship between a position on the coordinate system and a magnitude of weighting specified by the position on the coordinate system, and share, for example, one weight function (e.g., weight function A or weight function B (shown in FIG. 7 )).

[0132] 20, the point identification unit TTO identifies a first point P1 and a second point P2 within a first measurement region, for example, within measurement region R34, along the direction HO in which the pulse wave M of the subject HK propagates. The point identification unit TTO also sets a third point P3 and a fourth point P4 within a second measurement region, for example, within measurement region R56, along the direction HO in which the pulse wave M of the subject HK propagates.

[0133] 20, since measurement region R56 is wider than measurement region R34, the distance between the third point P3 and the fourth point P4 is longer than the distance between the first point P1 and the second point P2. In contrast to the above, when measurement region R56 is narrower than measurement region R34, the distance between the third point P3 and the fourth point P4 is shorter than the distance between the first point P1 and the second point P2.

[0134] 20, the luminance signal generating unit KSS outputs a first luminance signal KI1 indicating the luminance of a first point P1 in a first measurement region, for example, measurement region R34, and a second luminance signal KI2 indicating the luminance of a second point P2 in measurement region R34. The pulse wave signal generating unit MSS also outputs a third luminance signal KI3 indicating the luminance of a second measurement region, for example, a third point P3 in measurement region R56, and a fourth luminance signal KI4 indicating the luminance of a fourth point P4 in measurement region R56.

[0135] <Configuration of Embodiment 4> The hardware configuration of the pulse wave estimation device MSD of embodiment 4 and the hardware configuration based on software realization are similar to the hardware configuration of the pulse wave estimation device MSD of embodiment 1 (shown in FIG. 9) and the hardware configuration based on software realization (shown in FIG. 10).

[0136] <Operation of Embodiment 4> Fig. 21 is a flowchart showing the operation of the pulse wave estimation device MSD of embodiment 4. The operation of the pulse wave estimation device MSD of embodiment 4 will be described with reference to the flowchart of Fig. 21 .

[0137] Before step ST43: The pulse wave estimation device MSD of the fourth embodiment performs steps ST11 and ST12 (shown in FIG. 11).

[0138] Step ST43: The measurement area setting unit KRS (shown in FIG. 19) sets a first measurement area, i.e., measurement area R34, in the skin area S, as shown in FIG. 20. The measurement area setting unit KRS also sets a second measurement area, i.e., measurement area R56, in the skin area S, as shown in FIG. 20.

[0139] Step ST44: The point identification unit TTO (shown in Fig. 19) sets the above-mentioned first point P1 and second point P2 in the first measurement region, i.e., measurement region R34, as shown in Fig. 20. The point identification unit TTO also sets the given first point P1 and second point P2 in the second measurement region, i.e., measurement region R56, as shown in Fig. 20.

[0140] Step ST45: The luminance signal generation unit KSS (shown in FIG. 19) outputs a first luminance signal KI1 indicating the luminance of a first point P in the first measurement region, i.e., measurement region R34, and a second luminance signal KI2 indicating the luminance of a second point P in the measurement region R34, as shown in FIG. 20. The luminance signal generation unit KSS also outputs a third luminance signal KI3 indicating the luminance of a third point P3 in the second measurement region, i.e., measurement region R56, and a fourth luminance signal KI4 indicating the luminance of a fourth point P4 in the measurement region R56, as shown in FIG.

[0141] After step ST45: The pulse wave estimation device MSD of the fourth embodiment performs steps ST16 to ST18 (shown in FIG. 11).

[0142] Effect of Embodiment 4 As described above, the pulse wave estimation device MSD of Embodiment 4 generates the first luminance signal KI1 to the fourth luminance signal KI4 in the same manner as the pulse wave estimation device MSD of Embodiment 1, without using the weighting functions A and B of the pulse wave estimation device MSD of Embodiment 1. As a result, the pulse wave estimation device MSD of Embodiment 4 can improve the accuracy of estimating the pulse wave M of the subject HK while using the image Im of the subject HK captured under near-infrared light, in the same manner as the pulse wave estimation device MSD of Embodiment 1.

[0143] Fifth Embodiment <Fifth Embodiment> A state estimation system JSS according to a fifth embodiment will be described. <Functions of the Fifth Embodiment> Fig. 22 is a functional block diagram of the state estimation system JSS according to the fifth embodiment.

[0144] As shown in FIG. 22, the state estimation system JSS of the fifth embodiment includes a pulse wave estimation device MSD and a state estimation device JSD.

[0145] The pulse wave estimation device MSD is one of the pulse wave estimation devices MSD of the above-described embodiment 1 to embodiment 4. The pulse wave estimation device MSD outputs a pulse wave M estimated for a person HT (e.g., a vehicle driver), more precisely, pulse wave information MJ of the person HT (e.g., the pulse rate of the person HT), in the same way that the pulse wave estimation devices MSD of embodiments 1 to 4 output a pulse wave M estimated for a subject HK (e.g., shown in FIG. 1 ).

[0146] The state estimation device JSD estimates the state of the person HT based on the pulse wave information MJ of the person HT, and more specifically, estimates, for example, the level of wakefulness KT of the driver of the vehicle. The level of wakefulness KT is expressed, for example, in two stages (1: drowsy state, 2: wakeful state).

[0147] A vehicle driver is generally in an alert state immediately after starting to drive the vehicle. Therefore, the state estimation device JSD estimates which of the two states the vehicle driver is in, for example, based on the pulse rate from the time the vehicle driver starts driving until 10 minutes have elapsed. More specifically, when it is determined that the pulse rate of the vehicle driver is significantly lower than the standard, the state estimation device JSD estimates that the level of alertness KT of the vehicle driver is "1: drowsy state."

[0148] The state estimation device JSD may emit, for example, a warning sound along with the degree of wakefulness KT being the above-mentioned "1: sleepy state."

[0149] The person HT may be a driver of a vehicle, or may be, for example, a person in a home (e.g., a person sitting in a living room) or a person working in an office (e.g., a person traveling in an elevator).

[0150] <Effects of the Fifth Embodiment> As described above, the state estimation system JSS of the fifth embodiment includes one of the pulse wave estimation devices MSD of the first to fourth embodiments, and is thereby able to estimate the state of the person HT.

[0151] The above-described embodiments may be combined with each other without departing from the spirit of the present disclosure, and components in each embodiment may be deleted or modified, or other components may be added, as appropriate.

[0152] The pulse wave estimation device and the like according to the present disclosure can be used to improve the accuracy of estimating a subject's pulse wave even when using an image captured under near-infrared light.

[0153] A weighting function, B weighting function, GSY image acquisition unit, HK subject, HRK skin area detection unit, Im image, KI1 first luminance signal, KI2 second luminance signal, KI3 third luminance signal, KI4 fourth luminance signal, KRS measurement area setting unit, KSS luminance signal generation unit, M pulse wave, MM1 first pulse wave source signal, MM2 second pulse wave source signal, MMS pulse wave source signal generation unit, MSD pulse wave estimation device, MSS pulse wave signal generation unit, MSU pulse wave estimation unit, MY pulse wave signal, P1 first point, P2 second point, R measurement area, S skin area, TTO point identification unit.

Claims

1. an image capture unit that captures an image of a person under near-infrared light; a skin area detection unit that detects a skin area, which is a position of the person's skin, from the image; a measurement area setting unit that sets a plurality of measurement areas in the skin area for measuring the luminance of the human skin; a point specifying unit that specifies a first point and a second point within one of the plurality of measurement areas and along a direction in which a pulse wave of the person propagates in the image; a first distribution and a second distribution having mutually different correspondences between positions on a coordinate system and the magnitude of weighting specified by the positions on the coordinate system, the first distribution being superimposed on a luminance of the first point and a luminance of the second point to generate a first luminance signal indicating the luminance at the first point after being weighted by the first distribution, and a second luminance signal indicating the luminance at the second point after being weighted by the first distribution; and a luminance signal generating unit that generates a third luminance signal indicating the luminance at the first point after being weighted by the second distribution and a fourth luminance signal indicating the luminance at the second point after being weighted by the second distribution by superimposing the luminance at the first point and the luminance at the second point on the second distribution; generating a first pulse wave source signal by combining the first luminance signal and the second luminance signal; and a pulse wave origin signal generating unit that generates a second pulse wave origin signal by combining the third luminance signal and the fourth luminance signal; a pulse wave signal generating unit that generates a pulse wave signal indicative of the person's pulse wave by subtracting one of the first pulse wave source signal and the second pulse wave source signal from the other; a pulse wave estimation unit that estimates a pulse wave of the person based on the pulse wave signal; A pulse wave estimating device comprising:

2. an averaging unit that averages a plurality of pulse wave signals generated in the plurality of measurement regions to generate a single averaged pulse wave signal, The pulse wave estimation unit estimates a pulse wave of the person based on the averaged pulse wave signal. The pulse wave estimation device according to claim 1 .

3. The pulse wave signal generating unit generates the pulse wave signal. calculating a coefficient for adjusting the second pulse wave source signal so that a signal component for estimating a pulse wave of the person remains in the pulse wave signal obtained by subtracting the second pulse wave source signal from the first pulse wave source signal; multiplying the second pulse wave source signal by the coefficient; and and subtracting a result of the multiplication from the first pulse wave source signal. The pulse wave estimation device according to claim 1 .

4. the measurement area setting unit sets, as the multiple measurement areas, the one measurement area and another measurement area that has a common reference position on a coordinate system with the one measurement area and is wider or narrower than the one measurement area, and the one measurement area and the other measurement area share a corresponding relationship between a position on the coordinate system and a magnitude of a weighting specified by the position on the coordinate system; the point identifying unit identifies a third point and a fourth point in the other measurement area along a direction in which a pulse wave of the person propagates, the third point and the fourth point being longer or shorter than a distance between the third point and the fourth point; the luminance signal generating unit outputs a first luminance signal indicating the luminance of the first point in the one measurement area, a second luminance signal indicating the luminance of the second point in the one measurement area, a third luminance signal indicating the luminance of the third point in the other measurement area, and a fourth luminance signal indicating the luminance of the fourth point in the other measurement area, instead of generating the first luminance signal, the second luminance signal, the third luminance signal, and the fourth luminance signal using the first distribution and the second distribution. The pulse wave estimation device according to claim 1 .

5. A pulse wave estimation device according to any one of claims 1 to 4, a state estimation device that estimates a state of the person based on the pulse wave output from the pulse wave estimation device; A state estimation system comprising:

6. The image acquisition unit acquires an image of a person under near-infrared light, a skin area detection unit detects a skin area, which is a position of the person's skin, from the image; a measurement area setting unit setting a plurality of measurement areas in the skin area for measuring a luminance of the person's skin; the point identifying unit identifies a first point and a second point within one of the plurality of measurement areas and along a direction in which a pulse wave of the person propagates in the image; a luminance signal generating unit generates a first luminance signal indicating the luminance at the first point after being weighted by the first distribution and a second luminance signal indicating the luminance at the second point after being weighted by the first distribution by superimposing a luminance at the first point and a luminance at the second point on the first distribution, the first distribution being a first distribution and a second distribution having mutually different correspondence relationships between positions on a coordinate system and magnitudes of weighting specified by the positions on the coordinate system, and generating a third luminance signal indicating the luminance at the first point after being weighted by the second distribution and a fourth luminance signal indicating the luminance at the second point after being weighted by the second distribution by superimposing the luminance at the first point and the luminance at the second point on the second distribution; the pulse wave source signal generating unit generates a first pulse wave source signal by combining the first luminance signal and the second luminance signal; and generating a second pulse wave source signal by combining the third luminance signal and the fourth luminance signal; a pulse wave signal generating unit generating a pulse wave signal indicative of a pulse wave of the person by subtracting one of the first pulse wave source signal and the second pulse wave source signal from the other; The pulse wave estimation unit estimates a pulse wave of the person based on the pulse wave signal. Pulse wave estimation method.

7. the averaging unit averages the pulse wave signals generated in the plurality of measurement regions to generate a single averaged pulse wave signal; The pulse wave estimation unit estimates a pulse wave of the person based on the averaged pulse wave signal. The pulse wave estimating method according to claim 6.

8. The pulse wave signal generating unit generates the pulse wave signal. calculating a coefficient for adjusting the second pulse wave source signal so that a signal component for estimating a pulse wave of the person remains in the pulse wave signal obtained by subtracting the second pulse wave source signal from the first pulse wave source signal; multiplying the second pulse wave source signal by the coefficient; and and subtracting a result of the multiplication from the first pulse wave source signal. The pulse wave estimating method according to claim 6.

9. the measurement area setting unit sets, as the multiple measurement areas, the one measurement area and another measurement area that has a common reference position on a coordinate system with the one measurement area and is wider or narrower than the one measurement area, and the one measurement area and the other measurement area share a corresponding relationship between a position on the coordinate system and a magnitude of a weighting specified by the position on the coordinate system; the point identifying unit identifies a third point and a fourth point in the other measurement area along a direction in which a pulse wave of the person propagates, the third point and the fourth point being longer or shorter than a distance between the third point and the fourth point; the luminance signal generating unit outputs a first luminance signal indicating the luminance of the first point in the one measurement area, a second luminance signal indicating the luminance of the second point in the one measurement area, a third luminance signal indicating the luminance of the third point in the other measurement area, and a fourth luminance signal indicating the luminance of the fourth point in the other measurement area, instead of generating the first luminance signal, the second luminance signal, the third luminance signal, and the fourth luminance signal using the first distribution and the second distribution. The pulse wave estimating method according to claim 6.

10. A pulse wave estimating method according to any one of claims 6 to 9, a state estimation method for estimating a state of the person based on a pulse wave output from the pulse wave estimation method; A state estimation method comprising: