Pulse wave assessment device and pulse wave assessment method

Abstract

The invention comprises: an image acquisition unit (11) for acquiring an image depicting a person, a skin area detection unit (12) for detecting a skin area of ​​the person based on the image, a measurement range setting unit (13) for setting, in an area corresponding to the skin area on the image, several measurement ranges in order to extract a pulse wave origin signal representing luminance changes in time series in a first period, a pulse wave origin signal extraction unit (14) for extracting the pulse wave origin signal for each of the measurement ranges based on the luminance changes in the first period in the respective measurement ranges, and a segment generation unit (15) for generatingFor each of the measurement ranges, based on the pulse wave origin signal extracted from the corresponding measurement range, several pulse wave origin signal segments are generated according to segment generation conditions, wherein the signals are those in which a pulse wave origin signal for a second period has been partially extracted from the pulse wave origin signal in the first period, and a pulse wave estimation unit (16) for estimating the pulse wave of the person based on the several pulse wave origin signal segments generated for each of the measurement ranges.

Description

[Technical field]

[0001] The present disclosure relates to a pulse wave estimation apparatus and a pulse wave estimation method. [State of the art]

[0002] Conventionally, a technique for estimating a person's pulse wave is known in which subtle luminance changes on the surface of a person's skin are extracted based on luminance signals in multiple areas (hereinafter, "measurement areas") set within a range including the person's skin in an image image imaged by an imaging device (hereinafter, "skin area"), and pulse wave components representing these luminance changes are separated from signals using a signal separation technique such as independent component analysis or principal component analysis (for example, Patent Document 1). This technique utilizes the principle that the amount of light absorption in a person's bloodstream changes depending on the pulse rate. [Prior art documents][Patent documents]

[0003] [Patent Document 1] JP 2017-93760 A [Overview of the invention][Problem to be solved by the invention]

[0004] Human blood leaves the heart via the arteries, circulates throughout the body, and returns to the heart via the veins. Due to this characteristic of human blood flow, a large difference in distance from the heart between multiple measurement areas set for pulse wave estimation will result in a phase difference in the luminance signals extracted from these multiple measurement areas. If a phase difference occurs in the extracted luminance signals between multiple measurement areas, pulse wave components cannot be separated from the signals representing luminance changes, which may reduce the accuracy of pulse wave estimation. In the conventional technique, when a phase difference occurs in luminance signals extracted from multiple measurement areas, there is a problem that it may not be possible to estimate a person's pulse wave based on the luminance signals in which this phase difference occurs.

[0005] The purpose of the present disclosure is to solve the above-mentioned problems and to provide a pulse wave estimation device that can estimate a person's pulse wave based on these luminance signals even when a phase difference occurs in luminance signals extracted from a plurality of measurement areas. [Means of solving the problem]

[0006] The pulse wave estimation device according to the invention comprises an image acquisition unit for acquiring an image image on which a person is depicted, a skin area detection unit for detecting a skin area of ​​the person based on the image image, a measurement area setting unit for setting, in an area corresponding to the skin area on the image image, a plurality of measurement areas to extract a pulse wave original signal representing luminance changes in time series in a first period, a pulse wave original signal extraction unit for extracting the pulse wave original signal for each of the measurement areas based on the luminance changes in the first period in the respective measurement areas, a segment generation unit for generating,for each of the measurement areas, based on the pulse wave original signal extracted from the corresponding measurement area, according to segment generation conditions, a plurality of pulse wave original signal segments, which are signals in which a pulse wave original signal for a second period has been partially extracted from the pulse wave original signal in the first period, and a pulse wave estimation unit for estimating the pulse wave of the person based on the plurality of pulse wave original signal segments generated for each of the measurement areas. [Effect of the invention]

[0007] According to the present disclosure, even if a phase difference occurs in luminance signals extracted from multiple measurement areas, the pulse wave of a person can be estimated based on these luminance signals. [Brief explanation of the drawings] [ Fig. 1] is a drawing showing a structural example of a pulse wave estimation device according to Embodiment 1. [ Fig. [Figure 2] is a drawing to explain the phase difference of the luminance signals that occurs between several measurement areas due to the characteristics of human blood flow. [ Fig. 3] Fig. 3A, Fig. 3B and Fig. 3C are drawings for explaining an exemplary method of setting measurement ranges by the measurement range setting unit in the pulse wave estimation device according to Embodiment 1. [ Fig. 4] is a drawing for explaining an example of pulse wave original signal segments generated by the segment generation unit in Embodiment 1. [ Fig. 5] is a drawing for explaining operations in a signal separation unit of Embodiment 1, wherein principal components are analyzed based on a plurality of pulse wave original signal segments and separation signals representing the analyzed principal components are generated. [ Fig. 6] is a drawing for explaining a detailed example of separation signals generated by the signal separation unit of Embodiment 1 and showing the main components. [ Fig. 7] is a flowchart for explaining the operation of the pulse wave estimation device according to Embodiment 1. [ Fig. 8] is a flowchart for explaining the details of the pulse wave estimation processing by the pulse wave estimation unit in step ST6 of Fig. 7. [ Fig. 9] Fig. 9A and Fig. 9B are drawings showing examples of the hardware structure of a pulse wave estimation device according to Embodiment 1. [ Fig. 10] is a drawing showing a structural example of a pulse wave estimation device according to Embodiment 2. [ Fig. 11] is a flowchart for explaining the operation of the pulse wave estimation device according to Embodiment 2. [ Fig. 12] is a drawing showing a structural example of a pulse wave estimation device according to Embodiment 3. [ Fig. 13] is a drawing for explaining the pulse wave original signal segments used by the pulse wave estimation device in Embodiment 3 to estimate the pulse wave of a subject. [ Fig. 14] is a flowchart for explaining the operation of the pulse wave estimation device according to Embodiment 3. [ Fig. 15] is a drawing showing a structural example of a pulse wave estimation unit in Embodiments 1 to 3 in the case where the pulse wave estimation device has the function of setting component weights for a plurality of separation signals representing a plurality of principal components estimated using a known general signal separation technique, and restoring the pulse wave original signal per measurement area based on the plurality of separation signals and the set component weight per separation signal. [ Fig. 16] is a drawing showing an image of an example of projection coefficient information in time series generated by a component weight setting unit. [ Fig. 17] is a flowchart for explaining the operation of the pulse wave estimation unit in Embodiments 1 to 3 in the case where the pulse wave estimation device has the function of setting component weights for a plurality of separation signals representing a plurality of principal components estimated using a known general signal separation technique, and restoring the pulse wave original signal per measurement area based on the plurality of separation signals and the set component weight per separation signal. [Embodiments of the invention]

[0008] Hereinafter, embodiments of the present disclosure will be explained in detail with reference to the drawings. Embodiment 1.

[0009] Fig. 1 is a drawing showing a structural example of a pulse wave estimation device 1 according to Embodiment 1.

[0010] The pulse wave estimation device 1 estimates the pulse wave of a person based on an image image in which the person is imaged. In the following description, the person whose pulse wave is to be estimated by the pulse wave estimation device 1 is also referred to as a "subject." The pulse wave estimation device 1 acquires an image image consisting of a series of frames Im(k) depicting an area in which at least one skin region should be located as an area including the subject's skin (hereinafter, "skin presence area") at a predetermined frame rate Fr. Where, k represents the frame number assigned to each frame. For example, the frame given at the time point following frame Im(k) is frame Im(k+1). In Embodiment 1, the skin area is an area corresponding to the subject's face. This is merely an example, and the skin area may also be an area other than the subject's face. For example, the skin area may also be an area corresponding to a part of the subject's face, such as the subject's eyes, eyebrows, nose, mouth, forehead, cheeks, or chin. Furthermore, the skin area may also be an area corresponding to a body part other than the face, such as the subject's head, shoulders, hands, neck, or legs. The skin area may also be a plurality of areas, such as a skin area corresponding to the cheek, a skin area corresponding to the forehead, or a skin area corresponding to the neck.

[0011] The pulse wave estimation device 1 then estimates the pulse wave of the subject from a series of frames Im(k-Tp+1) to Im(k) per a certain frame rate Tp acquired in the period preset as the period for performing pulse wave estimation (hereinafter, the "first period"), and outputs the pulse wave estimation result P(t), which is information representing the estimated pulse wave (hereinafter, the "pulse wave information"). Specifically, the pulse wave estimation device 1 sets a plurality of regions (hereinafter, the "measurement regions") for the subject's skin area in a series of frames Im(k-Tp+1) to Im(k).The pulse wave estimation device 1 then separates pulse wave components for a signal representing luminance changes per measurement range (hereinafter, "pulse wave original signal") using a general signal separation technique such as independent component analysis (ICA) or principal component analysis (PCA), and estimates the subject's pulse wave based on the separated pulse wave components. When separating the pulse wave components, the pulse wave estimation device 1 generates a plurality of pulse wave original signals (hereinafter, "pulse wave original signal segments"), with a pulse wave original signal of a predetermined period (hereinafter, "second period") partially extracted from the pulse wave original signal of the first period, and performs ICA or PCA on the plurality of pulse wave original signal segments.

[0012] Here, t represents an output number assigned for each frame rate Tp acquired in the first period. For example, the pulse wave estimation result given at the time point following the pulse wave estimation result P(t) is the pulse wave estimation result P(t+1). The frame number k and the output number t are integers greater than or equal to 1. The frame rate Tp is an integer greater than or equal to 2.

[0013] Estimating a subject's pulse wave from luminance changes across multiple measurement areas set on the skin is effective. However, due to the characteristics of human blood flow, a large distance between multiple measurement areas can lead to a phase difference in the luminance signals extracted from these multiple measurement areas.

[0014] Fig. Figure 2 is a drawing to explain the phase difference of luminance signals that occurs between multiple measurement areas due to the characteristics of human blood flow.

[0015] In Fig. 2, the left side of the drawing shows the skin area on the image of the test person. In Fig. 2, "H" stands for the test person and "sr" for the skin area. It is assumed that four measurement ranges are set in the skin area. In Fig. 2, the four measurement areas are labeled "R1," "R2," "R3," and "R4." Furthermore, a method for detecting a skin area and a method for setting measurement areas are described in more detail below.

[0016] The right side of the figure shows luminance values ​​represented by the luminance signals extracted from the measurement area labelled “R1” on the time axis (in Fig. 2 designated 22) and luminance values, which are represented by the luminance signals extracted from the measuring area designated “R3”, on the time axis (in Fig. 2 marked 21).

[0017] The subject's blood is pumped by the heart and flows from the neck to the forehead. Therefore, when the distance between the measurement areas in the direction of blood flow is large, such as the measurement areas labeled "R1" and "R3," in other words, when the distance to the heart between the measurement areas is large, a phase difference occurs in the luminance signals extracted from the measurement areas.

[0018] When a phase difference occurs between multiple luminance signals extracted from multiple measurement areas, the separation of the pulse wave components fails if the separation of the pulse wave components was performed using general signal separation techniques such as ICA or PCA for the pulse wave original signal per measurement area.

[0019] The pulse wave estimation device 1 according to Embodiment 1 takes into account the occurrence of a phase difference as described above, generates a plurality of pulse wave original signal segments, and performs separation of the pulse wave components using a general signal separation technique such as ICA or PCA on the generated plurality of pulse wave original signal segments, so that even if a phase difference occurs between the plurality of luminance signals extracted from a plurality of measurement areas, this phase difference is absorbed and the pulse wave of the subject can be estimated.

[0020] The number of test subjects included in the image can be one or multiple people. To simplify the explanation, the following explanation of embodiment 1 assumes one person as the test subject included in the image.

[0021] In Embodiment 1, for example, the pulse wave estimation device 1 is installed in a vehicle (not shown), and the test subject is a driver of the vehicle. That is, the pulse wave estimation device 1 estimates the pulse wave of the driver of the vehicle. The driver's pulse wave information estimated by the pulse wave estimation device 1 is output to, for example, a condition estimation device (not shown) installed in the vehicle. The condition estimation device estimates the driver's condition based on the driver's pulse wave estimation result P(t) output by the pulse wave estimation device 1. For example, if the condition estimation device estimates that the driver's condition is unsuitable for driving, it outputs a warning sound to the driver.

[0022] As in Fig. 1, the pulse wave estimation device 1 includes an image acquisition unit 11, a skin area detection unit 12, a measurement area setting unit 13, a pulse wave original signal extraction unit 14, a segment generation unit 15, a pulse wave estimation unit 16, and an output unit 17. The pulse wave estimation unit 16 includes a signal separation unit 161, a restoration unit 162, and an estimation unit 163. The pulse wave assessment device 1 is connected to an imaging device 3 installed in the vehicle. The imaging device 3 is arranged to image the skin presence area of ​​the test subject, in this case the driver. The imaging device 3 can be used, for example, together with an imaging device of a so-called "driver monitoring system (DMS)" installed in a vehicle to monitor the driver's condition in the vehicle.

[0023] The image acquisition unit 11 acquires an image image on which a subject is imaged from the imaging device 3. The image acquisition unit 11 outputs the acquired image to the skin area detection unit 12.

[0024] The skin region detection unit 12 detects a skin region of the subject based on the frame Im(k) included in the image acquired by the image acquisition unit 11. The skin region detection unit 12 can detect the skin region using a known means. For example, the skin region detection unit 12 can detect a skin region using a cascade-type facial sensor that uses Haar-like feature sizes. The skin area detection unit 12 generates skin area information S(k) representing the detected skin area. The skin area information S(k) may include information representing the presence / absence of detection of a skin area, and information representing the position and size of a detected skin area in the image image. In Embodiment 1, a skin area is expressed by a rectangular area in the image image, and the skin area information S(k) includes information representing the position and size of the rectangular area in the image image. Specifically, the skin area information S(k) represents, for example, the presence / absence of detection of the subject's face, central coordinates Fc (Fcx, Fcy) of a rectangle enclosing the subject's face in the image, and a width Fcw and a height Fch of the rectangle if the skin area is an area corresponding to the subject's face. The presence / absence of detection of the subject's face is expressed, for example, by "1" if detection was possible and by "0" if detection was not possible. Furthermore, the central coordinates of the rectangle surrounding the face are represented by the coordinate system of the frame Im(k). Furthermore, the skin area detection unit 12 can also detect multiple skin areas. The skin area detection unit 12 outputs the generated skin area information S(k) to the measurement area setting unit 13.

[0025] Based on the frame Im(k) of the image image acquired by the image acquisition unit 11 and the skin area information S(k) output from the skin area detection unit 12, the measurement area setting unit 13 sets a plurality of measurement areas in the image area corresponding to the skin area represented by the skin area information S(k) in the frame Im(k) to extract therefrom a pulse wave original signal representing time-series luminance changes in a first period. Furthermore, the measurement area setting unit 13 may acquire the image image acquired by the image acquisition unit 11 via the skin area detection unit 12. When the measurement range setting unit 13 sets a plurality of measurement ranges, it generates measurement range information R(k) representing the set plurality of measurement ranges. The measurement range information R(k) includes information representing the position and size of N (an integer greater than or equal to 2) measurement ranges on the image map. The individual measurement ranges are the measurement ranges ri(k) (i = 2, ..., N). In Embodiment 1, a measurement range ri(k) is a quadrilateral, and the position and size of the measurement range ri(k) correspond to the coordinate values ​​of the four vertices of the quadrilateral on the image map.

[0026] Fig. 3A, Fig. 3B and Fig. 3C are drawings for explaining an exemplary method for setting measurement ranges ri(k) by the measurement range setting unit 13 in the pulse wave estimation device 1 according to Embodiment 1.

[0027] Based on Fig. 3, an example of a method for setting a plurality of measuring ranges ri(k) by the measuring range setting unit 13 is explained.

[0028] First, as in Fig. 3A and Fig. 3B, by the measurement area setting unit 13, in a skin area sr represented by the skin area information S(k), Ln-piece (positive integer) landmarks of facial organs such as the outer corner of the eye, the inner corner of the eye, the nose, and the mouth are detected. In Fig. 3A and Fig. In Figure 3B, the landmarks are marked by circles. A vector in which the measurement range setting unit 13 has stored the coordinate values ​​of the detected landmarks is specified as L(k).

[0029] Furthermore, the measurement area setting unit 13 can detect the facial organs by known means, for example, by a model called Constrained Local Model (CLM).

[0030] Subsequently, the measurement range setting unit 13 sets the corner point coordinates of quadrilaterals of the measurement ranges ri (k) based on the detected landmarks. For example, the measurement range setting unit 13 sets the corner point coordinates of quadrilaterals as shown in Fig. 3C, and sets N measuring ranges ri(k).

[0031] To explain the setting of the measurement areas ri(k) in a portion of the skin area sr corresponding to a cheek by the measurement area setting unit 13 using an example, a facial contour landmark LA1 and a nose contour landmark LA2 are selected by the measurement area setting unit 13. The measurement area setting unit 13 may first select the nose contour landmark LA2 and then select the facial contour landmark LA1 closest to the nose contour landmark LA2.

[0032] Then, the measuring range setting unit 13 sets auxiliary landmarks a1, a2 and a3 so that a line segment between the landmark LA1 and the landmark LA2 is divided into four parts.

[0033] Likewise, the measurement area setting unit 13 selects a facial contour landmark LB1 and a nose contour landmark LB2. Furthermore, the measurement area setting unit 13 sets auxiliary landmarks b1, b2, and b3 so that a line segment between the landmark LB1 and the landmark LB2 is divided into four parts. For example, facial contour landmarks and nose contour landmarks adjacent to the landmarks LA1 and LA2 can be selected as the landmarks LB1 and LB2, respectively. The measurement area setting unit 13 sets a quadrilateral area encompassing the auxiliary landmarks a1, b1, b2, and a2 as the measurement area R5. The auxiliary landmarks a1, b1, b2, and a2 each become the corner point coordinates corresponding to the measurement area R5.

[0034] Likewise, the measuring range setting unit 13 sets a measuring range R6 encompassing the auxiliary orientation points a2, b2, b3 and a3 and the corner point coordinates of the measuring range R6.

[0035] Furthermore, an example was explained here in which measurement areas ri(k) were set for a portion corresponding to the cheek, but the measurement area setting unit 13 similarly sets, for example, measurement areas ri(k) and vertex coordinates of these measurement areas ri(k) for the skin area sr of other portions of the cheek and portions corresponding to the chin. Fig. 3C, the measurement range setting unit 13 may further set the measurement ranges ri(k) for the portion of the subject's skin area sr corresponding to the forehead, the portion corresponding to the neck, or the portion corresponding to the tip of the nose.

[0036] The measurement range setting unit 13 may also set measurement ranges ri(k) using a method other than CLM. For example, the measurement range setting unit 13 may also set measurement ranges ri(k) using a tracking technique such as the Kanade-Lucas-Tomasi (KLT) tracker. Specifically, the measurement range setting unit 13 may also detect the coordinates of the facial organ points using CLM for the skin area in the first frame Im(1) of a series of frames Im(k-Tp+1) to Im(k), and track the facial organ points starting from the skin area in the next frame Im(2) using the KLT tracker to calculate the facial organ points for the skin area of ​​each frame Im(k). Since detection errors due to tracking accumulate in this case, the measurement range setting unit 13 may also perform CLM every few frames and reset the coordinate positions of the facial organ points or perform other reset processing.

[0037] The measurement range setting unit 13 outputs the generated measurement range information R(k) to the pulse wave original signal extraction unit 14.

[0038] The pulse wave origin signal extraction unit 14 extracts, based on the frame Im(k) of the image image acquired by the image acquisition unit 11 and the measurement range information R(k) output from the measurement range setting unit 13, a pulse wave origin signal representing luminance changes in the first period from each of the plurality of measurement ranges ri(k) represented by the measurement range information R(k) in the frame Im(k). Further, the pulse wave origin signal is a signal that is the origin of a pulse wave. The pulse wave estimation device 1 uses the pulse wave origin signal to estimate the pulse wave of the subject. The estimation of the subject's pulse wave is performed by the pulse wave estimation unit 16. The pulse wave estimation unit 16 will be explained in detail below.

[0039] The pulse wave origin signal extraction unit 14 can acquire the image image acquired by the image acquisition unit 11 via the skin area detection unit 12 and the measurement area setting unit 13.

[0040] Once the pulse wave original signal has been extracted, the pulse wave original signal extraction unit 14 generates pulse wave original signal information W(t) representing the extracted pulse wave original signal.

[0041] The pulse wave original signal information W(t) includes information representing a pulse wave original signal wi(t) extracted from the measurement range ri(k). The pulse wave original signal wi(t), which is time series data for the Tp component, is extracted based on, for example, frames Im(k-Tp+1), Im(k-Tp+2), ..., Im(k) of a past Tp component and measurement range information R(k-Tp+1), R(k-Tp+2), ..., R(k).

[0042] When extracting the pulse wave original signal wi(t), the pulse wave original signal extraction unit 14 calculates, for each frame Im(k) of the image image, a difference Gi(j) (j=k-Tp+1, k-Tp+2, ..., k) of the luminance feature size of each measurement region ri(k) from a previous frame Im(k-1). The luminance feature size is a value calculated for each measurement region ri(j) based on the luminance values ​​in frame Im(j) of the image image. The luminance feature size is an average or variance, etc., of the luminance values ​​of the pixels included in the measurement regions ri(j). In Embodiment 1, for example, the luminance feature size is the average of the luminance values ​​of the pixels included in the measurement regions ri(j).

[0043] The pulse wave original signal extraction unit 14 time-series-arrays the Gi(j) calculated for each frame Im(k) of the image acquired in the first period to determine the pulse wave original signal wi(t). That is, the pulse wave original signal extraction unit 14 determines the pulse wave original signal wi(t) = [Gi(k-Tp+1), Gi(k-Tp+2), ..., Gi(k)].

[0044] The pulse wave original signal extraction unit 14 may further extract the pulse wave original signal wi(t) by calculating the luminance feature size of each measurement area ri(k) for each frame Im(k) of the imaging image.

[0045] The pulse wave original signal extraction unit 14 generates pulse wave original signal information W(t) representing a pulse wave original signal wi(t) in each measurement range ri(k). The pulse wave original signal information W(t) includes the pulse wave original signal wi(t) in each measurement range ri(k) and information representing, for the respective pulse wave original signal wi(t), from which measurement range ri(k) the pulse wave original signal wi(t) was extracted. The pulse wave original signal extraction unit 14 outputs the generated pulse wave original signal information W(t) to the segment generation unit 15.

[0046] The segment generation unit 15 generates, based on the pulse wave original signal information W(t) output from the pulse wave original signal extraction unit 14, a plurality of pulse wave original signal segments per measurement range ri(k) according to predetermined conditions (hereinafter "segment generation conditions"), which are signals in which a pulse wave original signal for a second period has been partially extracted from the pulse wave original signal wi(t) in the first period.

[0047] The segment generation conditions are generated in advance by an administrator or the like and stored by the segment generation unit 15.

[0048] For example, the following conditions (1) to (5) are set as segment generation conditions. The segment generation unit 15 generates a plurality of pulse wave original signal segments so that all of the conditions (1) to (5) are satisfied. (1) The second period must be shorter than the first period. (2) Regarding the multiple pulse wave original signal segments in a certain measurement range, a certain pulse wave original signal segment is a signal that does not completely coincide with other pulse wave original signal segments on the time axis. (3) Regarding the multiple pulse wave original signal segments in a certain measurement range, a certain pulse wave original signal segment is a signal that partially overlaps with other pulse wave original signal segments on the time axis. (4) In each measurement range, the duration of partial overlap of a particular pulse wave original signal segment with other pulse wave original signal segments on the time axis is the same. (5) In each measurement range, the lengths of the multiple pulse wave original signal segments on the time axis are identical to each other.

[0049] Fig. 4 is a drawing for explaining an example of pulse wave original signal segments generated by the segment generation unit 15 in Embodiment 1.

[0050] In Fig. 4, the left drawing is a diagram showing the skin area sr for which the measurement areas ri(k) were set (hereinafter "skin area diagram"). In the skin area diagram, the subject is denoted by "H." As shown in the skin area diagram, it is assumed that four measurement areas ri(k) are set in the skin area sr. Specifically, it is assumed that two measurement areas ri(k) (denoted by "n1" and "n2" in the skin area diagram) are set in the subject's chin area, and two measurement areas ri(k) (denoted by "f1" and "f2" in the skin area diagram) are set in the subject's forehead area. In the following description, the measuring range ri(k) designated by “n1” is referred to as the “first measuring range”, the measuring range ri(k) designated by “n2” is referred to as the “second measuring range”, the measuring range ri(k) designated by “f1” is referred to as the “third measuring range” and the measuring range ri(k) designated by “f2” is referred to as the “fourth measuring range”.

[0051] In Fig. 4, the middle drawing is a diagram illustrating an example of pulse wave original signal segments generated by the segment generation unit 15 for the first measurement range and the third measurement range (hereinafter, "segment generation diagram"). In the segment generation diagram, the luminance values ​​represented by the pulse wave original signal wi(t) in the first period (indicated by "X" in the segment generation diagram) are displayed on the time axis. In the segment generation scheme, an example of a plurality of pulse wave original signal segments (= p1-1, p1-2, p1-3, p1-4, p1-5, ..., p1-M) for the first measurement range is shown at the top, which were generated by the segment generation unit 15 by partially extracting the pulse wave original signal wi(t) for the second period from the pulse wave original signal wi(t) extracted from the first measurement range (indicated by “401” in the segment generation scheme), while an example of a plurality of pulse wave original signal segments (= p3-1, p3-2, p3-3, p3-4, p1-5, ..., p1-M) for the third measurement range is shown at the bottom, which were generated by the segment generation unit 15 by partially extracting the pulse wave original signal wi(t) for the second period from the pulse wave original signal wi(t) extracted from the third measurement range. Pulse wave origin signal wi(t) (marked by “402” in the segment generation scheme) was generated. The segment generation unit 15 generates, for each measurement range ri(k) for the first period, which is the period for performing pulse wave estimation, according to the segment generation conditions, M pulse wave original signal segments (where M is an integer of 2 or more) in which the pulse wave original signal wi(t) for the second period has been partially extracted from the pulse wave original signal wi(t) of the first period.

[0052] Furthermore, each pulse wave original signal segment is assigned information indicating the measurement range ri(k) from which the pulse wave original signal segment was generated from the pulse wave original signal wi(t), the numbered pulse wave original signal segment it is on the time axis, and information indicating the time range on the time axis of the pulse wave original signal wi(t) in the time series from which the extraction was performed. For example, the pulse wave original signal segment p1-1 indicates that it is the first pulse wave original signal segment on the time axis of multiple pulse wave original signal segments generated for the pulse wave original signal wi(t) of the first measurement range, in other words, the pulse wave original signal segment of the earliest time period.The pulse wave original signal segment p1-1 is assigned information indicating that it is, for example, the pulse wave original signal wi(t) of the first measuring range from ◯ h:◯min:◯s to △h: △min: △s.

[0053] On the segment generation scheme of Fig. 4, for the sake of simplicity, M pulse wave original signal segments are shown for the first measurement range and the third measurement range, but the segment generation unit 15 also generates M pulse wave original signal segments for the second measurement range and the fourth measurement range, respectively. That is, the segment generation unit 15 also generates M pulse wave original signal segments (= p2-1, p2-2, p2-3, p2-4, p2-5, ..., p2-M) for the second measurement range and M pulse wave original signal segments (= p4-1, p4-2, p4-3, p4-4, p4-5, ..., p4-M) for the fourth measurement range.

[0054] As mentioned above, a large difference between the measurement ranges ri(k) with respect to the distance to the heart leads to a phase difference in the luminance signals extracted from the measurement ranges ri(k). Conversely, if there is almost no difference between the measurement ranges ri(k) with respect to the distance to the heart, there is no phase difference in the luminance signals extracted from the measurement ranges ri(k). Since the second measurement range shows little difference from the first measurement range with respect to the distance to the heart, the M pulse wave original signal segments for the second measurement range are content-identical to the M pulse wave original signal segments for the first measurement range.Furthermore, since the fourth measurement range differs little from the third measurement range in terms of distance to the heart, the M pulse wave source signal segments for the fourth measurement range are identical in content to the M pulse wave source signal segments for the third measurement range. Furthermore, the term "identical" in the first embodiment is not limited to an exact match, but also includes an "approximate match" within acceptable limits.

[0055] When the segment generation unit 15 generates a plurality of pulse wave original signal segments for each measurement range ri(k), it generates a trajectory matrix from the generated pulse wave original signal segments and stacks them in a storage unit (not shown). The storage unit is provided at a location where the pulse wave estimation device 1 can be referenced.

[0056] In Fig. 4, there are four measurement ranges ri(k), so that the segment generation unit 15, when generating M pulse wave original signal segments for each measurement range ri(k), stacks an M × 4-row trajectory matrix (= [p1-1, p1-2, p1-3, p1-4, p1-5, ..., p1-M], ..., [p4-1, p4-2, p4-3, p4-4, p4-5, ..., p4-M]) (see Fig. 4, right drawing).

[0057] Furthermore, there are four measurement ranges ri(k) here, but this is just an example, and N measurement ranges ri(k) can be set. The segment generation unit 15 stacks an M × N-row trajectory matrix (= [p1-1, p1-2, p1-3, p1-4, p1-5, ..., p1-M], ..., [pN-1, pN-2, pN-3, pN-4, pN-5, ..., pN-M]).

[0058] When the segment generation unit 15 stacks the trajectory matrix, it informs the pulse wave estimation unit 16.

[0059] When the pulse wave estimation unit 16 is informed by the segment generation unit 15 that the trajectory matrix has been stacked, it estimates the pulse wave of the subject based on the plurality of pulse wave original signal segments generated per measurement range ri(k). Specifically, the signal separation unit 161 first generates signals (hereinafter "separation signals") representing a plurality of signal components (hereinafter "principal components") based on the plurality of pulse wave original signal segments. Specifically, the signal separation unit 161 analyzes a plurality of principal components using a general signal separation technique such as PCA or ICA, and generates separation signals representing the analyzed plurality of principal components. By analyzing the signal components using a general signal separation technique such as PCA or ICA, the signal separation unit 161 separates the components typical of pulse wave components from the components typical of noise components from the plurality of pulse wave original signal segments.

[0060] Further, the signal separation unit 161 generates separation signals representing a plurality of principal components based on the plurality of pulse wave original signal segments corresponding to all measurement ranges ri(k).

[0061] Fig. 5 is a drawing for explaining operations in the signal separation unit 161 of Embodiment 1, wherein principal components are analyzed based on the plurality of pulse wave original signal segments and separation signals representing the analyzed principal components are generated.

[0062] It is assumed that the measuring ranges ri(k), as in Fig. 4, four measuring ranges ri(k) can be set, namely the first measuring range, the second measuring range, the third measuring range and the fourth measuring range.

[0063] The one on the left side of Fig. 5 until several pulse wave original signal segments have been generated for each of the first, second, third and fourth measurement areas and the M × 4-line trajectory matrix is ​​stacked, is the process flow performed by the segment generation unit 15 and already based on Fig. 4, which is why a redundant explanation is omitted.

[0064] The signal separation unit 161 analyzes the principal components using a well-known signal separation technique, such as PCA or ICA, for the M × 4-row trajectory matrix stacked by the segment generation unit 15. As a result, the signal separation unit 161 analyzes C (positive integer) principal components and generates separation signals representing the analyzed C principal components. Furthermore, in the general signal separation technique, generation is performed sequentially starting from the principal components that contain as much information as possible. That is, the first principal component c1, the second principal component c2, the third principal component c3, ..., and the C-th principal component cC are in the order of their information content.

[0065] Fig. 6 is a drawing for explaining a detailed example of separation signals generated by the signal separation unit 161 of Embodiment 1 and showing the main components.

[0066] Furthermore, Fig. 6 which in Fig. 5 shown separation signals (= [c1, c2, c3, c4, c5, ..., cC]) in detail, ie the ones based on the ones shown in Fig. 4 shown for each of the first, second, third and fourth measurement ranges generated separation signals (= [c1, c2, c3, c4, c5, ..., cC]) generated by the signal separation unit 161.

[0067] Each separation signal contains the trajectory matrix of several pulse wave original signal segments, from which the information content contained in the principal component represented by the separation signal was analyzed. Fig. 6, the separation signals (= [c1, c2, c3, c4, c5, ..., cC]) representing the first principal component c1 to the C-th principal component cC are separation signals generated based on the pulse wave original signal segments (= p1-1, p1-2, p1-3, p1-4, p1-5, ..., p1-M) generated from the pulse wave original signal wi(t) of the first measurement range, the pulse wave original signal segments (= p2-1, p2-2, p2-3, p2-4, p2-5, ..., p2-M) generated from the pulse wave original signal wi(t) of the second measurement range, the pulse wave original signal segments (= p3-1, p3-2, p3-3, p3-4, p3-5, ..., p3-M) and the pulse wave original signal wi(t) of the fourth measuring range (= p4-1, p4-2, p4-3, p4-4, p4-5, ..., p4-M).Consequently, the separation signals representing the first principal component c1 to the C-th principal component cC each comprise an M × 4-row trajectory matrix (= [p1-1, p1-2, p1-3, p1-4, p1-5, ..., p1-M], ..., [p4-1, p4-2, p4-3, p4-4, p4-5, ..., p4-M]).

[0068] Furthermore, each separation signal (= [c1, c2, c3, c4, c5, ..., cC]) is assigned a projection coefficient. Projection coefficients are coefficients assigned according to the information content of the principal components and are assigned during the analysis of the principal components using well-known general signal separation techniques. Projection coefficients are assigned to each pulse wave original signal segment in each measurement range ri(k) for each separation signal. Fig. 6, for example, the first principal component c1 is assigned projection coefficients corresponding to each of the M pulse wave original signal segments in the first measurement range, projection coefficients corresponding to each of the M pulse wave original signal segments in the second measurement range, projection coefficients corresponding to each of the M pulse wave original signal segments in the third measurement range, and projection coefficients corresponding to each of the M pulse wave original signal segments in the fourth measurement range.

[0069] In Fig. For example, Figure 6 shows the projection coefficient (q1,1-1) assigned to the separation signal representing the first principal component c1, which is the projection coefficient corresponding to the first pulse wave original signal segment on the time axis in the first measurement range of the first principal component c1. Furthermore, Fig. 6, for example, the projection coefficient (q1,4-M) assigned to the separation signal representing the first principal component c1 is the projection coefficient corresponding to the M-th pulse wave original signal segment on the time axis in the fourth measurement range of the first principal component c1. Furthermore, in Fig. 6 For example, the projection coefficient (qC,1-1) assigned to the separation signal representing the C-th principal component cC is the projection coefficient corresponding to the first pulse wave original signal segment on the time axis in the first measurement range of the C-th principal component cC.

[0070] The signal separation unit 161 outputs the generated plurality of separation signals to the restoration unit 162, specifically, the separation signal information Sep(t) regarding the plurality of separation signals representing the plurality of generated main components. The separation signal information Sep(t) includes C separation signals generated from a plurality of (M × N) pulse wave original signal segments generated based on the pulse wave original signal information W(t).

[0071] The restoration unit 162 restores the pulse wave original signal wi(t) per measurement range ri(k) based on the separation signal information Sep(t) output from the signal separation unit 161. As described above, each separation signal included in the separation signal information Sep(t) includes a plurality of pulse wave original signal segments generated based on each pulse wave original signal wi(t) of each measurement range ri(k), so that the restoration unit 162 can restore the pulse wave original signal wi(t) per measurement range ri(k) from the plurality of separation signals included in the separation signal information Sep(t).

[0072] When the restoration unit 162 restores the pulse wave original signal wi(t) per measurement range ri(k), it generates post-restoration pulse wave original signal information RW(t) representing the restored pulse wave original signal wi(t) per measurement range ri(k). The post-restoration pulse wave original signal information RW(t) includes the restored pulse wave original signal wi(t) (hereinafter, "post-restoration pulse wave original signal") per measurement range ri(k). The recovery unit 162 outputs the generated pulse wave original signal information after recovery RW(t) to the estimation unit 163.

[0073] The estimation unit 163 estimates the pulse wave of the subject based on the pulse wave original signal information RW(t) after restoration output from the restoration unit 162. The estimation unit 163 outputs the pulse wave estimation result P(t), which is pulse wave information representing the estimated pulse wave, to the output unit 17. The pulse wave information can be, for example, time series data of the subject's pulse wave estimated by the assessment unit 163, the subject's pulse rate, or the subject's pulse interval. For simplicity, it is assumed here that the pulse wave information is the subject's pulse rate (number of beats per minute).

[0074] The method for estimating the pulse wave of the subject by the estimation unit 163 is described in detail. For example, the estimation unit 163 determines the S / N ratio of the pulse wave original signal after restoration of each measurement range ri(k). The estimation unit 163 calculates synthetic pulse wave signal information D(t) by summing the restored pulse wave original signals corresponding to each measurement range ri(k) after weighting them based on the S / N ratio determined for the pulse wave original signals after restoration of each measurement range ri(k). That is, the estimation unit 163 calculates the synthetic pulse wave signal information D(t) for all measurement ranges ri(k). Since the weighting is performed based on the S / N ratio, the synthetic pulse wave signal information D(t) is assumed to be signals typical of pulse wave components from which noise components have been removed. The estimation unit 163 then performs a Fourier transform of the synthetic pulse wave signal information D(t) and calculates the peak frequency in the frequency power spectrum within a predetermined frequency range as the pulse frequency. The predetermined frequency range is set taking into account the range of the human heart rate.

[0075] The output unit 17 outputs the pulse wave estimation result P(t) output by the pulse wave estimation unit 16 to, for example, a state estimation device. The function of the output unit 17 can also be provided in the pulse wave estimation unit 16.

[0076] The operation of the pulse wave estimation device 1 according to Embodiment 1 will be described. Fig. 7 is a flowchart for explaining the operation of the pulse wave estimation device 1 according to Embodiment 1. The pulse wave estimation device 1 repeats, for example, the steps shown in the flowchart of Fig. 7 processing shown when the vehicle is turned on until the vehicle is turned off.

[0077] The image acquisition unit 11 acquires an image image on which a subject is imaged from the imaging device 3 (step ST1). The image acquisition unit 11 outputs the acquired image to the skin area detection unit 12.

[0078] The skin area detection unit 12 detects the skin area of ​​the subject from the frame Im(k) included in the image image acquired by the image acquisition unit 11 in step ST1 (step ST2). The skin area detection unit 12 outputs the generated skin area information S(k) to the measurement area setting unit 13.

[0079] The measurement area setting unit 13 sets a plurality of measurement areas ri(k) in the image area in the frame Im(k) corresponding to the skin area represented by the skin area information S(k) based on the frame Im(k) of the image image acquired by the image acquisition unit 11 in step ST1 and the skin area information S(k) outputted by the skin area detection unit 12 in step ST2, to extract the pulse wave original signal wi(t) representing time-series luminance changes in the first period (step ST3). The measurement area setting unit 13 outputs the generated measurement area information R(k) to the pulse wave original signal extraction unit 14.

[0080] The pulse wave original signal extraction unit 14 extracts, based on the frame Im(k) of the image image acquired by the image acquisition unit 11 in step ST1 and the measurement range information R(k) output from the measurement range setting unit 13 in step ST3, a pulse wave original signal wi(t) representing luminance changes in the first period from each of the plurality of measurement ranges ri(k) represented by the measurement range information R(k) in the frame Im(k) (step ST4). The pulse wave original signal extraction unit 14 generates pulse wave original signal information W(t) representing a pulse wave original signal wi(t) in each measurement range ri(k). The pulse wave original signal extraction unit 14 outputs the generated pulse wave original signal information W(t) to the segment generation unit 15.

[0081] The segment generation unit 15 generates, based on the pulse wave original signal information W(t) output from the pulse wave original signal extraction unit 14 in step ST4, a plurality of pulse wave original signal segments per measurement range ri(k) according to the segment generation conditions, which are signals in which the pulse wave original signal wi(t) for the second period has been partially extracted from the pulse wave original signal wi(t) of the first period (step ST5). When the segment generation unit 15 generates a plurality of pulse wave original signal segments for each measurement range ri(k), it generates a trajectory matrix from the generated pulse wave original signal segments and stacks them in a storage unit. When the segment generation unit 15 stacks the trajectory matrix, it informs the pulse wave estimation unit 16.

[0082] When the pulse wave estimation unit 16 is informed by the segment generation unit 15 that the trajectory matrix has been stacked in step ST5, it performs pulse wave estimation processing to estimate the subject's pulse wave based on the multiple pulse wave original signal segments generated per measurement area ri(k) (step ST6). The pulse wave estimation unit 16 outputs the pulse wave estimation result P(t), which is pulse wave information representing the estimated pulse wave, to the output unit 17. The output unit 17 outputs the pulse wave estimation result P(t) output by the pulse wave estimation unit 16 to, for example, a state estimation device.

[0083] Fig. Fig. 8 is a flowchart for explaining the details of the pulse wave estimation processing by the pulse wave estimation unit 16 in step ST6 of Fig. 7.

[0084] The signal separation unit 161 generates separation signals representing a plurality of principal components based on a plurality of pulse wave original signal segments (step ST11). The signal separation unit 161 outputs the generated plurality of separation signals to the restoration unit 162, specifically, the separation signal information Sep(t) regarding the separation signals representing the generated plurality of principal components.

[0085] The restoration unit 162 restores the pulse wave original signal wi(t) per measurement range ri(k) based on the separation signal information Sep(t) outputted by the signal separation unit 161 in step ST11 (step ST12). The recovery unit 162 outputs the pulse wave original signal information after recovery RW(t) to the estimation unit 163.

[0086] The estimation unit 163 estimates the pulse wave of the subject based on the pulse wave original signal information after restoration RW(t) outputted by the restoration unit 162 in step ST12 (step ST13). The estimation unit 163 outputs the pulse wave estimation result P(t), which is pulse wave information representing the estimated pulse wave, to the output unit 17. The output unit 17 outputs the pulse wave estimation result P(t) output by the pulse wave estimation unit 16 to, for example, a state estimation device.

[0087] In this way, the pulse wave estimation device 1 according to Embodiment 1 sets a plurality of measurement areas ri(k) in an area corresponding to the skin area on an image image detected from the image image depicting a subject, and for each measurement area ri(k), the pulse wave original signal wi(t) in the measurement area ri(k) concerned is extracted based on the luminance changes in the first period, which is the period for performing pulse wave estimation.For each measurement range ri(k), the pulse wave estimation device 1 generates, based on the pulse wave original signal wi(t) extracted from this measurement range ri(k), a plurality of pulse wave original signal segments in which the pulse wave original signal wi(t) for the second period is partially extracted from the pulse wave original signal wi(t) of the first period according to the segment generation conditions, and estimates the pulse wave of the subject based on the plurality of pulse wave original signal segments. Because the pulse wave estimation device 1 generates a plurality of pulse wave original signal segments based on the pulse wave original signal wi(t) of the first period, in which the pulse wave original signal wi(t) for the second period, which is shorter than the first period, has been partially extracted, the pulse wave original signal wi(t) can be in a state of in-phase pairs. Furthermore, because the pulse wave estimation device 1 estimates the subject's pulse wave based on the pulse wave original signal wi(t) in the state of in-phase pairs, that is, a plurality of pulse wave original signal segments, the separation of the pulse wave components can be performed accurately for a plurality of pulse wave original signals wi(t) using known general signal separation techniques such as ICA or PCA, thereby enabling the subject's pulse wave to be estimated.In this way, even if a phase difference occurs in the luminance signals extracted from the plurality of measurement areas ri(k), the pulse wave estimation device 1 can estimate the pulse wave of a subject based on the luminance signals.

[0088] Fig. 9A and Fig. 9B are drawings showing examples of the hardware structure of the pulse wave estimation device 1 according to Embodiment 1.

[0089] In Embodiment 1, the functions of the image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16, and the output unit 17 are realized by a processing circuit 101.That is, the pulse wave estimation device 1 includes the processing circuit 101 for generating a plurality of pulse wave original signal segments in which a pulse wave original signal wi(t) representing luminance changes in the plurality of measurement areas ri(k) set for a skin area of ​​a subject in an image acquired in the first period has been partially extracted, using techniques such as independent component analysis or principal component analysis, separating pulse wave components from the pulse wave original signal segments, and controlling the estimation of a subject's pulse wave based on the separated pulse wave components.

[0090] The processing circuit 101 may be special hardware, as in Fig. 9A, or a processor 104 executing a program stored in a memory, as shown in Fig. 9B shown.

[0091] If the processing circuit 101 is special hardware, the processing circuit 101 may be, for example, a single circuit, a complex circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0092] When the processing circuit is a processor 104, the functions of the image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16, and the output unit 17 are realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in a memory 105. The processor 104 reads and executes the program stored in the memory 105 to perform the functions of the image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16, and the output unit 17.That is, the pulse wave estimation device 1 includes the memory 105 for storing therein a program for executing steps ST1 to ST6 of . Fig. 7 are finally executed upon execution by the processor 104. Furthermore, it can also be said that the program stored in the memory 105 causes the computer to execute the procedures or processes of the image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16, and the output unit 17. Here, the term memory 105 refers to, for example, RAM, ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), or similar non-volatile or volatile semiconductor memories, magnetic disks, flexible disks, optical disks, compact discs, mini-disks, DVDs (Digital Versatile Discs), etc.

[0093] Furthermore, the functions of the image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave original signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16, and the output unit 17 can also be implemented partly with dedicated hardware and partly with software or firmware. For example, the functions of the image acquisition unit 11 and the output unit 17 can be implemented by the processing circuit 101 as dedicated hardware, while the functions of the skin area detection unit 12, the measurement area setting unit 13, the pulse wave original signal extraction unit 14, the segment generation unit 15, and the pulse wave estimation unit 16 can be implemented by the processor 104 reading and executing a program stored in the memory 105.A memory unit not shown is formed, for example, by the memory 105. Furthermore, the pulse wave estimation device 1 comprises an input interface device 102 and an output interface device 103 for wired or wireless communication with devices such as the imaging device 3.

[0094] Furthermore, in Embodiment 1 above, the test subject is the driver of a vehicle, but this is only an example. Test subjects can also be passengers other than the driver of the vehicle.

[0095] Furthermore, in the above embodiment 1, the pulse wave estimation device 1 is an in-vehicle device, and the map image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave original signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16, and the output unit 17 are provided in the in-vehicle device.

[0096] In addition, some of the image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16, and the output unit 17 may be installed in the in-vehicle device, while others are provided in a server connected to the in-vehicle device via a network, and the system may be composed of the in-vehicle device and the server.

[0097] Furthermore, the image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16, and the output unit 17 may also all be provided on the server.

[0098] Furthermore, the pulse wave estimation device 1 according to the above embodiment 1 is not limited to in-vehicle devices installed in vehicles, but can also be used for household appliances, for example. Furthermore, the test subject can be various people, not just vehicle passengers. To give a concrete example, the pulse wave estimation device 1 can be installed on a television set located in the living room of an apartment. In this case, the test subject is a resident of the apartment or other users. The pulse wave estimation device 1 estimates the user's pulse wave based on the image projected by the imaging device installed on the television set.

[0099] As described above, the pulse wave estimation device 1 according to Embodiment 1 includes an image acquisition unit 11 that acquires an image image in which a person (a test subject) is depicted; a skin area detection unit 12 that detects a skin area of ​​the person (the test subject) based on the image image; a measurement area setting unit 13 that sets a plurality of measurement areas ri(k) in an area corresponding to the skin area on the image image to extract therefrom a pulse wave original signal wi(t) representing luminance changes in time series in a first period; a pulse wave original signal extraction unit 14 that extracts a pulse wave original signal wi(t) for each measurement area ri(k) based on the luminance changes in the first period in the respective measurement area ri(k); a segment generation unit 15;which generates, for each measurement area ri(k), a plurality of pulse wave original signal segments based on the pulse wave original signal wi(t) extracted from the measurement area according to the segment generation conditions, in which a pulse wave original signal wi(t) for the second period has been partially extracted from the pulse wave original signal wi(t) of the first period; and a pulse wave estimation unit 16 which estimates the pulse wave of the subject (the test subject) based on the plurality of pulse wave original signal segments generated for each measurement area ri(k). Therefore, the pulse wave estimation device 1 can estimate a subject's pulse wave based on the luminance signals even if a phase difference occurs in the luminance signals extracted from the plurality of measurement areas ri(k). Embodiment 2.

[0100] In Embodiment 1, the length of the second period used to generate the pulse wave original signal segments was uniform.

[0101] In Embodiment 2, an embodiment is explained in which the length of the second period used to generate the pulse wave original signal segments is regulated.

[0102] Further, in the following Embodiment 2, as in Embodiment 1, it is assumed that the pulse wave estimation device is installed in a vehicle and the test subject is the driver of the vehicle.

[0103] Fig. 10 is a drawing showing a structural example of a pulse wave estimation device 1a according to Embodiment 2.

[0104] Regarding the structure of the pulse wave estimation device 1a according to Embodiment 2, the same structural elements as in the pulse wave estimation device 1a according to Embodiment 1 using Fig. 1 described pulse wave estimation device 1 uses the same symbols and redundant explanation is omitted.

[0105] The pulse wave estimation device 1a according to Embodiment 2 differs from the pulse wave estimation device 1 according to Embodiment 1 in that it is equipped with a parameter setting unit 18.

[0106] The parameter setting unit 18 sets segment parameters containing at least information representing the length of the second period based on the subject's pulse wave estimated by the pulse wave estimation unit 16. Specifically, the parameter setting unit 18 calculates the length of the second period based on the subject's pulse wave estimated by the pulse wave estimation unit 16 such that the second period is the time of one cycle of the subject's pulse wave, and sets segment parameters containing information representing the calculated length of the second period. Furthermore, the second period does not necessarily have to coincide with the time of one cycle of the pulse wave of the subject, as long as the parameter setting unit 18 calculates the second period so that the second period is longer than the time of one cycle of the pulse wave of the subject. Furthermore, in addition to the information about the length of the second period, the parameter setting unit 18 also sets information about, for example, the size of the partial overlap of the second period as a segment parameter. The parameter setting unit 18 can set the size of the overlap between pulse wave original signal segments to be variable depending on the length of the pulse wave original signal segment from the perspective of the processing load. The parameter setting unit 18 outputs the set segment parameters to the segment generation unit 15.

[0107] In Embodiment 2, the estimation unit 163 outputs the pulse wave estimation result P(t), which is pulse wave information representing the estimated pulse wave, to the output unit 17 and the parameter setting unit 18.

[0108] Furthermore, in Embodiment 2, the segment generation unit 15 generates pulse wave original signal segments based on the segment parameters and using the second period set by the parameter setting unit 18.

[0109] Specifically, after setting the second period length set by the parameter setting unit 18 to the second period length used to generate the pulse wave original signal segments based on the segment parameters, the segment generation unit 15 generates, for each measurement range ri(k), a plurality of pulse wave original signal segments in which the pulse wave original signal wi(t) for the second period has been partially extracted from the pulse wave original signal wi(t) of the first period, according to the segment generation conditions.

[0110] Furthermore, the information about the length of the second period set by the parameter setting unit 18 is used in the generation of the pulse wave original signal segments by the segment generation unit 15 after the parameter setting unit 18 sets the segment parameters. Therefore, it is possible that the segment parameters have not yet been obtained by the parameter setting unit 18 when the segment generation unit 15 generates the pulse wave original signal segments, for example, immediately after the pulse wave estimation device 1a starts the pulse wave estimation of a subject. In this case, the segment generation unit 15 may generate the pulse wave original signal segments using initial values ​​of the second period that were set in advance by an administrator or the like and stored by the segment generation unit 15. Thereafter, when the parameter setting unit 18 has received information for setting segment parameters, ie, results of pulse wave estimation, the parameter setting unit 18 continues the setting of the segment parameters that were last set among the segment parameters set in the past, or resets the second period to the initial values.

[0111] The operation of the pulse wave estimation device 1a according to Embodiment 2 will be described. Fig. 11 is a flowchart for explaining the operation of the pulse wave estimation device 1a according to Embodiment 2. The pulse wave estimation device 1a repeats, for example, the steps shown in the flowchart of Fig. 11 shown processing when the vehicle is turned on until the vehicle is turned off. In Fig. 11, the concrete operations of steps ST21 to ST24 and steps ST28 to ST29 are similar to the concrete operations of steps ST1 to ST6 of Fig. 7, which have already been explained in embodiment 1, which is why a redundant explanation is omitted.

[0112] When the pulse wave original signal information W(t) is output from the pulse wave original signal extraction unit 14 in step ST24, the segment generation unit 15 determines whether the segment parameters have already been set by the parameter setting unit 18, in other words, whether the segment parameters have been output from the parameter setting unit 18 (step ST25).

[0113] If the segment generation unit 15 determines that the segment parameters have already been set ("YES" in step ST25), the segment generation unit 15 sets the length of the second period set by the parameter setting unit 18 to the length of the second period used to generate the pulse wave original signal segments based on the segment parameters (step ST26).

[0114] If the segment generation unit 15 determines that the segment parameters have not been set yet ("NO" in step ST25), the segment generation unit 15 sets initial values ​​for the second period (step ST27).

[0115] When the pulse wave estimation unit 16 estimates the pulse wave of the subject in step ST 29, the parameter setting unit 18 sets segment parameters including information representing the length of the second period based on the pulse wave of the subject estimated by the pulse wave estimation unit 16 (step ST 30).

[0116] In this way, the pulse wave estimation device 1a calculates the second period based on the subject's pulse wave estimated by the pulse wave estimation unit 16. If segment parameters containing information representing the length of the second period are set, it sets the second period according to the set segment parameters. The pulse wave estimation device 1a then uses the second period set according to the segment parameters to generate a plurality of pulse wave original signal segments in which the pulse wave original signal wi(t) for the second period is partially extracted from the pulse wave original signal wi(t) of the first period. Pulse wave estimation requires detecting recurring pulse wave components of the subject. However, the pulse wave cycle of a subject varies depending on the subject's condition, the type of person, etc. For example, if the subject's condition changes, even if it is the same subject, or the subject changes, the time for one cycle of the subject's pulse wave that the pulse wave estimation device 1a attempts to estimate changes. For example, if the subject's pulse rate is 60 beats per minute (bpm), the time for one cycle of the subject's pulse wave is one second, and if the subject's pulse rate is 100 beats per minute (bpm), the time for one cycle of the subject's pulse wave is 0.6 seconds. The pulse wave estimation device 1a according to Embodiment 2 allows the length of the second period used to generate the pulse wave original signal segments for estimating the subject's pulse wave to be adjusted to a length that takes into account the subject's estimated pulse wave cycle. This allows the pulse wave estimation device 1a to estimate the subject's pulse wave with better accuracy than estimating the subject's pulse wave from pulse wave original signal segments generated using a second period set without considering the subject's pulse wave cycle.

[0117] In the above embodiment 2, it is assumed that once the pulse wave estimation device 1a starts estimating the pulse wave of the subject, it continues estimating the pulse wave of the subject for a certain period to the extent that the first period is repeated a plurality of times, and the parameter setting unit 18 calculates the second period based on the pulse wave of the subject estimated by the estimation unit 163 in the previous first period, but this is only an example.For example, the pulse wave estimation result P(t) of a subject estimated by the pulse wave estimation device 1a in the past before the pulse wave estimation device 1a begins estimating the subject's pulse wave may be stored in a storage unit, and the parameter setting unit 18 may calculate the second period based on the pulse wave estimation result P(t) stored in the storage unit. For example, the subject may be able to register information about the subject's average pulse wave in the storage unit, and the parameter setting unit 18 may be able to calculate the second period based on the information about the subject's average pulse wave registered in the storage unit. For example, the parameter setting unit 18 may also calculate the second period from the pulse wave estimation result P(t) obtained by referring to the storage unit or the average pulse wave of the subject and the pulse wave of the subject estimated by the estimation unit 163.

[0118] The hardware structure of the pulse wave estimation device 1a according to Embodiment 2 is similar to the hardware structure of the pulse wave estimation device 1 described in Embodiment 1 based on Fig. 9A and Fig. 9B, so that their presentation is omitted. In Embodiment 2, the functions of the image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16, the output unit 17, and the parameter setting unit 18 are realized by the processing circuit 101.That is, the pulse wave estimation device 1a includes the processing circuit 101 for generating a plurality of pulse wave original signal segments in which a pulse wave original signal wi(t) representing luminance changes in the plurality of measurement areas ri(k) set for a skin area of ​​a subject in an image acquired in the first period has been partially extracted, using techniques such as independent component analysis or principal component analysis, separating pulse wave components from the pulse wave original signal segments, and controlling the estimation of a subject's pulse wave based on the separated pulse wave components.

[0119] The processing circuit 101 reads and executes the program stored in the memory 105 to perform the functions of the image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave original signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16, the output unit 17, and the parameter setting unit 18. That is, the pulse wave estimation device 1a includes the memory 105 to store therein a program for executing steps ST21 to ST30 of Fig. 11 are finally executed upon execution by the processing circuit 101. Furthermore, it can also be said that the program stored in the memory 105 causes the computer to execute the procedures or processes of the processings of the image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave original signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16, the output unit 17, and the parameter setting unit 18. A memory unit not shown is formed, for example, by the memory 105. Furthermore, the pulse wave estimation device 1a comprises an input interface device 102 and an output interface device 103 for wired or wireless communication with devices such as the imaging device 3.

[0120] Furthermore, in the above embodiment 2, the test subject is the driver of a vehicle, but this is only an example. Test subjects can also be passengers other than the driver of the vehicle.

[0121] Furthermore, in the above embodiment 2, the pulse wave estimation device 1a is an in-vehicle device, and the map image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave original signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16, the output unit 17, and the parameter setting unit 18 are provided in the in-vehicle device. In addition, some of the image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16, the output unit 17, and the parameter setting unit 18 may be installed in the in-vehicle device, while others are provided in a server connected to the in-vehicle device via a network, and the system may be composed of the in-vehicle device and the server. Furthermore, the image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16, the output unit 17, and the parameter setting unit 18 may all be provided on the server.

[0122] Furthermore, the pulse wave estimation device 1a according to Embodiment 2 above is not limited to in-vehicle devices installed in vehicles, but can also be used for household appliances, for example. Furthermore, the test subject can be various people, not just vehicle passengers.

[0123] As described above, in addition to the structure of the pulse wave estimation device 1 according to Embodiment 1, the pulse wave estimation device 1a according to Embodiment 2 is configured to include a parameter setting unit 18 that calculates the second period based on the pulse wave of a person (a test subject) estimated by the pulse wave estimation unit 16 and sets segment parameters containing information representing the length of this second period, and the segment generation unit 15 sets the second period according to the segment parameters. Therefore, the pulse wave estimation device 1a can estimate a person's pulse wave with better accuracy than in the case of estimating a person's pulse wave from pulse wave original signal segments generated using a second period set without considering the person's pulse wave cycle.

[0124] Specifically, in the pulse wave estimation device 1a, the parameter setting unit 18 calculates the second period such that the second period is the time of one cycle of the pulse wave of the subject (the test subject) estimated by the pulse wave estimation unit 16, and sets segment parameters containing information representing the length of this second period. The segment generation unit 15 then sets the second period according to the segment parameters. Therefore, the pulse wave estimation device 1a can estimate a subject's pulse wave with better accuracy than in the case of estimating a subject's pulse wave from pulse wave original signal segments generated using a second period set without considering the subject's pulse wave cycle. Embodiment 3.

[0125] In Embodiment 1, the pulse wave estimation device uses the generated plurality of pulse wave original signal segments as they are to estimate the pulse wave of the subject.

[0126] Embodiment 3 describes an embodiment in which the pulse waves of a subject are estimated after the plurality of pulse wave original signal segments are weighted.

[0127] Further, in the following Embodiment 3, as in Embodiment 1, it is assumed that the pulse wave estimation device is installed in a vehicle and the test subject is the driver of the vehicle.

[0128] Fig. 12 is a drawing showing a structural example of a pulse wave estimation device 1b according to Embodiment 3. Regarding the structure of the pulse wave estimation device 1b according to Embodiment 3, the same structural elements as in the pulse wave estimation device 1b according to Embodiment 1 using Fig. 1 described pulse wave estimation device 1 uses the same symbols and redundant explanation is omitted. The pulse wave estimation device 1b according to Embodiment 3 differs from the pulse wave estimation device 1 according to Embodiment 1 in that it is equipped with a weighting coefficient calculation unit 19.

[0129] The weighting coefficient calculation unit 19 calculates weighting coefficients (hereinafter “segment weights”) for each of the plurality of pulse wave original signal segments generated per measurement range ri(k) by the segment generation unit 15. Specifically, the weighting coefficient calculation unit 19 calculates segment weights for each of the plurality of pulse wave original signal segments based on the magnitude of signal fluctuation in the pulse wave original signal wi(t) from which the pulse wave original signal segments were extracted. Furthermore, in Embodiment 3, the segment generation unit 15 outputs the generated information about the plurality of pulse wave original signal segments ri(k) of each measurement range and the pulse wave original signal information W(t) generated by the pulse wave original signal extraction unit 14 to the weighting coefficient calculation unit 19.

[0130] The weighting coefficient calculation unit 19 determines, for example, whether the pulse wave original signal wi(t) in the first-period time series from which the pulse wave original signal segments were extracted has fluctuated by more than a preset threshold (hereinafter, "fluctuation determination threshold"). If it is determined that the pulse wave original signal wi(t) has fluctuated by more than the fluctuation determination threshold, the weighting coefficient calculation unit 19 calculates the segment weight "0" for the pulse wave original signal segments extracted within the pulse wave original signal wi(t) in the first-period time series from the part of the pulse wave original signal in which the fluctuation in question occurred.It can be said that the signal of the part within the pulse wave original signal wi(t) that has fluctuated by more than the fluctuation determination threshold is a signal containing a large amount of noise. The weighting coefficient calculation unit 19 sets the segment weight "1" for the pulse wave original signal segments extracted from the part of the pulse wave original signal wi(t) that is not the part where the above fluctuation occurred.

[0131] On the other hand, when the weighting coefficient calculation unit 19 determines that the pulse wave original signal wi(t) in time series of the first period from which the pulse wave original signal segments were extracted has not fluctuated by more than the fluctuation determination threshold, it calculates the segment weight “1” for all pulse wave original signal segments extracted from the pulse wave original signal wi(t) in time series of the first period without such fluctuations.

[0132] The weighting coefficient calculation unit 19 outputs information about the segment weights (hereinafter "segment weight information") calculated for each of the plurality of pulse wave original signal segments generated for each measurement range ri(k) by the segment generation unit 15 to the pulse wave estimation unit 16. The segment weight information is information that can identify the measurement regions ri(k), information that can identify the pulse wave original signal segments, and information about the segment weights of these pulse wave original signal segments.

[0133] The segment weights calculated by the weighting coefficient calculation unit 19 are an indicator for the pulse wave estimation unit 16 to determine whether the pulse wave original signal segments generated by the segment generation unit 15 should be used in estimating the pulse wave of a subject. The pulse wave estimation unit 16 ensures that pulse wave original signal segments with the segment weight of "0" are not used to estimate the subject's pulse wave. Specifically, the signal separation unit 161 of the pulse wave estimation unit 16 discards the pulse wave original signal segments assigned the segment weight of "0," analyzes the multiple principal components based on the remaining multiple pulse wave original signal segments using a general signal separation technique such as PCA or ICA, and generates separation signals representing the analyzed multiple principal components. In this way, the pulse wave estimation unit 16 in Embodiment 3 estimates the pulse wave of a subject based on the pulse wave original signal segments and the segment weights.

[0134] Fig. 13 is a drawing for explaining the pulse wave original signal segments used by the pulse wave estimation unit 1b for estimating the pulse wave of a subject in Embodiment 3. Fig. 13 is a diagram showing, on the time axis, the luminance values ​​represented by the pulse wave original signal wi(t) in first-period time series extracted from a specific measurement range ri(k) of the plural measurement ranges ri(k).

[0135] In Fig. 13, within the pulse wave original signal wi(t) in time series of the first period, the pulse wave original signal wi(t) of the area enclosed by a dashed line shows a large signal fluctuation.

[0136] The weighting coefficient calculation unit 19 calculates a segment weight of “0” for the pulse wave original signal segments within the plurality of pulse wave original signal segments generated by the segment generation unit 15 for a specific measurement range ri(k) extracted from the range of the pulse wave original signal wi(t) enclosed by the dashed line.

[0137] The weighting coefficient calculation unit 19 calculates a segment weight of "1" for the pulse wave original signal segments within the plurality of pulse wave original signal segments generated by the segment generation unit 15 for a specific measurement range ri(k) that were extracted outside the range of the pulse wave original signal wi(t) enclosed by the dashed line. As a result, the pulse wave estimation unit 16 does not use the pulse wave original signal segments extracted from the range of the pulse wave original signal wi(t) enclosed by the dashed line and whose segment weight is "0" to estimate the subject's pulse wave.

[0138] The operation of the pulse wave estimation device 1b according to Embodiment 3 will be described. Fig. 14 is a flowchart for explaining the operation of the pulse wave estimation device 1b according to Embodiment 3. The pulse wave estimation device 1b repeats, for example, the steps shown in the flowchart of Fig. 14 processing when the vehicle is turned on until the vehicle is turned off. In Fig. 14, the concrete operations of steps ST31 to ST35 are similar to the concrete operations of steps ST1 to ST5 of Fig. 7, which have already been explained in embodiment 1, which is why a redundant explanation is omitted.

[0139] The weighting coefficient calculation unit 19 calculates segment weights for each of the plurality of pulse wave original signal segments generated by the segment generation unit 15 in step ST35 (step ST36). More specifically, the weighting coefficient calculation unit 19 calculates segment weights for each of the plurality of pulse wave original signal segments based on the magnitude of a signal fluctuation in the pulse wave original signal wi(t) from which the pulse wave original signal segments were extracted. The weighting coefficient calculation unit 19 outputs the segment weighting information to the pulse wave estimation unit 16.

[0140] When the pulse wave estimation unit 16 is notified by the segment generation unit 15 that the trajectory matrix has been stacked in step ST35, it performs pulse wave estimation processing to estimate the subject's pulse wave based on the plurality of pulse wave original signal segments generated per measurement range ri(k). At this time, the pulse wave estimation unit 16 ensures that the pulse wave original signal segments assigned the segment weight "0" based on the segment weight information output from the weight coefficient calculation unit are not used to estimate the subject's pulse wave (step ST37). Furthermore, the pulse wave estimation unit 16 may receive notification that the trajectory matrix has been stacked, for example, via the weight coefficient calculation unit 19. More specifically, the signal separation unit 161 discards when generating the separation signals (see step ST11 of Fig. 8) the pulse wave original signal segments with the segment weight “0”, analyzes the multiple principal components based on the remaining multiple pulse wave original signal segments by means of general signal separation technique such as PCA or ICA, and generates separation signals representing the analyzed multiple principal components.

[0141] In this way, the pulse wave estimation unit 1b calculates segment weights for each of the plurality of pulse wave original signal segments and estimates the pulse wave of a subject based on the pulse wave original signal segments and segment weights. Specifically, the pulse wave estimation device 1b calculates segment weights for each of the plurality of pulse wave original signal segments based on the magnitude of the signal fluctuation wi(t) in the pulse wave original signal from which the pulse wave original signal segments were extracted. As a result, when the pulse wave original signal wi(t) extracted from the multiple measurement ranges ri(k) contains many noise components, the pulse wave estimation device 1b can estimate the pulse wave of a subject by excluding them. Therefore, the pulse wave estimation device 1b can estimate the pulse wave of a subject with better accuracy than when ignoring whether the pulse wave original signal wi(t) extracted from the multiple measurement ranges ri(k) is a signal containing many noise components.

[0142] The hardware structure of the pulse wave estimation device 1b according to Embodiment 3 is similar to the hardware structure of the pulse wave estimation device 1 described in Embodiment 1 based on Fig. 9A and Fig. 9B, so that their presentation is omitted. In Embodiment 3, the functions of the image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave original signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16, the output unit 17, and the weighting coefficient calculation unit 19 are realized by the processing circuit 101.That is, the pulse wave estimation device 1b includes the processing circuit 101 for generating a plurality of pulse wave original signal segments in which a pulse wave original signal wi(t) representing luminance changes in the plurality of measurement areas ri(k) set for a skin area of ​​a subject in an image acquired in the first period has been partially extracted, using techniques such as independent component analysis or principal component analysis, separating pulse wave components from the pulse wave original signal segments, and controlling the estimation of a subject's pulse wave based on the separated pulse wave components.

[0143] The processing circuit 101 reads and executes the program stored in the memory 105 to perform the functions of the image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave original signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16, the output unit 17, and the weighting coefficient calculation unit 19. That is, the pulse wave estimation device 1b includes the memory 105 to store therein a program for executing steps ST31 to ST37 of Fig. 14 are finally executed upon execution by the processing circuit 101. Furthermore, it can also be said that the program stored in the memory 105 causes the computer to execute the procedures or processes of the processings of the map image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave original signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16, the output unit 17, and the weighting coefficient calculation unit 19. A memory unit not shown is formed, for example, by the memory 105. Furthermore, the pulse wave estimation device 1b comprises an input interface device 102 and an output interface device 103 for wired or wireless communication with devices such as the imaging device 3.

[0144] Furthermore, in the above embodiment 3, the test subject is the driver of a vehicle, but this is only an example. Test subjects can also be passengers other than the driver of the vehicle.

[0145] Furthermore, in the above embodiment 3, the pulse wave estimation device 1b is an in-vehicle device, and the map image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave original signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16, the output unit 17, and the weighting coefficient calculation unit 19 are provided in the in-vehicle device. In addition, some of the image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16, the output unit 17, and the weighting coefficient calculation unit 19 may be installed in the in-vehicle device, while others are provided in a server connected to the in-vehicle device via a network, and the system may be composed of the in-vehicle device and the server. Furthermore, the image acquisition unit 11, the skin area detection unit 12, the measurement area setting unit 13, the pulse wave original signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16, the output unit 17, and the weighting coefficient calculation unit 19 may also all be provided on the server.

[0146] Furthermore, the pulse wave estimation device 1b according to Embodiment 3 above is not limited to in-vehicle devices installed in vehicles, but can also be used for household appliances, for example. Furthermore, the test subject can be various people, not just vehicle passengers.

[0147] As described above, in addition to the structure of the pulse wave estimation device 1 according to Embodiment 1, the pulse wave estimation device 1b according to Embodiment 3 is configured to include a weighting coefficient calculation unit 19 for calculating weighting coefficients (segment weights) for each of the plurality of pulse wave original signal segments, and the pulse wave estimation unit 16 estimates the pulse wave of a person (a subject) based on the pulse wave original signal segments and the weighting coefficients (segment weights). Therefore, the pulse wave estimation device 1b can estimate the pulse wave of a person with better accuracy than when not considering whether the pulse wave original signal wi(t) extracted from the plurality of measurement areas ri(k) is a signal containing many noise components.

[0148] Specifically, in the pulse wave estimation device 1b, the weighting coefficient calculation unit 19 calculates weighting coefficients (segment weights) for each of the plurality of pulse wave original signal segments based on the magnitude of signal fluctuation in the pulse wave original signal wi(t) from which the pulse wave original signal segments were extracted. Therefore, the pulse wave estimation device 1b can estimate a person's pulse wave with better accuracy than when not considering whether the pulse wave original signal wi(t) extracted from the plurality of measurement areas ri(k) is a signal containing many noise components.

[0149] In the above embodiments 1 to 3, the pulse wave estimation device 1, 1a, 1b may have the function of setting weighting coefficients (hereinafter "component weights") for a plurality of separation signals representing a plurality of principal components estimated using known general signal separation techniques, and restoring the pulse wave original signal wi(t) per measurement range ri(k) based on the plurality of separation signals and the set component weight per separation signal.

[0150] Fig. 15 is a drawing showing a structural example of a pulse wave estimation unit 16 in Embodiments 1 to 3 in the case where the pulse wave estimation device 1, 1a, 1b has the function of setting component weights for a plurality of separation signals representing a plurality of principal components estimated using a known general signal separation technique, and restoring the pulse wave original signal wi(t) per measurement range ri(k) based on the plurality of separation signals and the set component weight per separation signal.

[0151] When the pulse wave estimation device 1, 1a, 1b has the above function, the pulse wave estimation device 1, 1a, 1b should be provided with a pulse wave estimation unit 16a as shown in Fig. 15, instead of the pulse wave estimation unit 16 in the configuration example of the pulse wave estimation device 1, 1a, 1b constructed using Fig. 1, Fig. 10 and 12 respectively in the above embodiments 1 to 3.

[0152] The restoration unit 162 of the pulse wave estimation unit 16a includes a component weight setting unit 1621 that sets a component weight per separation signal output from the signal separation unit 161. The restoration unit 162 restores the pulse wave original signal wi(t) per measurement range ri(k) based on the plurality of separation signals output from the signal separation unit 161 and the component weight set by the component weight setting unit 1621 per separation signal. The estimation unit 163 estimates the pulse wave of the subject based on the generated pulse wave original signal wi(t) restored by the restoration unit 162 based on the plurality of separation signals and the component weight per separation signal.

[0153] The following explains exemplary methods for setting the component weights by the component weight setting unit 1621.

[0154] The component weight setting unit 1621 sets a component weight per separation signal based on, for example, the similarity of the frequency characteristic between the plurality of separation signals. Specifically, the component weight setting unit 1621 first calculates the peak frequency in the frequency power spectrum of each separation signal by performing a fast Fourier transform or the like on each separation signal. Subsequently, the component weight setting unit 1621 pairs the adjacent separation signals with respect to the plurality of separation signals representing a plurality of principal components and calculates the peak frequency difference of the adjacent separation signals (hereinafter, "peak difference"). Furthermore, the term "adjacent principal components" means separation signals that are consecutive when the separation signals are arranged in order of information content. The component weight setting unit 1621 then determines whether the calculated peak difference is greater than a preset threshold (hereinafter, "peak difference determination threshold").The component weight setting unit 1621 sets the component weight of the separation signal for which the peak difference is calculated to “0” when the peak difference is larger than the threshold value for peak difference determination, and sets the component weight to “1” when the peak difference is smaller than the threshold value for peak difference determination.

[0155] When the component weight is "0," the similarity of the frequency characteristics between the separation signals is low; when the component weight is "1," the similarity of the frequency characteristics between the separation signals is high. Since pulse wave components have the characteristic of repeating at a certain period, it can be assumed that the high similarity of the frequency characteristics means that the separation signal with this frequency characteristic contains many pulse wave components, in other words, that it is a separation signal typical of pulse wave components. The component weight setting unit 1621 assigns larger component weights to the separation signals typical of pulse wave components, so that the estimation unit 163 can estimate the subject's pulse wave based on the separation signals typical of pulse wave components.

[0156] To give a concrete example, it is assumed that the plurality of separation signals output from the signal separation unit 161 are C separation signals representing the first principal component c1, ... to the C-th principal component cC, as shown in Fig. 5 and Fig. 6 in Embodiment 1. Here, the separation signal representing the first principal component c1 is the first separation signal, the separation signal representing the second principal component c2, the second separation signal, the separation signal representing the third principal component c3, the third separation signal, the separation signal representing the fourth principal component c4, the fourth separation signal, the separation signal representing the fifth principal component c5, the fifth separation signal, ..., the separation signal representing the (C -1)-th principal component c(C-1), the (C-1)-th separation signal, and the signal representing the C-th principal component cC is the C-th separation signal. In this case, the component weight setting unit 1621 pairs the first and second separation signals, calculates the peak difference between the peak frequency of the first separation signal and the peak frequency of the second separation signal, and determines whether the calculated peak difference is greater than the peak difference determination threshold. The component weight setting unit 1621 sets the component weights of the first and second separation signals to "0" when the peak difference is greater than the peak difference determination threshold, and sets the component weights of the first and second separation signals to "1" when the peak difference is smaller than the peak difference determination threshold. Furthermore, the component weight setting unit 1621 forms a pair of the third and fourth separation signals, calculates the peak difference between the peak frequency of the third separation signal and the peak frequency of the fourth separation signal, and determines whether the calculated peak difference is greater than the threshold for peak difference determination. The component weight setting unit 1621 sets the component weights of the third and fourth separation signals to "0" if the peak difference is greater than the threshold for peak difference determination, and sets the component weights of the third and fourth separation signals to "1" if the peak difference is smaller than the threshold for peak difference determination. The component weight setting unit 1621 also performs the above-described processing for the pair of the fifth and sixth separation signals, the pair of the seventh and eighth separation signals, etc.the pair of the (C-1)th and Cth separation signals and sets the component weights for all separation signals.

[0157] The restoration unit 162 discards, from the plurality of separation signals output from the signal separation unit 161, the separation signals for which the component weight setting unit 1621 has set the component weight “0”, and restores the pulse wave original signal wi(t) per measurement range ri(k) based only on the separation signals for which the component weight setting unit 1621 has set the component weight “1”. The estimation unit 163 then estimates the subject's pulse wave based on the pulse wave original signal wi(t) that was restored by the restoration unit 162 based only on the separation signals with the component weight "1." Since the estimation unit 163 can estimate the subject's pulse wave based on the separation signals typical of pulse wave components, the accuracy of the subject's pulse wave estimation can be improved.

[0158] Furthermore, in the above concrete example, the component weight setting unit 1621 forms pairs of adjacent separation signals and calculates the peak difference of the peak frequencies of the adjacent separation signals, but this is only an example. For example, the component weight setting unit 1621 may calculate the differences of all pairs and use the weighted average as the component weight. For example, the component weight setting unit 1621 calculates, for a specific separation signal (reference separation signal) among the C separation signals, the difference from the peak frequency of the other (C-1) separation signals, and determines the weighted average of the difference corresponding to the calculated (C-1) differences as the difference value of the reference separation signal. The component weight setting unit 1621 then sets the component weight for the reference separation signal to "0" when the difference value of the reference separation signal is greater than the threshold value, and the component weight for the reference separation signal to "1" when the difference value is equal to or less than the threshold value. The component weight setting unit 1621 performs the above processing with all separation signals as reference separation signals.The component weight setting unit 1621 may also use values ​​corresponding to the size of the weighted average as component weights.

[0159] For example, the component weight setting unit 1621 may also calculate correlation coefficients of the projection coefficients between the plurality of separation signals when generating the plurality of separation signals based on projection coefficients assigned to the separation signals, and set the component weight per separation signal based on the correlation coefficients.

[0160] Specifically, the component weight setting unit 1621 first generates, for each separation signal per measurement range ri(k), information expressing projection coefficients in time series associated with the plurality of pulse wave original signal segments corresponding to this measurement range ri(k) (hereinafter, "time series projection coefficient information"). The component weight setting unit 1621 generates projection coefficient information in time series per separation signal for the number (N) of measurement ranges ri(k). That is, given C separation signals, the component weight setting unit 1621 generates C × N projection coefficient information in time series. Subsequently, the component weight setting unit 1621 determines the correlation relationship of the generated time-series projection coefficient information for each separation signal and calculates the correlation coefficient. Specifically, for a specific separation signal, the component weight setting unit 1621 pairs all the paths for the generated N time-series projection coefficient information and calculates the correlation coefficient for each pair of time-series projection coefficient information. Furthermore, the correlation coefficient indicates the degree of phase matching of the time-series projection coefficient information. The component weight setting unit 1621 uses the average of the correlation coefficients calculated for each pair as the component weights for the separation signals.

[0161] Here, the pulse wave estimation device 1, 1a, 1b assumes that in-phase signal components are signals typical of pulse wave components, and that when the changes of the projection coefficients in time series between the measurement ranges ri(k) are similar, in other words, when the phases of the projection coefficient information in time series between the measurement ranges ri(k) are rectified, it is assumed that the separation signal to which such a projection coefficient is assigned for each pulse wave original signal segment in each measurement range ri(k) is a separation signal typical of pulse wave components having many pulse wave components.The component weight setting unit 1621 assigns a larger weight coefficient to the separation signals typical of pulse wave components, so that the estimation unit 163 can estimate the pulse wave of the subject based on the separation signals typical of pulse wave components.

[0162] To give a concrete example, it is assumed that the plurality of separation signals output from the signal separation unit 161 are C separation signals representing the first principal component c1, ... to the C-th principal component cC, as shown in Fig. 5 and Fig. 6 in Embodiment 1. Here, the separation signal representing the first principal component c1 is the first separation signal, the separation signal representing the second principal component c2, the second separation signal, the separation signal representing the third principal component c3, the third separation signal, the separation signal representing the fourth principal component c4, the fourth separation signal, the separation signal representing the fifth principal component c5, the fifth separation signal, ..., the separation signal representing the (C -1)-th principal component c(C-1), the (C-1)-th separation signal, and the signal representing the C-th principal component cC is the C-th separation signal. In this case, the component weight setting unit 1621 first generates projection coefficient information in time series for each separation signal per measurement range ri(k) (first measurement range, second measurement range, third measurement range, and fourth measurement range).

[0163] Fig. 16 is a drawing showing an image of an example of projection coefficient information in time series generated by the component weight setting unit 1621. Fig. 16 shows, by way of example, the time-series projection coefficient information generated by the component weight setting unit 1621 for the first separation signal. The component weight setting unit 1621 generates time-series projection coefficient information for the first separation signal for each of the first measurement range, the second measurement range, the third measurement range, and the fourth measurement range. Furthermore, Fig. 16, the projection coefficient information generated for the first separation signal is exemplified in time series, but the component weight setting unit 1621 also generates projection coefficient information in time series per measurement range ri(k) for the second separation signal to the C-th separation signal.

[0164] Subsequently, the component weight setting unit 1621 first forms, for example, in the first separation signal, pairs of all the paths for the projection coefficient information in time series of the first measurement range (in Fig. 16 designated 1601), the projection coefficient information in time series of the second measuring range (in Fig. 16 denoted by 1602), the projection coefficient information in time series of the third measurement range (in Fig. 16 designated 1603) and the projection coefficient information in time series of the fourth measurement range (in Fig. 16 marked 1604). Here, the component weight setting unit 1621 forms a pair of the projection coefficient information in time series of the first measurement range and the projection coefficient information in time series of the second measurement range (first pair), a pair of the projection coefficient information in time series of the first measurement range and the projection coefficient information in time series of the third measurement range (second pair), a pair of the projection coefficient information in time series of the first measurement range and the projection coefficient information in time series of the fourth measurement range (third pair), a pair of the projection coefficient information in time series of the second measurement range and the projection coefficient information in time series of the third measurement range (fourth pair),a pair of the projection coefficient information in time series of the second measurement range and the projection coefficient information in time series of the fourth measurement range (fifth pair) and a pair of the projection coefficient information in time series of the third measurement range and the projection coefficient information in time series of the fourth measurement range (sixth pair).

[0165] The component weight setting unit 1621 then calculates the correlation coefficient of the first pair, the correlation coefficient of the second pair, the correlation coefficient of the third pair, the correlation coefficient of the fourth pair, the correlation coefficient of the fifth pair, and the correlation coefficient of the sixth pair. The component weight setting unit 1621 takes the average of the calculated correlation coefficients of the first pair, the correlation coefficients of the second pair, the correlation coefficients of the third pair, the correlation coefficients of the fourth pair, the correlation coefficients of the fifth pair, and the correlation coefficients of the fifth pair as the component weight of the first separation signal representing the first principal component c1.

[0166] The component weight setting unit 1621 similarly forms all pairs of all the ranges of the projection coefficient information in the time series of the first measurement range, the projection coefficient information in the time series of the second measurement range, the projection coefficient information in the time series of the third measurement range, and the projection coefficient information in the time series of the fourth measurement range for the second separation signal to the Cth separation signal, and determines the correlation coefficients for each formed pair to calculate the correlation coefficients. The component weight setting unit 1621 then uses the average of the calculated correlation coefficients for each pair as component weights.

[0167] The restoration unit 162 restores the pulse wave original signal wi(t) per measurement range ri(k) by weighting the plurality of separation signals output from the signal separation unit 161 according to the component weights set by the component weight setting unit 1621. For example, the restoration unit 162 determines (component weight of the first separation signal × [p1-1, p1-2, ..., p1-M] + component weight of the second separation signal × [p1-1, p1-2, ..., p1-M] + component weight of the third separation signal × [p1-1, p1-2, ..., p1-M] + ... + component weight of the C-th separation signal × [p1-2, ..., p1-M]) as the restored pulse wave original signal (t) of the first measurement range.

[0168] The estimation unit 163 then estimates the subject's pulse wave based on the pulse wave original signal wi(t) recovered by the recovery unit 162 by weighting according to the component weights. Since the estimation unit 163 can estimate the subject's pulse wave based on the separation signals typical of pulse wave components, the accuracy of the subject's pulse wave estimation can be improved.

[0169] Furthermore, in the above example, the component weight setting unit 1621 forms pairs of all the lengths of the projection coefficient information in time series for each separation signal, but this is only an example. The component weight setting unit 1621 may, for example, also form a pair of two of the projection coefficient information in time series from the multiple pieces of projection coefficient information in time series. In the above concrete example, this would mean that the component weight setting unit 1621 could, for example, also form two pairs, namely a pair of the projection coefficient information in time series of the first measurement range and the projection coefficient information in time series of the second measurement range (seventh pair), and a pair of the projection coefficient information in time series of the third measurement range and the projection coefficient information in time series of the fourth measurement range (eighth pair).In this case, the component weight setting unit 1621 calculates the correlation coefficient of the seventh pair and the correlation coefficient of the eighth pair, and the calculated average of the correlation coefficient of the sixth pair and the correlation coefficient of the seventh pair is used as the component weight of the first separation signal representing the first principal component c1.

[0170] Furthermore, in the above example, the component weight setting unit 1621 uses the average of the calculated correlation coefficients as the component weight of the separation signal, but this is only an example. The component weight setting unit 1621 may also use the variance value of the calculated correlation coefficients as the component weight of the separation signal, for example.

[0171] The operation of the pulse wave estimation unit 16a in the above embodiments 1 to 3 will be explained with reference to Fig. 17 explains the case where the pulse wave estimation device 1, 1a, 1b has the function of setting component weights for a plurality of separation signals representing a plurality of principal components estimated using a known general signal separation technique, and restoring the pulse wave original signal wi(t) per measurement range ri(k) based on the plurality of separation signals and the set component weight per separation signal. In the case where, in Embodiments 1 to 3, the pulse wave estimation device 1, 1a, 1b has the function of setting component weights for a plurality of separation signals representing a plurality of principal components estimated using known general signal separation techniques, and restoring the pulse wave original signal wi(t) per measurement range ri(k) based on the plurality of separation signals and the set component weight per separation signal, the details of the pulse wave estimation processing of step ST6 in Fig. 7, step ST29 in Fig. 11 and step ST37 in Fig. 15, as described in embodiments 1-3, which is shown in the flowchart of Fig. 17 processing shown.

[0172] Furthermore, the concrete processes of steps ST11 and ST13 of Fig. 17 respectively those of steps ST11 and ST13 of Fig.8, which have already been explained, so a redundant explanation is omitted.

[0173] The component weight setting unit 1621 of the pulse wave estimation unit 16a sets the component weight per separation signal output from the signal separation unit 161 in step ST11 (step ST12-1).

[0174] The restoration unit 162 restores the pulse wave original signal wi(t) per measurement range ri(k) based on the plurality of separation signals output by the signal separation unit 161 in step ST1 and the component weight per separation signal set by the component weight setting unit 1621 in step ST12-1 (step ST12-2).

[0175] Further, in step ST13, the estimation unit 163 estimates the pulse wave of the subject based on the pulse wave original signal wi(t) restored by the restoration unit 162 based on the plurality of separation signals and the component weight per separation signal.

[0176] In the pulse wave estimation device 1 according to Embodiment 1, the pulse wave estimation device 1a according to Embodiment 2, and the pulse wave estimation device 1b according to Embodiment 3, respectively, the restoration unit 162a is provided with a component weight setting unit 1621 for setting a weighting coefficient (component weight) for each separation signal. By the restoration unit 162a restoring the pulse wave original signal wi(t) per measurement range ri(k) based on the plurality of separation signals and the weighting coefficient (component weight) for each separation signal, the pulse wave estimation device 1, 1a, 1b can estimate the pulse wave of a subject based on the separation signals typical of pulse wave components, thereby improving the accuracy of estimating the subject's pulse wave.

[0177] Furthermore, all embodiments can be freely combined, or components of the individual embodiments can be arbitrarily converted, or components of the individual embodiments can also be omitted.

[0178] In the following, various aspects of the present disclosure are summarized as additions. (Addition 1)

[0179] Pulse wave assessment device comprising an image acquisition unit that acquires an image depicting a person, a skin area detection unit that detects a skin area of ​​the person based on the image, a measurement area setting unit that sets a plurality of measurement areas in an area corresponding to the skin area on the image image to extract therefrom a pulse wave origin signal representing luminance changes in time series in a first period, a pulse wave original signal extraction unit that extracts a pulse wave original signal for each measurement area based on the luminance changes in the first period in the respective measurement area, a segment generation unit that generates a plurality of pulse wave original signal segments for each measurement area based on the pulse wave original signal extracted from the respective measurement area in accordance with segment generation conditions, said segments being signals in which a pulse wave original signal for a second period has been partially extracted from the pulse wave original signal in the first period, and a pulse wave estimation unit that estimates the person's pulse wave based on the plurality of pulse wave original signal segments generated per measurement area. (Addition 2)

[0180] Pulse wave estimation device according to addition 1, characterized in that the segment generation conditions include that the second period is shorter than the first period, with respect to the plurality of pulse wave original signal segments in a specific measurement range, a specific pulse wave original signal segment is a signal that does not completely coincide with other pulse wave original signal segments on the time axis, with regard to the plurality of pulse wave original signal segments in a specific measurement range, a specific pulse wave original signal segment is a signal that partially overlaps with other pulse wave original signal segments on the time axis, in each measurement range the duration of the partial overlap of a specific pulse wave original signal segment with other pulse wave original signal segments on the time axis is the same length, and in each measuring range, the lengths of the multiple pulse wave original signal segments on the time axis are identical to each other. (Addition 3)

[0181] Pulse wave estimation device according to addition 1 or addition 2, characterized in that a parameter setting unit is provided which calculates the second period on the basis of the pulse wave of a person estimated by the pulse wave estimation unit and sets segment parameters which contain at least information representing the length of the second period, wherein the segment generation unit sets the second period according to the segment parameters. (Addition 4)

[0182] A pulse wave estimation device according to claim 3, characterized in that the parameter setting unit calculates the second period such that the second period is the time of one cycle of the pulse wave of the person estimated by the pulse wave estimation unit. (Addition 5)

[0183] Pulse wave estimation device according to one of the additions 1 to 4, characterized in that a weighting coefficient calculation unit is provided to calculate weighting coefficients for each of the plurality of pulse wave original signal segments, and the pulse wave estimation unit estimates the pulse wave of the person based on the pulse wave original signal segments and the weighting coefficients. (Addition 6)

[0184] A pulse wave estimation device according to Addition 5, characterized in that the weighting coefficient calculation unit calculates weighting coefficients for each of the plurality of pulse wave original signal segments based on the magnitude of a signal fluctuation in the pulse wave original signal from which the pulse wave original signal segments were extracted. (Addition 7)

[0185] Pulse wave estimation device according to one of the additions 1 to 6, characterized in that the pulse wave assessment unit a signal separation unit that generates separation signals representing a plurality of signal components based on the plurality of pulse wave original signal segments, a recovery unit that recovers a pulse wave original signal per measurement range based on the plurality of separation signals, and an estimation unit that estimates the person's pulse wave based on the restored pulse wave original signal per measurement area, includes. (Addition 8)

[0186] Pulse wave estimation device according to Addition 7, characterized in that the recovery unit a component weight setting unit that sets weighting coefficients per separation signal, and restores the pulse wave original signal per measurement range based on the plurality of separation signals and the weighting coefficients per separation signal. (Addition 9)

[0187] Pulse wave estimation device according to Addition 8, characterized in that the component weight setting unit sets the weighting coefficients for each separation signal based on the similarity of the frequency characteristics between the plurality of separation signals. (Addition 10)

[0188] Pulse wave estimation device according to Addition 8, characterized in that the component weight setting unit when generating the plurality of separation signals, based on projection coefficients assigned to the separation signals, correlation coefficients of the projection coefficients between the plurality of separation signals are calculated, and the weighting coefficients for each separation signal are set based on the correlation coefficients. (Addition 11)

[0189] Pulse wave assessment procedure, comprehensive a step in which an image acquisition unit acquires an image depicting a person, a step in which a skin area detection unit detects a skin area of ​​the person based on the image, a step in which a measurement area setting unit sets a plurality of measurement areas in an area corresponding to the skin area on the image image to extract therefrom a pulse wave origin signal representing luminance changes in time series in a first period, a step in which a pulse wave original signal extraction unit extracts a pulse wave original signal for each measurement area based on the luminance changes in the first period in the measurement area in question, a step in which a segment generation unit generates, for each measurement area, a plurality of pulse wave original signal segments based on the pulse wave original signal extracted from the respective measurement area in accordance with segment generation conditions, said segments being signals in which a pulse wave original signal for a second period has been partially extracted from the pulse wave original signal in the first period, and a step in which a pulse wave estimation unit estimates the pulse wave of the person based on the plurality of pulse wave original signal segments generated per measurement area. [Possibilities of industrial application]

[0190] The pulse wave estimation device according to the present disclosure can, even when a phase difference occurs in the luminance signals extracted from a plurality of measurement areas, estimate the pulse wave of a person based on these luminance signals. [Explanation of reference symbols]

[0191] 1, 1a, 1b pulse wave estimation device, 11 imaging image acquisition unit, 12 skin area detection unit, 13 measurement area setting unit, 14 pulse wave original signal extraction unit, 15 segment generation unit, 16, 16a pulse wave estimation unit, 161 signal separation unit, 162, 162a restoration unit, 1621 component weight setting unit, 163 estimation unit, 17 output unit, 18 parameter setting unit, 19 weighting coefficient calculation unit, 3 imaging device, 101 processing circuit, 102 input interface device, 103 output interface device, 104 processor, 105 memory. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] JP 2017-93760 A

[0003]

Claims

[1] Pulse wave estimation device comprising an image acquisition unit for acquiring an image depicting a person, a skin area detection unit for detecting a skin area of ​​the person based on the image, a measurement area setting unit for setting, in an area corresponding to the skin area on the image image, a plurality of measurement areas to extract a pulse wave origin signal representing luminance changes in time series in a first period, a pulse wave original signal extraction unit for extracting the pulse wave original signal for each of the measurement areas based on the luminance changes in the first period in the respective measurement areas, a segment generation unit for generating, for each of the measurement areas based on the pulse wave original signal extracted from the corresponding measurement area according to segment generation conditions, a plurality of pulse wave original signal segments, which are signals in which a pulse wave original signal for a second period has been partially extracted from the pulse wave original signal in the first period, and a pulse wave estimation unit for estimating the pulse wave of the person based on the plurality of pulse wave original signal segments generated for each of the measurement areas. [2] Pulse wave estimation device according to claim 1, wherein the segment generation conditions include that the second period is shorter than the first period, that with respect to the plurality of pulse wave original signal segments in a certain measurement range, one of the pulse wave original signal segments is a signal that does not completely coincide with other pulse wave original signal segments on a time axis, that with respect to the plurality of pulse wave original signal segments in a certain measurement range, one of the pulse wave original signal segments is a signal that partially overlaps with other pulse wave original signal segments on the time axis, that in each of the measuring ranges, a duration of partial overlap of a specific pulse wave original signal segment with other pulse wave original signal segments on the time axis is of equal length, and that in each of the measuring ranges, lengths of the plurality of pulse wave original signal segments on the time axis are identical to each other. [3] Pulse wave estimation device according to claim 1, wherein a parameter setting unit for calculating, based on the pulse wave of the person estimated by the pulse wave estimation unit, the second period and for setting segment parameters containing at least information representing a length of the second period, wherein the segment generation unit sets the second period according to the segment parameters. [4] The pulse wave estimation device according to claim 3, wherein the parameter setting unit calculates the second period such that the second period is a time of one cycle of the pulse wave of the person estimated by the pulse wave estimation unit. [5] The pulse wave estimation device according to claim 1, comprising a weighting coefficient calculating unit for calculating weighting coefficients of the respective plurality of pulse wave original signal segments, and wherein the pulse wave estimation unit estimates the pulse wave of the person based on the pulse wave original signal segments and the weighting coefficients. [6] The pulse wave estimation device according to claim 5, wherein the weighting coefficient calculation unit calculates the weighting coefficients for the respective pulse wave original signal segments based on the magnitude of a signal fluctuation in the pulse wave original signal from which the pulse wave original signal segments were extracted. [7] The pulse wave estimation device according to claim 1, wherein the pulse wave estimation unit includes a signal separation unit for generating, based on the plurality of pulse wave original signal segments, separation signals representing a plurality of signal components, a recovery unit for recovering the pulse wave original signal for each of the measurement ranges based on the plurality of separation signals, and an estimation unit for estimating the pulse wave of the person based on the restored pulse wave original signal for each of the measurement areas. [8] Pulse wave estimation device according to claim 7, wherein the recovery unit a component weighting setting unit for setting weighting coefficients for the respective separation signals, and based on the plurality of separation signals and the weighting coefficients for the respective separation signals, restores the pulse wave original signal for each of the measurement ranges. [9] Pulse wave estimation device according to claim 8, wherein the component weighting adjustment unit sets the weighting coefficients for the respective separation signals based on a similarity of the frequency characteristics between the plurality of separation signals. [10] Pulse wave estimation device according to claim 8, wherein the component weighting adjustment unit when generating the plurality of separation signals, based on the projection coefficients assigned to the separation signals, correlation coefficients of projection coefficients between the plurality of separation signals are calculated, and the weighting coefficients for the respective separation signals are set based on the correlation coefficients. [11] Pulse wave assessment method, including a step of acquiring, by an image acquisition unit, an image depicting a person, a step of detecting, by a skin area detection unit, a skin area of ​​the person based on the image, a step of setting, by a measurement area setting unit, in an area corresponding to the skin area on the image image, a plurality of measurement areas to extract a pulse wave origin signal representing luminance changes in time series in a first period, a step of extracting, by a pulse wave original signal extraction unit for each of the measurement areas, based on the luminance changes in the first period in the respective measurement areas of a pulse wave original signal, a step of generating, by a segment generating unit, for each of the measurement areas based on the pulse wave original signal extracted from the corresponding measurement area, a plurality of pulse wave original signal segments according to segment generating conditions, wherein the segments are signals in each of which a pulse wave original signal for a second period has been partially extracted from the pulse wave original signal in the first period, and a step of estimating, by a pulse wave estimation unit, a pulse wave of the subject based on the plurality of pulse wave original signal segments generated for each of the measurement areas.

Images