Pulse wave assessment device and pulse wave assessment method

Abstract

Pulse wave assessment device, comprising a picture acquisition unit (11) for capturing a picture depicting a person, a skin area detection unit (12) for detecting a skin area of ​​the person based on the image, a measurement range setting unit (13) for setting, in an area corresponding to the skin area in the illustration, several measurement ranges to extract a pulse wave origin signal representing luminance changes in time series in a first period, a pulse wave origin signal extraction unit (14) for extracting the pulse wave origin signal for each of the measurement ranges based on the luminance changes in the first period in the respective measurement ranges, a segment generation unit (15) for generating, for each of the measurement ranges on the basis of the pulse wave origin signal extracted from the corresponding measurement range, several pulse wave origin signal segments according to segment generation conditions, wherein the signals are signals in which a pulse wave origin signal for a second period has been partially extracted from the pulse wave origin signal in the first period, a pulse wave estimation unit (16, 16a) for estimating the person's pulse wave based on the multiple pulse wave origin signal segments generated for each of the measurement ranges, and a parameter setting unit (18) for calculating, based on the pulse wave of the person estimated by the pulse wave estimation unit (16, 16a), the second period and for setting segment parameters which contain at least information representing a length of the second period, wherein the segment generation unit (15) sets the second period according to the segment parameters.

Description

[Technical field]

[0001] The present disclosure relates to a pulse wave assessment device and a pulse wave assessment method. [State of the art]

[0002] A conventional technique for assessing a person's pulse wave is known in which subtle luminance changes on the surface of the person's skin are extracted based on luminance signals in several areas (hereinafter referred to as "measurement areas") set within a region encompassing the person's skin in an image projected by an imaging device (hereinafter referred to as the "skin area"), and pulse wave components are separated from signals representing these luminance changes using a signal separation technique such as independent component analysis or principal component analysis (for example, patent document 1). This technique exploits the principle that the amount of light absorption in a person's bloodstream changes depending on the pulse rate. [Documents relating to the state of the art][Patent documents]

[0003] [Patent document 1]

[0004] JP 2017-93760 A [Overview of the invention][Problem to be solved by the invention]

[0005] Human blood leaves the heart through the arteries, circulates throughout the body, and returns to the heart via the veins. Due to this characteristic of human blood flow, a large difference in distance from the heart between multiple measurement ranges set for pulse wave assessment leads to a phase difference in the luminance signals extracted from these ranges. When a phase difference exists in the extracted luminance signals between multiple measurement ranges, pulse wave components cannot be distinguished from signals representing luminance changes, which can reduce the accuracy of the pulse wave assessment.When conventional techniques detect a phase difference in luminance signals extracted from multiple measurement ranges, the problem is that it may not be possible to estimate a person's pulse wave based on the luminance signals where this phase difference occurs.

[0006] The purpose of the present disclosure is to solve the above-mentioned problems and to provide a pulse wave estimation device which, even if a phase difference occurs in luminance signals extracted from several measurement ranges, can estimate the pulse wave of a person on the basis of these luminance signals. [Means of solving the problem]

[0007] The pulse wave assessment device according to the invention comprises an image acquisition unit for acquiring an image depicting a person, a skin area detection unit for detecting a skin area of ​​the person based on the image, a measurement range setting unit for setting, in an area corresponding to the skin area on the image, several measurement ranges in order to extract a pulse wave origin signal representing luminance changes in time series in a first period, a pulse wave origin signal extraction unit for extracting the pulse wave origin signal for each of the measurement ranges based on the luminance changes in the first period in the respective measurement ranges, and a segment generation unit for generatingFor each of the measurement ranges, based on the pulse wave origin signal extracted from the corresponding measurement range, several pulse wave origin signal segments are generated according to segment generation conditions, wherein signals are those in which a pulse wave origin signal for a second period has been partially extracted from the pulse wave origin signal in the first period; a pulse wave estimation unit for estimating the pulse wave of the person based on the several pulse wave origin signal segments generated for each of the measurement ranges; and a parameter setting unit (18) for calculating, based on the pulse wave of the person estimated by the pulse wave estimation unit (16, 16a), the second period and for setting segment parameters that contain at least information representing a length of the second period, wherein the segment generation unit (15) sets the second period according to the segment parameters. [Effect of the invention]

[0008] According to the present disclosure, even if a phase difference occurs in luminance signals extracted from several measurement ranges, the pulse wave of a person can be estimated on the basis of these luminance signals. [Brief explanation of the drawings] [ Fig. Figure 1] is a drawing showing a structural example of a pulse wave assessment device according to embodiment 1. [ Fig. Figure 2] is a drawing to illustrate the phase difference of the luminance signals that occurs between several measurement ranges due to the characteristics of human blood flow. [ Fig. 3] Fig. 3A, Fig. 3B and Fig. Figure 3C are drawings illustrating an exemplary method for setting measuring ranges by the measuring range setting unit in the pulse wave estimation device according to embodiment 1. [ Fig. Figure 4] is a drawing to illustrate an example of pulse wave origin signal segments generated by the segment generation unit in embodiment 1. [ Fig. Figure 5] is a drawing illustrating the processes in a signal separation unit of embodiment 1, wherein main components are analyzed on the basis of several pulse wave origin signal segments and separation signals are generated that represent the analyzed main components. [ Fig. Figure 6] is a drawing illustrating a detailed example of isolation signals generated by the signal isolation unit of embodiment 1, which represent the main components. [ Fig. Figure 7] is a flowchart to explain the operation of the pulse wave assessment device according to embodiment 1. [ Fig. [8] is a flowchart to explain the details of the pulse wave estimation processing by the pulse wave estimation unit in step ST6 of Fig. 7. [ Fig. 9] Fig. 9A and Fig. Figure 9B shows examples of the hardware setup of a pulse wave assessment device according to embodiment 1. [ Fig. Figure 10] is a drawing showing a structural example of a pulse wave assessment device according to embodiment 2. [ Fig.

[11] is a flowchart to explain the operation of the pulse wave assessment device according to embodiment 2. [ Fig.

[12] is a drawing showing a structural example of a pulse wave assessment device according to embodiment 3. [ Fig. Figure 13] is a drawing to illustrate the pulse wave origin signal segments used in embodiment 3 by the pulse wave estimation device to estimate the pulse wave of a test subject. [ Fig.

[14] is a flowchart to explain the operation of the pulse wave assessment device according to embodiment 3. [ Fig. Figure 15] is a drawing showing a structural example of a pulse wave estimation unit in embodiments 1 to 3 in the case that the pulse wave estimation device has the function of setting component weights for several separation signals representing several principal components that were estimated using known general signal separation techniques, and of restoring the pulse wave origin signal for each measurement range on the basis of the several separation signals and the set component weight for each separation signal. [ Fig.

[16] is a drawing showing an image of an example of projection coefficient information in time series generated by a component weighting setting unit. [ Fig.

[17] is a flowchart to explain the operation of the pulse wave estimation unit in embodiments 1 to 3 in the case that the pulse wave estimation device has the function of setting component weights for several separation signals representing several main components that were estimated using known general signal separation techniques and restoring the pulse wave origin signal for each measurement range on the basis of the several separation signals and the set component weight for each separation signal. [Embodiments of the invention]

[0009] The embodiments of the present disclosure are explained in detail below with reference to the drawings. Design 1.

[0010] Fig. Figure 1 is a drawing showing a structural example of a pulse wave assessment device 1 according to embodiment 1.

[0011] The pulse wave assessment device 1 assesses the pulse wave of a person based on an image depicting that person. In the following description, the person whose pulse wave is to be assessed by the pulse wave assessment device 1 is also referred to as the "test subject." The pulse wave assessment device 1 acquires an image consisting of a series of frames Im(k), which depicts an area in which at least one skin area should be located as the area encompassing the skin of the test subject (hereinafter referred to as the "skin presence area") at a predetermined frame rate Fr. Here, k represents the frame number assigned to the respective frame. For example, the frame that is provided at the time following frame Im(k) is frame Im(k+1). In embodiment 1, the skin area is an area corresponding to the face of the test subject.This is merely an example, and the skin area can also refer to a different area than the test subject's face. For instance, the skin area could correspond to a part of the face, such as the eyes, eyebrows, nose, mouth, forehead, cheeks, or chin. Furthermore, the skin area could correspond to a body part other than the face, such as the head, shoulders, hands, neck, or legs. The skin area could also consist of multiple areas, such as a skin area corresponding to the cheek, a skin area corresponding to the forehead, or a skin area corresponding to the neck.

[0012] The pulse wave assessment device 1 then assesses the subject's pulse wave from a series of frames Im(k-Tp+1) to Im(k) per a specific frame rate Tp acquired in the period preset as the period for performing a pulse wave assessment (hereinafter referred to as the "first period"), and outputs the pulse wave assessment result P(t), which is information representing the assessed pulse wave (hereinafter referred to as "pulse wave information"). Specifically, the pulse wave assessment device 1 sets several areas (hereinafter referred to as "measurement areas") for the subject's skin in a series of frames Im(k-Tp+1) to Im(k).The pulse wave assessment device 1 then separates pulse wave components for a signal representing luminance changes per measurement range (hereinafter referred to as the "pulse wave source signal") using a general signal separation technique, such as independent component analysis (hereinafter referred to as "ICA") or principal component analysis (hereinafter referred to as "PCA"), and assesses the test subject's pulse wave based on the separated pulse wave components. In separating the pulse wave components, the pulse wave assessment device 1 generates multiple pulse wave source signals (hereinafter referred to as "pulse wave source signal segments"), from which a pulse wave source signal of a predetermined period (hereinafter referred to as the "second period") has been partially extracted, and performs ICA or PCA on the multiple pulse wave source signal segments.

[0013] Here, t represents an output number assigned to each specific frame rate Tp acquired in the first period. For example, the pulse wave assessment result given at the time following the pulse wave assessment result P(t) is the pulse wave assessment result P(t+1). The frame number k and the output number t are integers greater than or equal to 1. The frame rate Tp is an integer greater than or equal to 2.

[0014] Assessing a test subject's pulse wave from luminance changes across multiple measurement zones set on the skin is effective. However, due to the characteristics of human blood flow, a large distance between multiple measurement zones can lead to phase differences in the luminance signals extracted from these zones.

[0015] Fig. Figure 2 is a drawing to illustrate the phase difference of the luminance signals that occurs between several measurement ranges due to the characteristics of human blood flow.

[0016] In Fig. Figure 2 shows the left side of the drawing, which depicts the skin area on the image of the test subject. Fig. 2 stands for "H" for the test subject and "sr" for the skin area. It is assumed that four measurement ranges are set in the skin area.

[0017] Fig. The four measuring ranges are designated "R1", "R2", "R3", and "R4". Furthermore, a method for detecting a skin area and a method for setting measuring ranges are described in more detail below.

[0018] The right side of the drawing shows luminance values ​​represented by the luminance signals extracted from the measurement range labeled "R1" on the time axis (in Fig. 2 labelled 22) and luminance values ​​represented by the luminance signals extracted from the measurement range labelled “R3”, on the time axis (in Fig. 2 (labeled 21).

[0019] The test subject's blood is pumped by the heart and flows from the neck to the forehead. Therefore, if the distance between the measurement ranges in the direction of blood flow is large, as with the measurement ranges labeled "R1" and "R3"—in other words, if the distance to the heart between the measurement ranges is large—a phase difference occurs in the luminance signals extracted from the measurement ranges.

[0020] If a phase difference occurs between multiple luminance signals extracted from multiple measurement ranges, the separation of the pulse wave components will fail if the separation of the pulse wave components was performed using general signal separation techniques, such as ICA or PCA, for the pulse wave origin signal for each measurement range.

[0021] The pulse wave assessment device 1 according to embodiment 1 takes into account the occurrence of a phase difference as described above, generates several pulse wave origin signal segments and performs a separation of the pulse wave components using general signal separation techniques, such as ICA or PCA, on the generated several pulse wave origin signal segments, so that even if a phase difference occurs between the several luminance signals extracted from several measurement ranges, this phase difference is absorbed and the pulse wave of the test subject can be assessed.

[0022] The number of test subjects depicted in the illustration can be one or more. For the sake of simplicity, the following explanation of embodiment 1 will assume one test subject depicted in the illustration.

[0023] In embodiment 1, the pulse wave assessment device 1 is installed, for example, in a vehicle (not shown), and the test subject is the driver of the vehicle. That is, the pulse wave assessment device 1 assesses the pulse wave of the vehicle driver. The pulse wave information of the driver assessed by the pulse wave assessment device 1 is output, for example, to a condition assessment device (not shown) installed in the vehicle. The condition assessment device assesses the driver's condition based on the pulse wave assessment result P(t) of the driver, which was output by the pulse wave assessment device 1. If, for example, the condition assessment device assesses that the driver's condition is unfit to drive, it emits a warning tone to the driver.

[0024] As in Fig. As shown in Figure 1, the pulse wave assessment device 1 comprises an imaging unit 11, a skin area detection unit 12, a measurement range setting unit 13, a pulse wave origin signal extraction unit 14, a segment generation unit 15, a pulse wave assessment unit 16, and an output unit 17. The pulse wave assessment unit 16 comprises a signal separation unit 161, a recovery unit 162, and an assessment unit 163.

[0025] The pulse wave assessment device 1 is connected to an imaging device 3 installed in the vehicle. The imaging device 3 is positioned to image the skin presence area of ​​the test subject, in this case the driver. The imaging device 3 can be used, for example, in conjunction with an imaging device of a so-called "driver monitoring system (DMS)" installed in a vehicle to monitor the driver's condition while in the vehicle.

[0026] The image acquisition unit 11 acquires an image from the imaging device 3, on which a test subject is depicted. The image acquisition unit 11 outputs the acquired image to the skin area detection unit 12.

[0027] The skin area detection unit 12 detects a skin area of ​​the test subject based on the frame Im(k) included in the image acquired by the image acquisition unit 11. The skin area detection unit 12 can detect the skin area using a known method. For example, the skin area detection unit 12 can detect a skin area using a cascade-type facial sensor that uses hair-like feature sizes.

[0028] The skin area detection unit 12 generates skin area information S(k) that represents the detected skin area.

[0029] The skin area information S(k) can include information indicating the presence or absence of a skin area and information representing the position and size of a detected skin area in the image. In embodiment 1, a skin area is represented by a rectangular area in the image, and the skin area information S(k) includes information representing the position and size of the rectangular area in the image.

[0030] Specifically, the skin area information S(k) represents, for example, the presence / absence of evidence of the test subject's face, the central coordinates Fc (Fcx, Fcy) of a rectangle enclosing the test subject's face in the image, and the width Fcw and height Fch of the rectangle, if the skin area corresponds to the test subject's face. The presence / absence of evidence of the test subject's face is expressed, for example, by "1" if evidence was possible and by "0" if no evidence was possible. Furthermore, the central coordinates of the rectangle surrounding the face are represented by the coordinate system of frame Im(k).

[0031] Furthermore, the skin area detection unit 12 can also detect multiple skin areas.

[0032] The skin area detection unit 12 outputs the generated skin area information S(k) to the measuring range setting unit 13.

[0033] Based on the frame Im(k) of the image acquired by the image acquisition unit 11 and the skin area information S(k) output by the skin area detection unit 12, the measurement range setting unit 13 sets several measurement ranges in the image area corresponding to the skin area represented by the skin area information S(k) in frame Im(k) in order to extract a pulse wave origin signal representing luminance changes in time series in a first period. Furthermore, the measurement range setting unit 13 can acquire the image acquired by the image acquisition unit 11 via the skin area detection unit 12.

[0034] When the measuring range setting unit 13 sets multiple measuring ranges, it generates measuring range information R(k) representing the set multiple measuring ranges. The measuring range information R(k) contains information representing the position and size of N (an integer greater than or equal to 2) measuring ranges on the illustration. The individual measuring ranges are the measuring ranges ri(k) (i = 2, ..., N). In embodiment 1, a measuring range ri(k) is a quadrilateral, and the position and size of the measuring range ri(k) correspond to the coordinate values ​​of the four vertices of the quadrilateral on the illustration.

[0035] Fig. 3A, Fig. 3B and Fig. Figure 3C shows drawings illustrating an exemplary method for setting measuring ranges ri(k) by the measuring range setting unit 13 in the pulse wave estimation device 1 according to embodiment 1.

[0036] Based on Fig. Section 3 explains an example of a procedure for setting multiple measuring ranges ri(k) using the measuring range setting unit 13. First, as in Fig. 3A and Fig. Figure 3B shows that, through the measuring range setting unit 13 in a skin area sr, represented by the skin area information S(k), Ln-piece (positive integer) orientation points of facial organs such as the outer corner of the eye, the inner corner of the eye, the nose and the mouth are detected. Fig. 3A and Fig. In 3B, the orientation points are marked by circles. A vector in which the measuring range setting unit 13 has stored the coordinate values ​​of the detected orientation points is specified as L(k).

[0037] Furthermore, the measuring range setting unit 13 can detect the facial organs using known means, for example with a model called Constrained Local Model (CLM).

[0038] The measuring range setting unit 13 then sets the vertex coordinates of quadrilaterals of the measuring ranges ri (k) based on the proven orientation points. For example, the measuring range setting unit 13 sets the vertex coordinates of quadrilaterals as shown in Fig. 3C is shown, and sets N measuring ranges ri(k).

[0039] To illustrate the setting of the measurement ranges ri(k) by the measurement range setting unit 13 in a section of the skin area sr corresponding to a cheek, the measurement range setting unit 13 selects an orientation point LA1 of the facial contour and an orientation point LA2 of the nose. The measurement range setting unit 13 can first select the orientation point LA2 of the nose, and then select the orientation point LA1 of the facial contour that is closest to the orientation point LA2 of the nose.

[0040] The measuring range setting unit then sets 13 auxiliary orientation points a1, a2 and a3, so that a line segment between orientation point LA1 and orientation point LA2 is divided into four parts.

[0041] Similarly, the measuring range setting unit 13 selects an orientation point LB1 of the facial contour and an orientation point LB2 of the nose. Furthermore, the measuring range setting unit 13 sets auxiliary orientation points b1, b2, and b3, so that a line segment between orientation point LB1 and orientation point LB2 is divided into four parts. Orientation points LB1 and LB2 can, for example, be selected as orientation points of the facial contour and the nose, respectively, that are adjacent to orientation points LA1 and LA2. The measuring range setting unit 13 sets a quadrilateral area encompassing the auxiliary orientation points a1, b1, b2, and a2 as measuring range R5. The auxiliary orientation points a1, b1, b2, and a2 each become the corner point coordinates corresponding to measuring range R5.

[0042] Likewise, the measuring range setting unit 13 sets a measuring range R6 encompassing the auxiliary orientation points a2, b2, b3 and a3, and the corner point coordinates of the measuring range R6.

[0043] Furthermore, an example was explained here in which measurement ranges ri(k) were set for a section corresponding to the cheek, but the measurement range setting unit 13 similarly sets measurement ranges ri(k) and corner point coordinates of these measurement ranges ri(k) for the skin area sr of other sections of the cheek and the chin corresponding to sections. Although in Fig. Not shown in Figure 3C, the measuring range setting unit 13 can further set the measuring ranges ri(k) for the section of the skin area sr of the test subject that corresponds to the forehead, the section that corresponds to the neck, or the section that corresponds to the tip of the nose.

[0044] The measurement range setting unit 13 can also set measurement ranges ri(k) using a method other than CLM. For example, the measurement range setting unit 13 can also set measurement ranges ri(k) using a tracking technique such as the Kanade-Lucas-Tomasi (KLT) tracker. Specifically, the measurement range setting unit 13 can also detect the coordinates of the facial organ points using CLM for the skin area in the first frame Im(1) of a series of frames Im(k-Tp+1) to Im(k), and then, starting with the skin area in the next frame Im(2), track the facial organ points with the KLT tracker to calculate the facial organ points for the skin area of ​​each frame Im(k). Since detection errors accumulate in this case due to tracking, the measurement range setting unit 13 can also perform CLM every few frames and reset the coordinate positions of the facial organ points or perform other reset processing.

[0045] The measuring range setting unit 13 outputs the generated measuring range information R(k) to the pulse wave origin signal extraction unit 14.

[0046] The pulse wave origin signal extraction unit 14 extracts a pulse wave origin signal, representing luminance changes in the first period, from each of the multiple measurement ranges ri(k) represented by the measurement range information R(k) in frame Im(k), based on the image frame Im(k) acquired by the image acquisition unit 11 and the measurement range information R(k) output by the measurement range setting unit 13. The pulse wave origin signal is a signal that represents the origin of a pulse wave. The pulse wave assessment device 1 uses the pulse wave origin signal to assess the test subject's pulse wave. The assessment of the test subject's pulse wave is performed by the pulse wave assessment unit 16. The pulse wave assessment unit 16 is described in detail below.

[0047] The pulse wave origin signal extraction unit 14 can acquire the image acquired by the image acquisition unit 11 via the skin area detection unit 12 and the measurement range setting unit 13.

[0048] Once the pulse wave origin signal has been extracted, the pulse wave origin signal extraction unit generates 14 pulse wave origin signal information W(t), which represents the extracted pulse wave origin signal.

[0049] The pulse wave origin signal information W(t) comprises information representing a pulse wave origin signal wi(t) extracted from the measurement range ri(k). The pulse wave origin signal wi(t), which consists of time series data for the Tp component, is extracted, for example, based on frames Im(k-Tp+1), Im(k-Tp+2), ..., Im(k) of a past Tp component and measurement range information R(k-Tp+1), R(k-Tp+2), ..., R(k).

[0050] When extracting the pulse wave origin signal wi(t), the pulse wave origin signal extraction unit 14 calculates, for each frame Im(k) of the image, a difference Gi(j) (j=k-Tp+1, k-Tp+2, ..., k) of the luminance feature of each measurement range ri(k) compared to a previous frame Im(k-1). The luminance feature is a value calculated for each measurement range ri(j) based on the luminance values ​​in frame Im(j) of the image. The luminance feature is an average or variance, etc., of the luminance values ​​of the image elements contained in the measurement ranges ri(j). In embodiment 1, for example, the luminance feature is the average of the luminance values ​​of the image elements contained in the measurement ranges ri(j).

[0051] The pulse wave origin signal extraction unit 14 sequences the Gi(j), calculated for each frame Im(k) of the image acquired in the first period, into time series to determine the pulse wave origin signal wi(t). That is, the pulse wave origin signal extraction unit 14 determines the pulse wave origin signal wi(t) = [Gi(k-Tp+1), Gi(k-Tp+2), ..., Gi(k)].

[0052] The pulse wave origin signal extraction unit 14 can further extract the pulse wave origin signal wi(t) by calculating the luminance feature of each measurement range ri(k) for each frame Im(k) of the image.

[0053] The pulse wave origin signal extraction unit 14 generates pulse wave origin signal information W(t), which represents a pulse wave origin signal wi(t) in each measurement range ri(k). The pulse wave origin signal information W(t) comprises the pulse wave origin signal wi(t) in each measurement range ri(k) and information indicating, for the respective pulse wave origin signal wi(t), from which measurement range ri(k) the pulse wave origin signal wi(t) was extracted. The pulse wave origin signal extraction unit 14 outputs the generated pulse wave origin signal information W(t) to the segment generation unit 15.

[0054] The segment generation unit 15 generates several pulse wave origin signal segments per measurement range ri(k) according to predefined conditions (hereinafter referred to as "segment generation conditions") based on the pulse wave origin signal information W(t) output by the pulse wave origin signal extraction unit 14, wherein these are signals in which a pulse wave origin signal for a second period has been partially extracted from the pulse wave origin signal wi(t) in the first period.

[0055] The segment generation conditions are generated in advance by an administrator or the like and stored by the segment generation unit 15.

[0056] The following conditions (1) to (5) are set as segment generation conditions, for example. The segment generation unit 15 generates several pulse wave origin signal segments such that all conditions (1) to (5) are met. (1) The second period must be shorter than the first period. (2) With regard to the multiple pulse wave origin signal segments in a given measurement range, a particular pulse wave origin signal segment is a signal that does not completely coincide with other pulse wave origin signal segments on the time axis. (3) With regard to the multiple pulse wave origin signal segments in a given measurement range, a given pulse wave origin signal segment is a signal that partially overlaps with other pulse wave origin signal segments on the time axis. (4) In each measurement range, the duration of the partial overlap of a particular pulse wave origin signal segment with other pulse wave origin signal segments on the time axis is the same. (5) In each measurement range, the lengths of the multiple pulse wave origin signal segments are identical to each other on the time axis.

[0057] Fig. Figure 4 is a drawing to illustrate an example of pulse wave origin signal segments generated by the segment generation unit 15 in embodiment 1.

[0058] In Fig. Figure 4, the left-hand diagram, is a schematic showing the skin area sr for which the measurement ranges ri(k) were set (hereinafter referred to as the "skin area diagram"). In the skin area diagram, the test subject is denoted by "H". As shown in the skin area diagram, it is assumed that four measurement ranges ri(k) are set in the skin area sr. Specifically, it is assumed that two measurement ranges ri(k) (denoted "n1" and "n2" in the skin area diagram) are set in the area of ​​the test subject's chin and two measurement ranges ri(k) (denoted "f1" and "f2" in the skin area diagram) are set in the area of ​​the test subject's forehead. In the following description, the measuring range ri(k) designated by "n1" is referred to as the "first measuring range", the measuring range ri(k) designated by "n2" as the "second measuring range", the measuring range ri(k) designated by "f1" as the "third measuring range" and the measuring range ri(k) designated by "f2" as the "fourth measuring range".

[0059] In Fig. Figure 4, the middle diagram, is a diagram illustrating an example of pulse wave origin signal segments generated by the segment generation unit 15 for the first and third measurement ranges (hereinafter referred to as the "segment generation diagram"). In the segment generation diagram, the luminance values ​​represented by the pulse wave origin signal wi(t) in the first period (labeled "X" in the segment generation diagram) are plotted on the time axis.

[0060] The segment generation scheme shows an example above of several pulse wave origin signal segments (= p1-1, p1-2, p1-3, p1-4, p1-5, ..., p1-M) for the first measurement range, which were generated by the segment generation unit 15 by partially extracting the pulse wave origin signal wi(t) for the second period from the pulse wave origin signal wi(t) extracted from the first measurement range (labeled "401" in the segment generation scheme), while below shows an example of several pulse wave origin signal segments (= p3-1, p3-2, p3-3, p3-4, p1-5, ..., p1-M) for the third measurement range, which were generated by the segment generation unit 15 by partially extracting the pulse wave origin signal wi(t) for the second period from the pulse wave origin signal extracted from the third measurement range. wi(t) (in the segment generation scheme marked by “402”) were generated.

[0061] The segment generation unit 15 generates, for each measurement range ri(k) for the first period, which represents the period for performing a pulse wave assessment, M pulse wave origin signal segments (where M is an integer of 2 or more) according to the segment generation conditions, in which the pulse wave origin signal wi(t) for the second period has been partially extracted from the pulse wave origin signal wi(t) of the first period.

[0062] Furthermore, each pulse wave origin signal segment is assigned information indicating from which measurement range ri(k) the pulse wave origin signal segment was generated from the pulse wave origin signal wi(t), and which pulse wave origin signal segment it is on the time axis. It is also assigned information indicating the time range on the time axis of the pulse wave origin signal wi(t) in time series from which the extraction took place. For example, the pulse wave origin signal segment p1-1 indicates that it is the first pulse wave origin signal segment on the time axis of several pulse wave origin signal segments generated for the pulse wave origin signal wi(t) of the first measurement range; in other words, the pulse wave origin signal segment of the earliest time period.The pulse wave origin signal segment p1-1 is assigned information indicating, for example, that it is the pulse wave origin signal wi(t) of the first measurement range from 〇 h:〇min:〇s to △h: △min: △s.

[0063] On the segment generation scheme of Fig. For simplicity, Figure 4 shows M pulse wave origin signal segments for the first and third measurement ranges, but the segment generation unit 15 also generates M pulse wave origin signal segments for the second and fourth measurement ranges. That is, the segment generation unit 15 also generates M pulse wave origin signal segments (= p2-1, p2-2, p2-3, p2-4, p2-5, ..., p2-M) for the second measurement range and M pulse wave origin signal segments (= p4-1, p4-2, p4-3, p4-4, p4-5, ..., p4-M) for the fourth measurement range.

[0064] As mentioned previously, a large difference between the measurement ranges ri(k) with respect to their distance from the heart leads to a phase difference in the luminance signals extracted from the measurement ranges ri(k). Conversely, there is no phase difference in the luminance signals extracted from the measurement ranges ri(k) if there is almost no difference between the measurement ranges ri(k) with respect to their distance from the heart. Since the second measurement range shows hardly any difference from the first measurement range with respect to its distance from the heart, the M pulse wave origin signal segments for the second measurement range are identical in content to the M pulse wave origin signal segments for the first measurement range.Since the fourth measurement range shows virtually no difference to the third measurement range with respect to the distance to the heart, the M pulse wave origin signal segments for the fourth measurement range are identical in content to the M pulse wave origin signal segments for the third measurement range. Furthermore, in the first embodiment, the term "identical" is not limited to an exact match, but also includes an "approximate match" within acceptable limits.

[0065] When the segment generation unit 15 generates multiple pulse wave origin signal segments for each measurement range ri(k), it creates a trajectory matrix from the generated pulse wave origin signal segments and stacks them in a storage unit (not shown). The storage unit is located at a point where it can be referenced by the pulse wave estimation device 1.

[0066] In Fig. 4 There are four measurement ranges ri(k) such that the segment generation unit 15, when it generates M pulse wave origin signal segments for each measurement range ri(k), stacks an M × 4-row trajectory matrix (= [p1-1, p1-2, p1-3, p1-4, p1-5, ..., p1-M], ..., [p4-1, p4-2, p4-3, p4-4, p4-5, ..., p4-M]) (see Fig. 4, right drawing).

[0067] Furthermore, there are four measuring ranges ri(k) here, but this is merely an example, and N measuring ranges ri(k) can be set. The segment generation unit 15 stacks an M x N row trajectory matrix (= [p1-1, p1-2, p1-3, p1-4, p1-5, ..., p1-M], ..., [pN-1, pN-2, pN-3, pN-4, pN-5, ..., pN-M]).

[0068] When the segment generation unit 15 stacks the trajectory matrix, it informs the pulse wave estimation unit 16.

[0069] When the pulse wave estimation unit 16 is informed by the segment generation unit 15 that the trajectory matrix has been stacked, it estimates the test subject's pulse wave based on the multiple pulse wave origin signal segments generated for each measurement range ri(k).

[0070] More precisely, the signal separation unit 161 first generates signals (hereinafter referred to as "separation signals") based on the multiple pulse-wave origin signal segments, representing several signal components (hereinafter referred to as "principal components"). Specifically, the signal separation unit 161 analyzes several principal components using general signal separation techniques, such as PCA or ICA, and generates separation signals that represent the analyzed multiple principal components. By analyzing the signal components using general signal separation techniques, such as PCA or ICA, the signal separation unit 161 separates the components typical of pulse-wave components from the components typical of noise components within the multiple pulse-wave origin signal segments.

[0071] Furthermore, the signal separation unit 161 generates separation signals representing several main components based on the multiple pulse wave origin signal segments corresponding to all measurement ranges ri(k).

[0072] Fig. Figure 5 is a drawing illustrating the processes in the signal separation unit 161 of embodiment 1, wherein main components are analyzed on the basis of the multiple pulse wave origin signal segments and separation signals are generated that represent the analyzed main components.

[0073] It is assumed here that the measurement ranges are ri(k), as in Fig. Figure 4 shows that four measuring ranges ri(k) can be set, namely the first measuring range, the second measuring range, the third measuring range and the fourth measuring range.

[0074] The one on the left side of Fig. The process flow shown in Figure 5, from generating multiple pulse wave origin signal segments for each of the first, second, third, and fourth measuring ranges and stacking the M x 4-row trajectory matrix, is the process flow carried out by the segment generation unit 15 and is already based on Fig. Section 4 explained why a redundant explanation is omitted.

[0075] The signal separation unit 161 analyzes the principal components using generally known signal separation techniques, such as PCA or ICA, for the M x 4-row trajectory matrix stacked by the segment generation unit 15. As a result, the signal separation unit 161 analyzes C (positive integer) principal components and generates separation signals representing the analyzed C principal components. Furthermore, using general signal separation techniques, the generation proceeds sequentially, starting with the principal components containing the most information. That is, the first principal component c1, the second principal component c2, the third principal component c3, ... and the Cth principal component cC are generated in order of their information content.

[0076] Fig. Figure 6 is a drawing illustrating a detailed example of isolation signals generated by the signal isolation unit 161 of embodiment 1, which represent the main components.

[0077] Furthermore, it explains Fig. 6 the in Fig. The 5 separation signals shown (= [c1, c2, c3, c4, c5, ..., cC]) are detailed, i.e., those based on the information in Fig. 4 shown for each of the first, second, third and fourth measurement ranges generated by the signal separation unit 161 are separation signals (= [c1, c2, c3, c4, c5, ..., cC]).

[0078] Each separation signal contains the trajectory matrix of several pulse wave origin signal segments, from which the information content contained in the principal component represented by the separation signal was analyzed. In the Fig. In the example shown, the separation signals (= [c1, c2, c3, c4, c5, ..., cC]) represent the first principal component c1 to the C-th principal component cC, separation signals are based on the pulse wave origin signal segments (= p1-1, p1-2, p1-3, p1-4, p1-5, ..., p1-M) generated from the pulse wave origin signal wi(t) of the first measurement range, the pulse wave origin signal segments (= p2-1, p2-2, p2-3, p2-4, p2-5, ..., p2-M) generated from the pulse wave origin signal wi(t) of the second measurement range, and the pulse wave origin signal segments (= p3-1, p3-2, p3-3, p3-4, p3-5, ..., ...) generated from the pulse wave origin signal wi(t) of the third measurement range. The pulse wave origin signal segments (= p4-1, p4-2, p4-3, p4-4, p4-5, ..., p4-M) were generated from the pulse wave origin signal wi(t) of the fourth measurement range.Consequently, the separation signals representing the first principal component c1 to the C-th principal component cC each comprise an M x 4-row trajectory matrix (= [p1-1, p1-2, p1-3, p1-4, p1-5, ..., p1-M], ..., [p4-1, p4-2, p4-3, p4-4, p4-5, ..., p4-M]).

[0079] Furthermore, each separation signal (= [c1, c2, c3, c4, c5, ..., cC]) is assigned a projection coefficient. Projection coefficients are coefficients assigned according to the information content of the principal components and are assigned during the analysis of the principal components using known general signal separation techniques.

[0080] Each pulse wave origin signal segment in each measurement range ri(k) is assigned projection coefficients for each cutoff signal. In the Fig. In the example shown in 6, the first principal component c1 is assigned projection coefficients corresponding to each of the M pulse wave origin signal segments in the first measurement range, projection coefficients corresponding to each of the M pulse wave origin signal segments in the second measurement range, projection coefficients corresponding to each of the M pulse wave origin signal segments in the third measurement range, and projection coefficients corresponding to each of the M pulse wave origin signal segments in the fourth measurement range.

[0081] In Fig. Figure 6 shows, for example, that the projection coefficient (q1,1-1) assigned to the cutoff signal representing the first principal component c1 is the projection coefficient corresponding to the first pulse wave origin signal segment on the time axis in the first measurement range of the first principal component c1. Furthermore, Figure 6 shows that... Fig. 6 For example, the projection coefficient (q1,4-M) assigned to the cutoff signal representing the first principal component c1 is the projection coefficient corresponding to the M-th pulse wave origin signal segment on the time axis in the fourth measurement range of the first principal component c1. Furthermore, in Fig. 6 for example the projection coefficient (qC,1-1) assigned to the separation signal representing the C-th principal component cC, is the projection coefficient corresponding to the first pulse wave origin signal segment on the time axis in the first measurement range of the C-th principal component cC.

[0082] The signal separation unit 161 outputs the generated multiple separation signals to the recovery unit 162, specifically the separation signal information Sep(t) regarding the multiple separation signals representing the multiple generated main components.

[0083] The separation signal information Sep(t) comprises C separation signals generated from multiple (M x N) pulse wave origin signal segments, which are generated based on the pulse wave origin signal information W(t).

[0084] The recovery unit 162 restores the pulse wave origin signal wi(t) for each measurement range ri(k) on the basis of the separation signal information Sep(t) output by the signal separation unit 161.

[0085] As described above, each separation signal contained in the separation signal information Sep(t) comprises several pulse wave origin signal segments that are generated on the basis of each pulse wave origin signal wi(t) of each measurement range ri(k), so that the recovery unit 162 can recover the pulse wave origin signal wi(t) for each measurement range ri(k) from the multiple separation signals contained in the separation signal information Sep(t).

[0086] When the recovery unit 162 recovers the pulse wave origin signal wi(t) per measurement range ri(k), it generates post-recovery pulse wave origin signal information RW(t) that represents the recovered pulse wave origin signal wi(t) per measurement range ri(k).

[0087] The post-recovery pulse wave origin signal information RW(t) includes the recovered pulse wave origin signal wi(t) (hereinafter referred to as "post-recovery pulse wave origin signal") for each measurement range ri(k).

[0088] The recovery unit 162 outputs the generated pulse wave origin signal information after recovery RW(t) to the assessment unit 163.

[0089] The assessment unit 163 assesses the test subject's pulse wave based on the post-recovery pulse wave origin signal information RW(t) provided by the recovery unit 162.

[0090] The assessment unit 163 outputs the pulse wave assessment result P(t), which is pulse wave information representing the assessed pulse wave, to the output unit 17.

[0091] The pulse wave information could, for example, consist of time series data of the test subject's pulse wave as assessed by assessment unit 163, the test subject's pulse rate, or the test subject's pulse interval. For the sake of simplicity, it is assumed here that the pulse wave information is the test subject's pulse rate (number of beats per minute).

[0092] The procedure for assessing the pulse wave of the test subject by the assessment unit 163 is described in detail.

[0093] For example, the assessment unit 163 determines the signal-to-noise ratio (S / N) of the pulse wave source signal after reconstructing each measurement range ri(k). The assessment unit 163 calculates synthetic pulse wave signal information D(t) by summing the restored pulse wave source signals corresponding to each measurement range ri(k), after weighting them based on the S / N ratio determined for the pulse wave source signals after reconstructing each measurement range ri(k). That is, the assessment unit 163 calculates the synthetic pulse wave signal information D(t) for all measurement ranges ri(k). Since the weighting is performed based on the S / N ratio, the synthetic pulse wave signal information D(t) is assumed to be signals typical of pulse wave components, with the noise components removed.

[0094] The assessment unit 163 then performs a Fourier transform of the synthetic pulse wave signal information D(t) and calculates the peak frequency in the frequency power spectrum within a predefined frequency range as the pulse frequency. The predefined frequency range is set taking into account the range of the human heart rate.

[0095] Output unit 17 outputs the pulse wave assessment result P(t) output by pulse wave assessment unit 16, for example, to a state assessment device. The function of output unit 17 can also be implemented within pulse wave assessment unit 16.

[0096] The operation of the pulse wave assessment device 1 according to embodiment 1 is described.

[0097] Fig. Figure 7 is a flowchart to explain the operation of the pulse wave assessment device 1 according to embodiment 1.

[0098] The pulse wave assessment device 1, for example, repeats the process shown in the flowchart of Fig. 7. Processing shown, from when the vehicle is switched on until the vehicle is switched off.

[0099] The image acquisition unit 11 acquires an image from the imaging device 3, on which a test subject is depicted (step ST1).

[0100] The image acquisition unit 11 outputs the acquired image to the skin area detection unit 12.

[0101] The skin area detection unit 12 identifies the skin area of ​​the test subject from frame Im(k), which is contained in the image acquired by the image acquisition unit 11 in step ST1 (step ST2). The skin area detection unit 12 outputs the generated skin area information S(k) to the measurement range setting unit 13.

[0102] Based on the frame Im(k) of the image acquired in step ST1 by the image acquisition unit 11 and the skin area information S(k) output by the skin area detection unit 12 in step ST2, the measurement range setting unit 13 sets several measurement ranges ri(k) in the image area in frame Im(k) corresponding to the skin area represented by the skin area information S(k) in order to extract the pulse wave origin signal wi(t), which represents luminance changes in time series in the first period (step ST3). The measurement range setting unit 13 outputs the generated measurement range information R(k) to the pulse wave origin signal extraction unit 14.

[0103] The pulse wave origin signal extraction unit 14 extracts, on the basis of the frame Im(k) of the image acquired in step ST1 by the image acquisition unit 11 and the measurement range information R (k) output in step ST3 by the measurement range setting unit 13, from each of the several measurement ranges ri(k) represented by the measurement range information R (k) in frame lm (k), a pulse wave origin signal wi(t) representing luminance changes in the first period (step ST4).

[0104] The pulse wave origin signal extraction unit 14 generates pulse wave origin signal information W(t) that represents a pulse wave origin signal wi(t) in each measurement range ri(k).

[0105] The pulse wave origin signal extraction unit 14 outputs the generated pulse wave origin signal information W(t) to the segment generation unit 15.

[0106] The segment generation unit 15 generates several pulse wave origin signal segments per measurement range ri(k) according to the segment generation conditions, based on the pulse wave origin signal information W(t) output by the pulse wave origin signal extraction unit 14 in step ST4, where these are signals in which the pulse wave origin signal wi(t) for the second period has been partially extracted from the pulse wave origin signal wi(t) of the first period (step ST5).

[0107] When the segment generation unit 15 generates multiple pulse wave origin signal segments for each measurement range ri(k), it generates a trajectory matrix from the generated pulse wave origin signal segments and stacks them in a storage unit.

[0108] When the segment generation unit 15 stacks the trajectory matrix, it informs the pulse wave estimation unit 16.

[0109] When the pulse wave estimation unit 16 is informed by the segment generation unit 15 in step ST5 that the trajectory matrix has been stacked, it performs pulse wave estimation processing to estimate the test subject's pulse wave based on the multiple pulse wave origin signal segments generated for each measurement range ri(k) (step ST6).

[0110] The pulse wave assessment unit 16 outputs the pulse wave assessment result P(t), which is pulse wave information representing the assessed pulse wave, to the output unit 17.

[0111] The output unit 17 outputs the pulse wave assessment result P(t) output by the pulse wave assessment unit 16, for example to a state assessment device.

[0112] Fig. Figure 8 is a flowchart to explain the details of the pulse wave assessment processing by the pulse wave assessment unit 16 in step ST6 of Fig. 7.

[0113] The signal separation unit 161 generates separation signals based on several pulse wave origin signal segments, which represent several main components (step ST11).

[0114] The signal separation unit 161 outputs the generated multiple separation signals to the recovery unit 162, specifically the separation signal information Sep(t) regarding the separation signals representing the multiple generated main components.

[0115] The recovery unit 162 restores the pulse wave origin signal wi(t) per measurement range ri(k) based on the separation signal information Sep(t) output by the signal separation unit 161 in step ST11 (step ST12).

[0116] The recovery unit 162 outputs the pulse wave origin signal information after recovery RW(t) to the assessment unit 163.

[0117] Based on the pulse wave origin signal information provided by the recovery unit 162 in step ST12, the assessment unit 163 assesses the pulse wave of the test subject after recovery RW(t) (step ST13).

[0118] The assessment unit 163 outputs the pulse wave assessment result P(t), which is pulse wave information representing the assessed pulse wave, to the output unit 17.

[0119] The output unit 17 outputs the pulse wave assessment result P(t) output by the pulse wave assessment unit 16, for example to a state assessment device.

[0120] In this way, the pulse wave assessment device 1 according to embodiment 1 sets several measurement ranges ri(k) in an area corresponding to the skin area on an image image that was detected from the image image depicting a test subject, and for each measurement range ri(k) the pulse wave origin signal wi(t) is extracted in the respective measurement range ri(k) on the basis of the luminance changes in the first period, which represents the period for performing a pulse wave assessment.For each measurement range ri(k), the pulse wave estimation device 1 generates several pulse wave origin signal segments based on the pulse wave origin signal wi(t) extracted from this measurement range ri(k), according to the segment generation conditions, in which the pulse wave origin signal wi(t) for the second period was partially extracted from the pulse wave origin signal wi(t) of the first period, and estimates the pulse wave of the test subject based on the several pulse wave origin signal segments.

[0121] Because the pulse wave assessment device 1 generates multiple pulse wave origin signal segments based on the pulse wave origin signal wi(t) of the first period, from which the pulse wave origin signal wi(t) for the second period (shorter than the first period) has been partially extracted, the pulse wave origin signal wi(t) can exist in a state of in-phase pairs. Since the pulse wave assessment device 1 assesses the test subject's pulse wave based on the pulse wave origin signal wi(t) in a state of in-phase pairs, i.e., multiple pulse wave origin signal segments, the separation of the pulse wave components can be performed flawlessly for multiple pulse wave origin signals wi(t) using known general signal separation techniques such as ICA or PCA, thus enabling the assessment of the test subject's pulse wave.In this way, even if a phase difference occurs in the luminance signals extracted from the multiple measurement ranges ri(k), the pulse wave assessment device 1 can assess the pulse wave of a test subject based on the luminance signals.

[0122] Fig. 9A and Fig. Figure 9B shows examples of the hardware setup of the pulse wave assessment device 1 according to embodiment 1.

[0123] In embodiment 1, the functions of the image acquisition unit 11, the skin area detection unit 12, the measuring range setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, the pulse wave assessment unit 16 and the output unit 17 are implemented by a processing circuit 101.That is, the pulse wave assessment device 1 comprises the processing circuit 101 to generate multiple pulse wave origin signal segments, in which a pulse wave origin signal wi(t), representing luminance changes in the multiple measurement ranges ri(k) set for a skin area of ​​a test subject in an imaging image acquired in the first period, has been partially extracted using techniques such as independent component analysis or principal component analysis to separate pulse wave components from the pulse wave origin signal segments and to control the assessment of a test subject's pulse wave based on the separated pulse wave components.

[0124] The processing circuit 101 may be special hardware, as in Fig. 9A shown, or to use a processor 104 that executes a program stored in memory, as in Fig. 9B shown.

[0125] If the processing circuit 101 is special hardware, the processing circuit 101 can be, for example, a single circuit, a complex circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0126] If the processing circuit is a processor 104, the functions of the image acquisition unit 11, the skin area detection unit 12, the measurement range setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, the pulse wave assessment unit 16, and the output unit 17 are implemented by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in a memory 105. The processor 104 reads the program stored in the memory 105 and executes it to perform the functions of the image acquisition unit 11, the skin area detection unit 12, the measurement range setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, the pulse wave assessment unit 16, and the output unit 17.That is, the pulse wave assessment device 1 includes memory 105 to store a program with which steps ST1 to ST6 of . Fig. 7 is ultimately executed by the processor 104. Furthermore, it can also be said that the program stored in memory 105 causes the computer to execute the procedures or processes of the image acquisition unit 11, the skin area detection unit 12, the measurement range setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16, and the output unit 17. Here, the term memory 105 refers, for example, to RAM, ROM (Read Only Memory), Flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), or similar non-volatile or volatile semiconductor memories, magnetic disks, flexible disks, optical disks, compact discs, mini-discs, DVDs (Digital Versatile Discs), etc.

[0127] Furthermore, the functions of the image acquisition unit 11, the skin area detection unit 12, the measurement range setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, the pulse wave assessment unit 16, and the output unit 17 can be implemented partly with special hardware and partly with software or firmware. For example, the functions of the image acquisition unit 11 and the output unit 17 can be implemented by the processing circuit 101 as special hardware, while the functions of the skin area detection unit 12, the measurement range setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, and the pulse wave assessment unit 16 can be implemented by the processor 104 reading and executing a program stored in memory 105.An unspecified storage unit is formed, for example, by memory 105.

[0128] Furthermore, the pulse wave assessment device 1 includes an input interface device 102 and an output interface device 103 for wired or wireless communication with devices such as the imaging device 3.

[0129] Furthermore, in the embodiment 1 above, the test subject is the driver of a vehicle, but this is only an example. Test subjects can also be other occupants besides the driver of the vehicle.

[0130] Furthermore, in the above embodiment 1, the pulse wave assessment device 1 is an in-vehicle device, and the image acquisition unit 11, the skin area detection unit 12, the measuring range setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, the pulse wave assessment unit 16 and the output unit 17 are provided in the in-vehicle device.

[0131] Furthermore, some of the imaging unit 11, the skin area detection unit 12, the measurement range setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16 and the output unit 17 may be installed in the in-vehicle device, while others may be provided in a server connected to the in-vehicle device via a network, and the system may consist of the in-vehicle device and the server.

[0132] Furthermore, the image acquisition unit 11, the skin area detection unit 12, the measurement range setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, the pulse wave assessment unit 16 and the output unit 17 can all also be provided on the server.

[0133] Furthermore, the pulse wave assessment device 1 according to the above embodiment 1 is not limited to in-vehicle devices installed in vehicles, but can also be used, for example, for household appliances. Furthermore, the test subject can be various individuals, not just vehicle occupants.

[0134] To give a concrete example, the pulse wave assessment device 1 can be installed on a television set located in the living room of an apartment. In this case, the test subject is a resident of the apartment or another user. The pulse wave assessment device 1 assesses the user's pulse wave based on the image projected by the imaging device installed on the television set.

[0135] As described above, the pulse wave assessment device 1 according to embodiment 1 comprises an image acquisition unit 11, which acquires an image depicting a person (a test subject); a skin area detection unit 12, which detects a skin area of ​​the person (the test subject) based on the image; a measurement range setting unit 13, which sets several measurement ranges ri(k) in an area corresponding to the skin area on the image in order to extract a pulse wave origin signal wi(t) representing luminance changes in time series in a first period; a pulse wave origin signal extraction unit 14, which extracts a pulse wave origin signal wi(t) for each measurement range ri(k) based on the luminance changes in the first period in the respective measurement range ri(k); and a segment generation unit 15.The device generates several pulse wave origin signal segments for each measurement range ri(k) based on the pulse wave origin signal wi(t) extracted from the measurement range, according to the segment generation conditions. These segments are used to partially extract a pulse wave origin signal wi(t) for the second period from the pulse wave origin signal wi(t) of the first period. A pulse wave estimation unit 16 is also included, which estimates the pulse wave of the person (the test subject) based on the multiple pulse wave origin signal segments generated for each measurement range ri(k). Therefore, even if a phase difference occurs in the luminance signals extracted from the multiple measurement ranges ri(k), the pulse wave estimation device 1 can still estimate a person's pulse wave based on the luminance signals. Design 2.

[0136] In embodiment 1, the length of the second period used to generate the pulse wave origin signal segments was uniform.

[0137] In embodiment 2, an embodiment is described in which the length of the second period used to generate the pulse wave origin signal segments is regulated.

[0138] Furthermore, in the following embodiment 2, as in embodiment 1, it is assumed that the pulse wave assessment device is installed in a vehicle and that the test subject is the driver of the vehicle.

[0139] Fig. Figure 10 is a drawing showing a structural example of a pulse wave assessment device 1a according to embodiment 2.

[0140] Regarding the construction of the pulse wave assessment device 1a according to embodiment 2, the same structural elements are used as in embodiment 1, using Fig. The same symbols are used for the pulse wave assessment device described in section 1, and a redundant explanation is omitted.

[0141] The embodiment 2 pulse wave assessment device 1a differs from the embodiment 1 pulse wave assessment device 1 in that it is equipped with a parameter setting unit 18.

[0142] The parameter setting unit 18 sets segment parameters based on the pulse wave of the test subject assessed by the pulse wave assessment unit 16. These parameters contain at least information representing the length of the second period. More precisely, based on the pulse wave of the test subject assessed by the pulse wave assessment unit 16, the parameter setting unit 18 calculates the length of the second period such that the second period corresponds to the time of one cycle of the test subject's pulse wave and sets segment parameters containing information representing the calculated length of the second period.

[0143] Furthermore, the second period does not necessarily have to correspond to the time of a cycle of the test subject's pulse wave, as long as the parameter setting unit 18 calculates the second period such that the second period is longer than the time of a cycle of the test subject's pulse wave.

[0144] Furthermore, in addition to information about the length of the second period, parameter setting unit 18 also sets information about the size of the partial overlap of the second period as a segment parameter. Parameter setting unit 18 can adjust the size of the overlap between pulse wave origin signal segments so that it is variable depending on the length of the pulse wave origin signal segment from the perspective of the processing load.

[0145] The parameter setting unit 18 outputs the set segment parameters to the segment generation unit 15.

[0146] In embodiment 2, the assessment unit 163 outputs the pulse wave assessment result P(t), which is pulse wave information representing the assessed pulse wave, to the output unit 17 and the parameter setting unit 18.

[0147] Furthermore, in embodiment 2, the segment generation unit 15 generates pulse wave origin signal segments based on the segment parameters and using the second period set by the parameter setting unit 18.

[0148] Specifically, the segment generation unit 15, after adjusting the length of the second period set by the parameter setting unit 18 to the length of the second period used to generate the pulse wave origin signal segments, generates several pulse wave origin signal segments per measurement range ri(k) according to the segment generation conditions, in which the pulse wave origin signal wi(t) for the second period has been partially extracted from the pulse wave origin signal wi(t) of the first period.

[0149] Furthermore, the information about the length of the second period, set by the parameter setting unit 18, is used by the segment generation unit 15 to generate the pulse wave origin signal segments after the parameter setting unit 18 has set the segment parameters. Consequently, it is possible that the segment parameters have not yet been received from the parameter setting unit 18 when the segment generation unit 15 generates the pulse wave origin signal segments, for example, immediately after the pulse wave assessment of a test subject by the pulse wave assessment device 1a has begun. In this case, the segment generation unit 15 can generate the pulse wave origin signal segments using initial values ​​for the second period that were previously set by an administrator or the like and stored by the segment generation unit 15.If the parameter setting unit 18 subsequently receives information for setting segment parameters, i.e., results of the pulse wave assessment, the parameter setting unit 18 either continues setting the segment parameters that were last set among the segment parameters set in the past, or resets the second period to the initial values.

[0150] The operation of the pulse wave assessment device 1a according to embodiment 2 is described.

[0151] Fig. Figure 11 is a flowchart to explain the operation of the pulse wave assessment device 1a according to embodiment 2.

[0152] The pulse wave assessment device 1a, for example, repeats the process shown in the flowchart of Fig. 11. Processing shown, when the vehicle is switched on, until the vehicle is switched off.

[0153] In Fig. 11. The specific processes of steps ST21 to ST24 and steps ST28 to ST29 are similar to the specific processes of steps ST1 to ST6 of Fig. 7, which have already been explained in embodiment 1, therefore a redundant explanation is omitted.

[0154] When the pulse wave origin signal information W(t) is output by the pulse wave origin signal extraction unit 14 in step ST24, the segment generation unit 15 determines whether the segment parameters have already been set by the parameter setting unit 18, in other words, whether the segment parameters have been output by the parameter setting unit 18 (step ST25).

[0155] If the segment generation unit 15 determines that the segment parameters have already been set (“YES” in step ST25), the segment generation unit 15, based on the segment parameters, sets the second period length set by the parameter setting unit 18 to the second period length used to generate the pulse wave origin signal segments (step ST26).

[0156] If the segment generation unit 15 determines that the segment parameters have not yet been set (“NO” in step ST25), the segment generation unit 15 sets initial values ​​for the second period (step ST27).

[0157] When the pulse wave assessment unit 16 assesses the test subject's pulse wave in step ST 29, the parameter setting unit 18 sets segment parameters based on the test subject's pulse wave assessed by the pulse wave assessment unit 16, which contain information representing the length of the second period (step ST 30).

[0158] In this way, the pulse wave assessment device 1a calculates the second period based on the test subject's pulse wave assessed by the pulse wave assessment unit 16. If segment parameters are set that contain information representing the length of the second period, it adjusts the second period according to the set segment parameters. The pulse wave assessment device 1a then uses the second period set according to the segment parameters to generate several pulse wave origin signal segments, in which the pulse wave origin signal wi(t) for the second period has been partially extracted from the pulse wave origin signal wi(t) of the first period.

[0159] Pulse wave assessment requires the recording of repeatedly occurring pulse wave components of the test subject. However, a test subject's pulse wave cycle varies depending on their condition, the nature of the test subject, and other factors. For example, if the test subject's condition changes, even if it is the same test subject, or if the test subject changes, the time for one pulse wave cycle, as assessed by the pulse wave assessment device 1a, will also change. For instance, if the test subject's pulse rate is 60 bpm (beats per minute), the time for one pulse wave cycle will be one second, and if the test subject's pulse rate is 100 bpm, the time for one pulse wave cycle will be 0.6 seconds.

[0160] The pulse wave assessment device 1a according to embodiment 2 makes it possible to regulate the length of the second period, which is used to generate the pulse wave origin signal segments for assessing the test subject's pulse wave, to a length that takes into account the assessed pulse wave cycle of the test subject. This allows the pulse wave assessment device 1a to assess a test subject's pulse wave with greater accuracy than when assessing a test subject's pulse wave from pulse wave origin signal segments generated using a second period set without considering the test subject's pulse wave cycle.

[0161] In the embodiment 2 above, it is assumed that once the pulse wave estimation device 1a begins estimating the pulse wave of the test subject, it continues to estimate the pulse wave of the test subject for a certain period to such an extent that the first period is repeated several times, and the parameter setting unit 18 calculates the second period based on the pulse wave of the test subject that was estimated by the estimation unit 163 in the previous first period, but this is only an example.For example, the pulse wave assessment result P(t) of a test subject, previously assessed by the pulse wave assessment device 1a, can be stored in a memory unit before the pulse wave assessment device 1a begins assessing the test subject's pulse wave. The parameter setting unit 18 can then calculate the second period based on the pulse wave assessment result P(t) stored in the memory unit. For example, the test subject may be able to record information about the test subject's average pulse wave in the memory unit, and the parameter setting unit 18 may be able to calculate the second period based on the information about the test subject's average pulse wave recorded in the memory unit.

[0162] For example, the parameter setting unit 18 can also calculate the second period from the pulse wave assessment result P(t) obtained with reference to the storage unit or from the average pulse wave of the test subject and the pulse wave of the test subject assessed by the assessment unit 163.

[0163] The hardware design of the pulse wave assessment device 1a according to embodiment 2 is similar to the hardware design of the pulse wave assessment device 1, which in embodiment 1 is based on Fig. 9A and Fig. Section 9B has been described, so its presentation is omitted.

[0164] In embodiment 2, the functions of the image acquisition unit 11, the skin area detection unit 12, the measuring range setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, the pulse wave assessment unit 16, the output unit 17 and the parameter setting unit 18 are implemented by the processing circuit 101.That is, the pulse wave assessment device 1a includes the processing circuit 101 to generate multiple pulse wave origin signal segments, in which a pulse wave origin signal wi(t), representing luminance changes in the multiple measurement ranges ri(k) set for a skin area of ​​a test subject in an imaging image acquired in the first period, has been partially extracted using techniques such as independent component analysis or principal component analysis to separate pulse wave components from the pulse wave origin signal segments and to control the assessment of a test subject's pulse wave based on the separated pulse wave components.

[0165] The processing circuit 101 reads the program stored in memory 105 and executes it to perform the functions of the image acquisition unit 11, the skin area detection unit 12, the measurement range setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, the pulse wave assessment unit 16, the output unit 17, and the parameter setting unit 18. That is, the pulse wave assessment device 1a includes memory 105 to store a program that executes steps ST21 to ST30 of Fig. 11 are finally executed by the processing circuit 101. Furthermore, it can also be said that the program stored in memory 105 causes the computer to execute the procedures or processes of the image acquisition unit 11, the skin area detection unit 12, the measurement range setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, the pulse wave assessment unit 16, the output unit 17 and the parameter setting unit 18.

[0166] An unspecified storage unit is formed, for example, by memory 105.

[0167] Furthermore, the pulse wave assessment device 1a includes an input interface device 102 and an output interface device 103 for wired or wireless communication with devices such as the imaging device 3.

[0168] Furthermore, in the embodiment 2 above, the test subject is the driver of a vehicle, but this is only an example. Test subjects can also be other occupants besides the driver of the vehicle.

[0169] Furthermore, in the above embodiment 2, the pulse wave assessment device 1a is an in-vehicle device, and the image acquisition unit 11, the skin area detection unit 12, the measuring range setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, the pulse wave assessment unit 16, the output unit 17 and the parameter setting unit 18 are provided in the in-vehicle device.Furthermore, some of the imaging unit 11, skin area detection unit 12, measurement range setting unit 13, pulse wave origin signal extraction unit 14, segment generation unit 15, pulse wave assessment unit 16, output unit 17, and parameter setting unit 18 can be installed in the vehicle's internal device, while others are provided on a server connected to the vehicle's internal device via a network. The system can consist of the vehicle's internal device and the server. Alternatively, the imaging unit 11, skin area detection unit 12, measurement range setting unit 13, pulse wave origin signal extraction unit 14, segment generation unit 15, pulse wave assessment unit 16, output unit 17, and parameter setting unit 18 can all be provided on the server.

[0170] Furthermore, the pulse wave assessment device 1a according to embodiment 2 above is not limited to in-vehicle devices installed in vehicles, but can also be used, for example, for household appliances. Furthermore, the test subject can be various individuals, not just vehicle occupants.

[0171] As described above, the pulse wave assessment device 1a according to embodiment 2 is, in addition to the structure of the pulse wave assessment device 1 according to embodiment 1, designed to include a parameter setting unit 18 which, based on the pulse wave of a person (a test subject) assessed by the pulse wave assessment unit 16, calculates the second period and sets segment parameters containing information representing the length of this second period, wherein the segment generation unit 15 sets the second period according to the segment parameters. Therefore, the pulse wave assessment device 1a can assess a person's pulse wave with greater accuracy than in the case of an assessment of a person's pulse wave from pulse wave origin signal segments generated using a second period set without considering the person's pulse wave cycle.

[0172] More precisely, in the pulse wave assessment device 1a, the parameter setting unit 18 calculates the second period such that the second period corresponds to the time of one cycle of the pulse wave of the person (the test subject) assessed by the pulse wave assessment unit 16, and sets segment parameters containing information representing the length of this second period. The segment generation unit 15 then sets the second period according to the segment parameters. Therefore, the pulse wave assessment device 1a can assess a person's pulse wave with greater accuracy than if the person's pulse wave were assessed from pulse wave origin signal segments generated using a second period set without considering the person's pulse wave cycle. Design 3.

[0173] In embodiment 1, the pulse wave assessment device uses the generated multiple pulse wave origin signal segments unchanged to assess the pulse wave of the test subject.

[0174] Embodiment 3 describes an embodiment in which the pulse waves of a test subject are assessed after the multiple pulse wave origin signal segments have been weighted.

[0175] Furthermore, in the following embodiment 3, as in embodiment 1, it is assumed that the pulse wave assessment device is installed in a vehicle and that the test subject is the driver of the vehicle.

[0176] Fig. Figure 12 is a drawing showing a structural example of a pulse wave assessment device 1b according to embodiment 3.

[0177] Regarding the construction of the pulse wave assessment device 1b according to embodiment 3, the same structural elements are used as in embodiment 1, using Fig. The same symbols are used for the pulse wave assessment device described in section 1, and a redundant explanation is omitted.

[0178] The embodiment 3 pulse wave assessment device 1b differs from the embodiment 1 pulse wave assessment device 1 in that it is equipped with a weighting coefficient calculation unit 19.

[0179] The weighting coefficient calculation unit 19 calculates weighting coefficients (hereinafter referred to as "segment weights") for each of the multiple pulse wave origin signal segments that were generated per measurement range ri(k) by the segment generation unit 15.

[0180] More precisely, the weighting coefficient calculation unit calculates 19 segment weights for each of the multiple pulse wave origin signal segments based on the magnitude of a signal fluctuation in the pulse wave origin signal wi(t) from which the pulse wave origin signal segments were extracted.

[0181] Furthermore, in embodiment 3, the segment generation unit 15 outputs the generated information about the multiple pulse wave origin signal segments of each measurement range ri(k) and the pulse wave origin signal information W(t) generated by the pulse wave origin signal extraction unit 14 to the weighting coefficient calculation unit 19.

[0182] The weighting coefficient calculation unit 19 determines, for example, whether the pulse wave origin signal wi(t) in first-period time series from which the pulse wave origin signal segments were extracted has fluctuated by more than a preset threshold (hereinafter referred to as the "fluctuation threshold"). If it is determined that the pulse wave origin signal wi(t) has fluctuated by more than the fluctuation threshold, the weighting coefficient calculation unit 19 calculates the segment weight "0" for the pulse wave origin signal segments that were extracted within the pulse wave origin signal wi(t) in first-period time series from the portion of the pulse wave origin signal in which the fluctuation occurred.It can be said that the portion of the pulse wave origin signal wi(t) that fluctuated by more than the fluctuation threshold is a signal containing a lot of noise. The weighting coefficient calculation unit 19 sets the segment weight "1" for the pulse wave origin signal segments extracted from the portion of the pulse wave origin signal wi(t) that is not the part where the above fluctuation occurred.

[0183] On the other hand, if the weighting coefficient calculation unit 19 determines that the pulse wave origin signal wi(t) in time series of the first period from which the pulse wave origin signal segments were extracted has not fluctuated by more than the threshold for fluctuation determination, it calculates the segment weight “1” for all pulse wave origin signal segments that were extracted from the pulse wave origin signal wi(t) in time series of the first period without such fluctuations.

[0184] The weighting coefficient calculation unit 19 outputs information about the segment weights (hereinafter referred to as “segment weighting information”) calculated for each of the multiple pulse wave origin signal segments generated for each measurement range ri(k) by the segment generation unit 15 to the pulse wave estimation unit 16.

[0185] The segment weighting information is information that can identify the measurement ranges ri(k), information that can identify the pulse wave origin signal segments, and information about the segment weightings of these pulse wave origin signal segments.

[0186] The segment weights calculated by the weighting coefficient calculation unit 19 are an indicator for the pulse wave assessment unit 16 to determine whether the pulse wave origin signal segments generated by the segment generation unit 15 should be used in the assessment of a test subject's pulse wave.

[0187] The pulse wave assessment unit 16 ensures that pulse wave origin signal segments with a segment weight of "0" are not used to assess the test subject's pulse wave. More precisely, the signal separation unit 161 of the pulse wave assessment unit 16 discards the pulse wave origin signal segments assigned a segment weight of "0", analyzes the multiple principal components based on the remaining multiple pulse wave origin signal segments using general signal separation techniques such as PCA or ICA, and generates separation signals representing the analyzed multiple principal components.

[0188] In this way, the pulse wave estimation unit 16 in embodiment 3 estimates the pulse wave of a test subject on the basis of the pulse wave origin signal segments and the segment weightings.

[0189] Fig. Figure 13 is a drawing to illustrate the pulse wave origin signal segments which, in embodiment 3, are used by the pulse wave assessment unit 1b to assess the pulse wave of a test subject.

[0190] Fig. Figure 13 is a diagram showing on the time axis the luminance values ​​represented by the pulse wave origin signal wi(t) in first period time series extracted from a specific measurement range ri(k) of the several measurement ranges ri(k).

[0191] In Fig. 13 Within the pulse wave origin signal wi(t) in time series of the first period, the pulse wave origin signal wi(t) of the area enclosed by a dashed line shows a large signal fluctuation.

[0192] The weighting coefficient calculation unit 19 calculates a segment weight of “0” for the pulse wave origin signal segments within the multiple pulse wave origin signal segments generated by the segment generation unit 15 for a specific measurement range ri(k), which were extracted from the area of ​​the pulse wave origin signal wi(t) enclosed by the dashed line.

[0193] The weighting coefficient calculation unit 19 calculates a segment weight of “1” for the pulse wave origin signal segments within the multiple pulse wave origin signal segments generated by the segment generation unit 15 for a specific measurement range ri(k) that were extracted outside the area of ​​the pulse wave origin signal wi(t) enclosed by the dashed line.

[0194] As a result, the pulse wave assessment unit 16 does not use the pulse wave origin signal segments extracted from the area of ​​the pulse wave origin signal wi(t) enclosed by the dashed line, whose segment weight is “0”, to assess the test subject's pulse wave.

[0195] The operation of the pulse wave assessment device 1b according to embodiment 3 is described.

[0196] Fig. Figure 14 is a flowchart to explain the operation of the pulse wave assessment device 1b according to embodiment 3.

[0197] The pulse wave assessment device 1b, for example, repeats the process shown in the flowchart of Fig. 14. Processing shown, when the vehicle is switched on, until the vehicle is switched off.

[0198] In Fig. 14. The specific processes of steps ST31 to ST35 are similar to the specific processes of steps ST1 to ST5 of Fig. 7, which have already been explained in embodiment 1, therefore a redundant explanation is omitted.

[0199] The weighting coefficient calculation unit 19 calculates segment weights for each of the multiple pulse wave origin signal segments that were generated by the segment generation unit 15 in step ST35 (step ST36).

[0200] More precisely, the weighting coefficient calculation unit calculates 19 segment weights for each of the multiple pulse wave origin signal segments based on the magnitude of a signal fluctuation in the pulse wave origin signal wi(t) from which the pulse wave origin signal segments were extracted.

[0201] The weighting coefficient calculation unit 19 outputs the segment weighting information to the pulse wave assessment unit 16.

[0202] When the pulse wave assessment unit 16 is informed by the segment generation unit 15 in step ST35 that the trajectory matrix has been stacked, it performs pulse wave assessment processing to assess the subject's pulse wave based on the multiple pulse wave origin signal segments generated for each measurement range ri(k). At this point, the pulse wave assessment unit 16 ensures that the pulse wave origin signal segments assigned a segment weight of "0" based on segment weighting information provided by the weighting coefficient calculation unit are not used to assess the subject's pulse wave (step ST37). Furthermore, the pulse wave assessment unit 16 may receive notification that the trajectory matrix has been stacked, for example, from the weighting coefficient calculation unit 19.

[0203] More precisely, the signal separation unit 161 discards the separation signals during their generation (see step ST11 of Fig. 8) analyzes the pulse wave origin signal segments with segment weighting “0”, analyzes the multiple principal components based on the remaining multiple pulse wave origin signal segments using general signal separation techniques such as PCA or ICA, and generates separation signals that represent the analyzed multiple principal components.

[0204] In this way, the pulse wave estimation unit 1b calculates segment weights for each of the multiple pulse wave source signal segments and estimates a test subject's pulse wave based on the pulse wave source signal segments and segment weights. More precisely, the pulse wave estimation device 1b calculates segment weights for each of the multiple pulse wave source signal segments based on the magnitude of the signal fluctuation in the pulse wave source signal wi(t) from which the pulse wave source signal segments were extracted.

[0205] This allows the pulse wave assessment device 1b to assess a test subject's pulse wave while excluding noise components from the pulse wave source signal wi(t) extracted from the multiple measurement ranges ri(k). Therefore, the pulse wave assessment device 1b can assess a test subject's pulse wave with greater accuracy than if it does not consider whether the pulse wave source signal wi(t) extracted from the multiple measurement ranges ri(k) contains many noise components.

[0206] The hardware design of the pulse wave assessment device 1b according to embodiment 3 is similar to the hardware design of the pulse wave assessment device 1, which in embodiment 1 is based on Fig. 9A and Fig. Section 9B has been described, so its presentation is omitted.

[0207] In embodiment 3, the functions of the image acquisition unit 11, the skin area detection unit 12, the measuring range setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, the pulse wave assessment unit 16, the output unit 17 and the weighting coefficient calculation unit 19 are implemented by the processing circuit 101.That is, the pulse wave assessment device 1b comprises the processing circuit 101 to generate multiple pulse wave origin signal segments, in which a pulse wave origin signal wi(t), representing luminance changes in the multiple measurement ranges ri(k) set for a skin area of ​​a test subject in an imaging image acquired in the first period, has been partially extracted using techniques such as independent component analysis or principal component analysis to separate pulse wave components from the pulse wave origin signal segments and to control the assessment of a test subject's pulse wave based on the separated pulse wave components.

[0208] The processing circuit 101 reads the program stored in memory 105 and executes it to perform the functions of the image acquisition unit 11, the skin area detection unit 12, the measurement range setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16, the output unit 17, and the weighting coefficient calculation unit 19. That is, the pulse wave estimation device 1b includes memory 105 to store a program that executes steps ST31 to ST37 of Fig. 14 are finally executed by the processing circuit 101. Furthermore, it can also be said that the program stored in memory 105 causes the computer to execute the procedures or processes of the image acquisition unit 11, the skin area detection unit 12, the measurement range setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, the pulse wave assessment unit 16, the output unit 17, and the weighting coefficient calculation unit 19.

[0209] An unspecified storage unit is formed, for example, by memory 105.

[0210] Furthermore, the pulse wave assessment device 1b includes an input interface device 102 and an output interface device 103 for wired or wireless communication with devices such as the imaging device 3.

[0211] Furthermore, in embodiment 3 above, the test subject is the driver of a vehicle, but this is only an example. Test subjects can also be other occupants besides the driver of the vehicle.

[0212] Furthermore, in the above embodiment 3, the pulse wave assessment device 1b is an in-vehicle device, and the image acquisition unit 11, the skin area detection unit 12, the measuring range setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, the pulse wave assessment unit 16, the output unit 17 and the weighting coefficient calculation unit 19 are provided in the in-vehicle device.

[0213] Furthermore, some of the imaging unit 11, the skin area detection unit 12, the measurement range setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, the pulse wave estimation unit 16, the output unit 17 and the weighting coefficient calculation unit 19 may be installed in the in-vehicle device, while others may be provided in a server connected to the in-vehicle device via a network, and the system may consist of the in-vehicle device and the server.

[0214] Furthermore, the image acquisition unit 11, the skin area detection unit 12, the measurement range setting unit 13, the pulse wave origin signal extraction unit 14, the segment generation unit 15, the pulse wave assessment unit 16, the output unit 17 and the weighting coefficient calculation unit 19 can all also be provided on the server.

[0215] Furthermore, the pulse wave assessment device 1b according to embodiment 3 above is not limited to in-vehicle devices installed in vehicles, but can also be used, for example, for household appliances. Furthermore, the test subject can be various individuals, not just vehicle occupants.

[0216] As described above, the pulse wave assessment device 1b according to embodiment 3, in addition to the structure of the pulse wave assessment device 1 according to embodiment 1, is designed to include a weighting coefficient calculation unit 19 for calculating weighting coefficients (segment weights) for each of the multiple pulse wave source signal segments, and the pulse wave assessment unit 16 assesses the pulse wave of a person (a test subject) based on the pulse wave source signal segments and the weighting coefficients (segment weights). Therefore, the pulse wave assessment device 1b can assess the pulse wave of a person with better accuracy than if it does not take into account whether the pulse wave source signal wi(t) extracted from the multiple measurement ranges ri(k) is a signal with many noise components.

[0217] More precisely, in the pulse wave assessment device 1b, the weighting coefficient calculation unit calculates 19 weighting coefficients (segment weights) for each of the multiple pulse wave source signal segments based on the magnitude of a signal fluctuation in the pulse wave source signal wi(t) from which the pulse wave source signal segments were extracted. Therefore, the pulse wave assessment device 1b can assess a person's pulse wave with greater accuracy than if it did not take into account whether the pulse wave source signal wi(t) extracted from the multiple measurement ranges ri(k) is a signal with many noise components.

[0218] In the above embodiments 1 to 3, the pulse wave estimation device 1, 1a, 1b can have the function of setting weighting coefficients (hereinafter referred to as “component weights”) for several separation signals representing several principal components that were estimated using known general signal separation techniques, and of restoring the pulse wave origin signal wi(t) for each measurement range ri(k) on the basis of the several separation signals and the set component weight for each separation signal.

[0219] Fig. Figure 15 is a drawing showing a structural example of a pulse wave estimation unit 16 in embodiments 1 to 3 in the case that the pulse wave estimation device 1, 1a, 1b has the function of setting component weights for several separation signals representing several main components that were estimated using known general signal separation techniques and of restoring the pulse wave origin signal wi(t) for each measurement range ri(k) on the basis of the several separation signals and the set component weight for each separation signal.

[0220] If the pulse wave assessment device 1, 1a, 1b has the above function, the pulse wave assessment device 1, 1a, 1b should be connected to a pulse wave assessment unit 16a, as shown in Fig. 15 shown, be equipped, instead of the pulse wave assessment unit 16 in the configuration example of the pulse wave assessment device 1, 1a, 1b, which uses Fig. 1, Fig. 10 or 12 in the above embodiments 1 to 3 has been explained.

[0221] The recovery unit 162 of the pulse wave assessment unit 16a includes a component weighting setting unit 1621, which sets a component weighting for each separation signal output by the signal separation unit 161.

[0222] The recovery unit 162 restores the pulse wave origin signal wi(t) for each measurement range ri(k) on the basis of the multiple separation signals output by the signal separation unit 161 and the component weighting set by the component weighting setting unit 1621 for each separation signal.

[0223] The assessment unit 163 assesses the pulse wave of the test subject based on the generated pulse wave origin signal wi(t), which was recovered by the recovery unit 162 based on the multiple separation signals and the component weighting per separation signal.

[0224] The following are exemplary procedures for setting the component weights using the component weight setting unit 1621.

[0225] For example, the component weighting setting unit 1621 sets a component weighting for each separator signal based on the similarity of the frequency characteristics between the multiple separator signals.

[0226] More precisely, the Component Weighting Setting Unit 1621 first calculates the peak frequency in the frequency-power spectrum of each separator signal by performing a fast Fourier transform or similar operation on each separator signal. The Component Weighting Setting Unit 1621 then forms pairs of adjacent separators with respect to the multiple separators representing multiple principal components and calculates the peak frequency difference of the adjacent separators (hereinafter referred to as the "peak difference"). Furthermore, the term "adjacent principal components" means separators that follow one another when the separators are arranged in order of their information content. The Component Weighting Setting Unit 1621 then determines whether the calculated peak difference is greater than a preset threshold (hereinafter referred to as the "peak difference determination threshold").The component weighting setting unit 1621 sets the component weighting of the separation signal for which the peak difference is calculated to “0” if the peak difference is greater than the threshold for peak difference determination, and sets the component weighting to “1” if the peak difference is less than the threshold for peak difference determination.

[0227] When the component weighting is "0", the similarity of the frequency characteristics between the separating signals is low; when the component weighting is "1", the similarity of the frequency characteristics between the separating signals is high. Since pulse wave components have the property of repeating themselves within a specific period, it can be assumed that the high similarity of the frequency characteristics means that the separating signal with this frequency characteristic contains many pulse wave components—in other words, that it is a separating signal typical of pulse wave components. The component weighting setting unit 1621 assigns larger component weights to the separating signals typical of pulse wave components, so that the assessment unit 163 can assess the test subject's pulse wave based on the separating signals typical of pulse wave components.

[0228] To give a concrete example, it is assumed that the several isolation signals output by the signal isolation unit 161 are C isolation signals, representing the first principal component c1, ... up to the C-th principal component cC, as shown by Fig. 5 and Fig. 6 in embodiment 1. Here, the isolation signal representing the first principal component c1 is the first isolation signal, the isolation signal representing the second principal component c2 is the second isolation signal, the isolation signal representing the third principal component c3 is the third isolation signal, the isolation signal representing the fourth principal component c4 is the fourth isolation signal, the isolation signal representing the fifth principal component c5 is the fifth isolation signal, ..., the isolation signal representing the (C-1)th principal component c(C-1) is the (C-1)th isolation signal, and the signal representing the Cth principal component cC is the Cth isolation signal.

[0229] In this case, the component weighting setting unit 1621 forms a pair from the first and second cutoff signals, calculates the peak difference between the peak frequency of the first cutoff signal and the peak frequency of the second cutoff signal, and determines whether the calculated peak difference is greater than the peak difference threshold. The component weighting setting unit 1621 sets the component weights of the first and second cutoff signals to "0" if the peak difference is greater than the peak difference threshold, and sets the component weights of the first and second cutoff signals to "1" if the peak difference is less than the peak difference threshold.

[0230] Furthermore, the component weighting setting unit 1621 forms a pair from the third and fourth cutoff signals, calculates the peak difference between the peak frequency of the third cutoff signal and the peak frequency of the fourth cutoff signal, and determines whether the calculated peak difference is greater than the threshold for peak difference determination. The component weighting setting unit 1621 sets the component weights of the third and fourth cutoff signals to "0" if the peak difference is greater than the threshold for peak difference determination, and sets the component weights of the third and fourth cutoff signals to "1" if the peak difference is less than the threshold for peak difference determination.

[0231] The component weighting setting unit 1621 also performs the processing described above for the pair of the fifth and sixth separation signals, the pair of the seventh and eighth separation signals, ... the pair of the (C-1)th and C-th separation signals and sets the component weights for all separation signals.

[0232] The recovery unit 162 discards, from the multiple separation signals output by the signal separation unit 161, the separation signals for which the component weighting setting unit 1621 has set the component weighting “0”, and restores the pulse wave origin signal wi(t) per measurement range ri(k) only on the basis of the separation signals for which the component weighting setting unit 1621 has set the component weighting “1”.

[0233] The assessment unit 163 then assesses the test subject's pulse wave based on the pulse wave origin signal wi(t), which was reconstructed by the recovery unit 162 solely on the basis of the separation signals with a component weight of "1". Since the assessment unit 163 can assess the test subject's pulse wave based on the separation signals typical for pulse wave components, the accuracy of the pulse wave assessment of the test subject can be improved.

[0234] Furthermore, in the specific example above, the component weighting setting unit 1621 forms pairs of adjacent cutoff signals and calculates the peak difference of the peak frequencies of the adjacent cutoff signals, but this is just one example. For instance, the component weighting setting unit 1621 can calculate the differences of all pairs and use the weighted average as the component weight.

[0235] For example, the component weighting setting unit 1621 calculates the difference between a given cutoff signal (reference cutoff signal) and the peak frequency of the other (C-1) cutoff signals within the C cutoff signals, and determines the weighted average of the difference based on the calculated (C-1) differences as the difference value of the reference cutoff signal. The component weighting setting unit 1621 then sets the component weight for the reference cutoff signal to "0" if the difference value of the reference cutoff signal is greater than the threshold, and the component weight for the reference cutoff signal to "1" if the difference value is equal to or less than the threshold. The component weighting setting unit 1621 performs the above processing with all cutoff signals as reference cutoff signals.The component weighting setting unit 1621 can also use values ​​corresponding to the size of the weighted average as component weights.

[0236] The component weighting setting unit 1621 can, for example, also calculate correlation coefficients of the projection coefficients between the multiple separation signals when generating the multiple separation signals based on projection coefficients assigned to the separation signals, and set the component weighting per separation signal based on the correlation coefficients.

[0237] More precisely, the component weighting setting unit 1621 first generates, for each cutoff signal per measurement range ri(k), information expressing projection coefficients in time series that are assigned to the multiple pulse-wave origin signal segments corresponding to that measurement range ri(k) (hereinafter referred to as "projection coefficient information in time series"). The component weighting setting unit 1621 generates projection coefficient information in time series for each cutoff signal for the number (N) of measurement ranges ri(k). That is, if C cutoff signals are given, the component weighting setting unit 1621 generates C x N projection coefficient information in time series.

[0238] Subsequently, the component weighting setting unit 1621 determines the correlation relationship of the generated projection coefficient information in time series for each cutoff signal and calculates the correlation coefficient. Specifically, for a given cutoff signal, the component weighting setting unit 1621 forms pairs of all paths for the generated N projection coefficient information in time series and calculates the correlation coefficient for each pair of projection coefficient information in time series. Furthermore, the correlation coefficient indicates the degree of phase agreement of the projection coefficient information in time series. The component weighting setting unit 1621 uses the average of the correlation coefficients calculated for each pair as the component weights for the cutoff signals.

[0239] Here, the pulse wave estimation device 1, 1a, 1b assumes that phase-coherent signal components are signals typical for pulse wave components, and that if the changes in the projection coefficients in time series between the measurement ranges ri(k) are similar, in other words, if the phases of the projection coefficient information in time series between the measurement ranges ri(k) are aligned, it is assumed that the separating signal to which such a projection coefficient is assigned for each pulse wave origin signal segment in each measurement range ri(k) is a separating signal typical for pulse wave components with many pulse wave components.The component weighting setting unit 1621 assigns a larger weighting coefficient to the separation signals typical for pulse wave components, so that the assessment unit 163 can assess the pulse wave of the test subject on the basis of the separation signals typical for pulse wave components.

[0240] To give a concrete example, it is assumed that the several isolation signals output by the signal isolation unit 161 are C isolation signals, representing the first principal component c1, ... up to the C-th principal component cC, as shown by Fig. 5 and Fig. 6 in embodiment 1. Here, the isolation signal representing the first principal component c1 is the first isolation signal, the isolation signal representing the second principal component c2 is the second isolation signal, the isolation signal representing the third principal component c3 is the third isolation signal, the isolation signal representing the fourth principal component c4 is the fourth isolation signal, the isolation signal representing the fifth principal component c5 is the fifth isolation signal, ..., the isolation signal representing the (C-1)th principal component c(C-1) is the (C-1)th isolation signal, and the signal representing the Cth principal component cC is the Cth isolation signal.

[0241] In this case, the component weighting setting unit 1621 first generates projection coefficient information in time series for each separation signal per measurement range ri(k) (first measurement range, second measurement range, third measurement range and fourth measurement range).

[0242] Fig. Figure 16 is a drawing showing an image of an example of projection coefficient information in time series generated by the component weighting setting unit 1621.

[0243] Fig. Figure 16 shows, as an example, the projection coefficient information in time series generated by the component weighting setting unit 1621 for the first separation signal. The component weighting setting unit 1621 generates projection coefficient information in time series for the first measurement range, the second measurement range, the third measurement range, and the fourth measurement range for the first separation signal.

[0244] Furthermore, in Fig. 16 the projection coefficient information generated for the first separation signal is shown in time series as an example, but the component weighting setting unit 1621 also generates projection coefficient information in time series per measurement range ri(k) for the second separation signal up to the C-th separation signal.

[0245] Subsequently, the component weighting setting unit 1621, for example, first forms pairs of all paths in the first separation signal for the projection coefficient information in time series of the first measurement range (in Fig. 16, designated 1601), the projection coefficient information in time series of the second measurement range (in Fig. 16, designated 1602), the projection coefficient information in time series of the third measurement range (in Fig. 16 (designated 1603) and the projection coefficient information in time series of the fourth measurement range (in Fig. 16 (designated with 1604).

[0246] Here, the component weighting setting unit 1621 forms a pair of the projection coefficient information in time series of the first measurement range and the projection coefficient information in time series of the second measurement range (first pair), a pair of the projection coefficient information in time series of the first measurement range and the projection coefficient information in time series of the third measurement range (second pair), a pair of the projection coefficient information in time series of the first measurement range and the projection coefficient information in time series of the fourth measurement range (third pair), a pair of the projection coefficient information in time series of the second measurement range and the projection coefficient information in time series of the third measurement range (fourth pair),a pair of projection coefficient information in time series of the second measurement range and the projection coefficient information in time series of the fourth measurement range (fifth pair) and a pair of projection coefficient information in time series of the third measurement range and the projection coefficient information in time series of the fourth measurement range (sixth pair).

[0247] The component weighting setting unit 1621 then calculates the correlation coefficient of the first pair, the correlation coefficient of the second pair, the correlation coefficient of the third pair, the correlation coefficient of the fourth pair, the correlation coefficient of the fifth pair, and the correlation coefficient of the sixth pair. The component weighting setting unit 1621 takes the average of the calculated correlation coefficients of the first pair, the second pair, the third pair, the fourth pair, the fifth pair, and the sixth pair as the component weight of the first separation signal, which represents the first principal component c1.

[0248] The component weighting setting unit 1621 similarly forms all pairs of all segments of projection coefficient information in time series of the first measurement range, the projection coefficient information in time series of the second measurement range, the projection coefficient information in time series of the third measurement range, and the projection coefficient information in time series of the fourth measurement range for the second cutoff signal up to the C-th cutoff signal, and determines the correlation coefficients for each formed pair in order to calculate the correlation coefficients. The component weighting setting unit 1621 then uses the average of the calculated correlation coefficients for each pair as the component weights.

[0249] The recovery unit 162 restores the pulse wave origin signal wi(t) for each measurement range ri(k) by weighting the multiple separation signals output by the signal separation unit 161 according to the component weights set by the component weighting setting unit 1621.

[0250] For example, the recovery unit 162 determines (component weighting of the first separation signal x [p1-1, p1-2, ..., p1-M] + component weighting of the second separation signal x [p1-1, p1-2, ..., p1-M] + component weighting of the third separation signal x [p1-1, p1-2, ..., p1-M] + ... + component weighting of the C-th separation signal x [p1-2, ..., p1-M]) as the recovered pulse wave origin signal (t) of the first measurement range.

[0251] The assessment unit 163 then estimates the test subject's pulse wave based on the pulse wave origin signal wi(t) recovered by the recovery unit 162 through weighting according to the component weights. Since the assessment unit 163 can estimate the test subject's pulse wave based on the separation signals typical for pulse wave components, the accuracy of the pulse wave assessment can be improved.

[0252] Furthermore, in the example above, the component weighting setting unit 1621 forms pairs of all the projection coefficient information in time series for each separation signal, but this is just one example. The component weighting setting unit 1621 can, for instance, also form a pair of any two projection coefficient information from the multiple projection coefficient information in time series. In the specific example above, this would mean that the component weighting setting unit 1621 could, for example, also form two pairs: one pair of the projection coefficient information in time series from the first measurement range and the projection coefficient information in time series from the second measurement range (seventh pair), and one pair of the projection coefficient information in time series from the third measurement range and the projection coefficient information in time series from the fourth measurement range (eighth pair).In this case, the component weighting setting unit 1621 calculates the correlation coefficient of the seventh pair and the correlation coefficient of the eighth pair, and the calculated average of the correlation coefficient of the seventh pair and the correlation coefficient of the eighth pair is used as the component weight of the first separation signal, which represents the first principal component c1.

[0253] Furthermore, in the example above, the component weighting setting 1621 uses the average of the calculated correlation coefficients as the component weight of the separating signal, but this is just one example. Component weighting setting 1621 could, for instance, also use the variance value of the calculated correlation coefficients as the component weight of the separating signal.

[0254] The operation of the pulse wave assessment unit 16a in the above embodiments 1 to 3 is described by means of Fig. 17 for the case that the pulse wave estimation device 1, 1a, 1b has the function of setting component weights for several separation signals representing several principal components that have been estimated using known general signal separation techniques and of restoring the pulse wave origin signal wi(t) for each measurement range ri(k) on the basis of the several separation signals and the set component weight for each separation signal.

[0255] In the case that, in embodiments 1 to 3, the pulse wave estimation device 1, 1a, 1b has the function of setting component weights for several separation signals representing several principal components that have been estimated using known general signal separation techniques, and of restoring the pulse wave origin signal wi(t) for each measurement range ri(k) based on the several separation signals and the set component weight for each separation signal, the details of the pulse wave estimation processing of step ST6 in Fig. 7, Step ST29 in Fig. 11 and step ST37 in Fig. 15, as described in embodiments 1-3, which is shown in the flowchart of Fig. 17 processing shown.

[0256] Furthermore, the specific processes of steps ST11 and ST13 are similar to those of Fig. 17 each of those of steps ST11 and ST13 of Fig.8, which have already been explained, therefore a redundant explanation is omitted.

[0257] The component weighting setting unit 1621 of the pulse wave estimation unit 16a sets the component weighting for each separation signal output by the signal separation unit 161 in step ST11 (step ST12-1).

[0258] The recovery unit 162 restores the pulse wave origin signal wi(t) per measurement range ri(k) on the basis of the multiple separation signals output by the signal separation unit 161 in step ST11 and the component weighting per separation signal set by the component weighting setting unit 1621 in step ST12-1 (step ST 12-2).

[0259] Furthermore, in step ST13, the assessment unit 163 estimates the pulse wave of the test subject on the basis of the pulse wave origin signal wi(t) recovered by the recovery unit 162 on the basis of the multiple separation signals and the component weighting per separation signal.

[0260] In the pulse wave assessment device 1 according to embodiment 1, the pulse wave assessment device 1a according to embodiment 2, or the pulse wave assessment device 1b according to embodiment 3, the recovery unit 162a is provided with a component weighting setting unit 1621 for setting a weighting coefficient (component weighting) for each separation signal, and by restoring the pulse wave origin signal wi(t) for each measurement range ri(k) based on the multiple separation signals and the weighting coefficient (component weighting) for each separation signal, the pulse wave assessment device 1, 1a, 1b can assess the pulse wave of a test subject based on the separation signals typical for pulse wave components, thereby improving the accuracy of the assessment of the test subject's pulse wave.

[0261] Furthermore, all embodiments can be freely combined, or components of the individual embodiments can be arbitrarily transformed, or components of the individual embodiments can also be omitted.

[0262] The following section summarizes various aspects of the present revelation as additions. (Addendum 1)

[0263] Pulse wave assessment device, comprising a picture acquisition unit that acquires a picture depicting a person, a skin area detection unit that detects a skin area of ​​the person based on the image, a measurement range setting unit that sets several measurement ranges in an area corresponding to the skin area in the image, in order to extract a pulse wave origin signal that represents luminance changes in time series in a first period, a pulse wave origin signal extraction unit that extracts a pulse wave origin signal for each measurement range based on the luminance changes in the first period in the measurement range in question, a segment generation unit which, for each measuring range, generates several pulse wave origin signal segments based on the pulse wave origin signal extracted from the measuring range in question, according to segment generation conditions, wherein these are signals in which a pulse wave origin signal for a second period has been partially extracted from the pulse wave origin signal in the first period, and a pulse wave estimation unit that estimates the person's pulse wave based on the multiple pulse wave origin signal segments generated for each measurement range. (Addendum 2)

[0264] Pulse wave estimation device according to addition 1, characterized in that the segment generation conditions include that the second period is shorter than the first period, With regard to the multiple pulse wave origin signal segments in a specific measurement range, a specific pulse wave origin signal segment is a signal that does not completely coincide with other pulse wave origin signal segments on the time axis. With regard to the multiple pulse wave origin signal segments in a specific measurement range, a specific pulse wave origin signal segment is a signal that partially overlaps with other pulse wave origin signal segments on the time axis, and in each measurement range the duration of the partial overlap of a specific pulse wave origin signal segment with other pulse wave origin signal segments on the time axis is the same length. and in each measurement range, the lengths of the multiple pulse wave origin signal segments are identical to each other on the time axis. (Addendum 3)

[0265] Pulse wave assessment device according to addition 1 or addition 2, characterized in that a parameter setting unit is provided which, based on the pulse wave of a person estimated by the pulse wave estimation unit, calculates the second period and sets segment parameters that contain at least information representing the length of the second period, where the segment generation unit sets the second period according to the segment parameters. (Addendum 4)

[0266] Pulse wave assessment device according to addition 3, characterized in that the parameter setting unit calculates the second period such that the second period is the time of one cycle of the pulse wave of the person assessed by the pulse wave assessment unit. (Addendum 5)

[0267] Pulse wave assessment device according to one of the additions 1 to 4, characterized in that A weighting coefficient calculation unit is provided to calculate weighting coefficients for each of the multiple pulse wave origin signal segments, and the pulse wave assessment unit assesses the person's pulse wave based on the pulse wave origin signal segments and the weighting coefficients. (Addendum 6)

[0268] Pulse wave assessment device according to supplement 5, characterized in that The weighting coefficient calculation unit calculates weighting coefficients for each of the multiple pulse wave origin signal segments based on the magnitude of a signal fluctuation in the pulse wave origin signal from which the pulse wave origin signal segments were extracted. (Addendum 7)

[0269] Pulse wave assessment device according to one of the additions 1 to 6, characterized in that the pulse wave assessment unit a signal separation unit that generates separation signals based on the multiple pulse wave origin signal segments, representing multiple signal components, a recovery unit that restores a pulse wave origin signal for each measurement range based on the multiple cut-off signals, and an assessment unit that assesses the person's pulse wave based on the restored pulse wave origin signal for each measurement range, includes. (Addendum 8)

[0270] Pulse wave assessment device according to supplement 7, characterized in that the recovery unit a component weighting setting unit that sets weighting coefficients for each separation signal, and, based on the multiple separation signals and the weighting coefficients for each separation signal, restores the pulse wave origin signal for each measurement range. (Addendum 9)

[0271] Pulse wave assessment device according to supplement 8, characterized in that the component weighting setting unit Based on the similarity of the frequency characteristics between the multiple separation signals, the weighting coefficients for each separation signal are set. (Addendum 10)

[0272] Pulse wave assessment device according to supplement 8, characterized in that the component weighting setting unit When generating the multiple separation signals based on projection coefficients assigned to the separation signals, correlation coefficients of the projection coefficients between the multiple separation signals are calculated and the weighting coefficients for each separation signal are set based on the correlation coefficients. (Addendum 11)

[0273] Pulse wave assessment method, comprehensive a step in which an image acquisition unit acquires an image depicting a person, a step in which a skin area detection unit detects a skin area of ​​the person based on the image, a step in which a measurement range setting unit sets several measurement ranges in an area corresponding to the skin area in the image, in order to extract a pulse wave origin signal that represents luminance changes in time series in a first period, a step in which a pulse wave origin signal extraction unit extracts a pulse wave origin signal for each measurement range based on the luminance changes in the first period in the measurement range in question, a step in which a segment generation unit generates several pulse wave origin signal segments for each measurement range based on the pulse wave origin signal extracted from the measurement range in question, according to segment generation conditions, wherein these are signals in which a pulse wave origin signal for a second period has been partially extracted from the pulse wave origin signal in the first period, and a step in which a pulse wave estimation unit estimates the person's pulse wave based on the multiple pulse wave origin signal segments generated for each measurement range. [Possibilities for industrial application]

[0274] The pulse wave estimation device according to the present disclosure can, even if a phase difference occurs in the luminance signals extracted from several measurement ranges, estimate the pulse wave of a person on the basis of these luminance signals. [Explanation of reference symbols]

[0275] 1, 1a, 1b Pulse wave estimation device, 11 Imaging unit, 12 Skin area detection unit, 13 Measurement range setting unit, 14 Pulse wave origin signal extraction unit, 15 Segment generation unit, 16, 16a Pulse wave estimation unit, 161 Signal separation unit, 162, 162a Recovery unit, 1621 Component weighting setting unit, 163 Estimation unit, 17 Output unit, 18 Parameter setting unit, 19 Weighting coefficient calculation unit, 3 Imaging device, 101 Processing circuit, 102 Input interface device, 103 Output interface device, 104 Processor, 105 Memory.

Claims

Pulse wave estimation device comprising an image acquisition unit (11) for acquiring an image depicting a person, a skin area detection unit (12) for detecting a skin area of ​​the person based on the image, a measurement range setting unit (13) for setting, in an area corresponding to the skin area on the image, several measurement ranges in order to extract a pulse wave origin signal representing luminance changes in time series in a first period, a pulse wave origin signal extraction unit (14) for extracting the pulse wave origin signal for each of the measurement ranges based on the luminance changes in the first period in the respective measurement ranges, and a segment generation unit (15) for generatingfor each of the measurement ranges, based on the pulse wave origin signal extracted from the corresponding measurement range according to segment generation conditions, several pulse wave origin signal segments, wherein signals are those in which a pulse wave origin signal for a second period has been partially extracted from the pulse wave origin signal in the first period, a pulse wave estimation unit (16, 16a) for estimating the pulse wave of the person based on the several pulse wave origin signal segments generated for each of the measurement ranges, and a parameter setting unit (18) for calculating, based on the pulse wave of the person estimated by the pulse wave estimation unit (16, 16a), the second period and for setting segment parameters that contain at least information representing a length of the second period,wherein the segment generation unit (15) sets the second period according to the segment parameters. Pulse wave assessment device according to claim 1, the parameter setting unit (18) calculates the second period such that the second period is a time of one cycle of the pulse wave of the person assessed by the pulse wave assessment unit (16, 16a). Pulse wave estimation device comprising an image acquisition unit (11) for acquiring an image depicting a person, a skin area detection unit (12) for detecting a skin area of ​​the person based on the image, a measurement range setting unit (13) for setting, in an area corresponding to the skin area on the image, several measurement ranges in order to extract a pulse wave origin signal representing luminance changes in time series in a first period, a pulse wave origin signal extraction unit (14) for extracting the pulse wave origin signal for each of the measurement ranges based on the luminance changes in the first period in the respective measurement ranges, and a segment generation unit (15) for generatingFor each of the measurement ranges, based on the pulse wave origin signal extracted from the corresponding measurement range according to segment generation conditions, several pulse wave origin signal segments are generated, wherein these are signals in which a pulse wave origin signal for a second period has been partially extracted from the pulse wave origin signal in the first period; a pulse wave estimation unit (16, 16a) for estimating the pulse wave of the person based on the several pulse wave origin signal segments generated for each of the measurement ranges; and a weighting coefficient calculation unit (19) for calculating weighting coefficients of the respective several pulse wave origin signal segments; and wherein the pulse wave estimation unit (16, 16a) estimates the pulse wave of the person based on the pulse wave origin signal segments and the weighting coefficients. Pulse wave estimation device according to claim 3, wherein the weighting coefficient calculation unit (19) calculates the weighting coefficients for the respective pulse wave origin signal segments based on the magnitude of a signal fluctuation in the pulse wave origin signal from which the pulse wave origin signal segments were extracted. Pulse wave estimation device comprising an image acquisition unit (11) for acquiring an image depicting a person, a skin area detection unit (12) for detecting a skin area of ​​the person based on the image, a measurement range setting unit (13) for setting, in an area corresponding to the skin area on the image, several measurement ranges in order to extract a pulse wave origin signal representing luminance changes in time series in a first period, a pulse wave origin signal extraction unit (14) for extracting the pulse wave origin signal for each of the measurement ranges based on the luminance changes in the first period in the respective measurement ranges, and a segment generation unit (15) for generatingfor each of the measurement ranges, based on the pulse wave origin signal extracted from the corresponding measurement range according to segment generation conditions, several pulse wave origin signal segments are generated, wherein signals are those in which a pulse wave origin signal for a second period has been partially extracted from the pulse wave origin signal in the first period; a pulse wave estimation unit (16, 16a) for estimating the pulse wave of the person based on the several pulse wave origin signal segments generated for each of the measurement ranges; and the pulse wave estimation unit (16, 16a) includes a signal separation unit (161) for generating, based on the several pulse wave origin signal segments, separation signals representing several signal components; and a restoration unit (162, 162a) for restoring the pulse wave origin signal for each of the measurement ranges based on the several separation signals.and an assessment unit (163) for assessing the person's pulse wave based on the restored pulse wave origin signal for each of the measurement ranges. Pulse wave estimation device according to claim 5, wherein the recovery unit (162a) includes a component weighting adjustment unit (1621) for setting weighting coefficients for the relevant cut-off signals, and restores the pulse wave origin signal for each of the measurement ranges based on the multiple cut-off signals and the weighting coefficients for the relevant cut-off signals. Pulse wave assessment device according to claim 6, wherein the component weighting adjustment unit (1621) adjusts the weighting coefficients for the relevant separation signals based on a similarity of the frequency characteristics between the multiple separation signals. Pulse wave assessment device according to claim 6, wherein the component weighting adjustment unit (1621) calculates correlation coefficients of projection coefficients between the multiple separation signals based on the projection coefficients assigned to the separation signals when generating the multiple separation signals and adjusts the weighting coefficients for the separation signals in question based on the correlation coefficients. Pulse wave estimation procedure comprising a step of acquisition, by an image acquisition unit (11), of an image depicting a person; a step of detection, by a skin area detection unit (12), of a skin area of ​​the person based on the image; a step of setting, by a measurement area setting unit (13), in an area corresponding to the skin area on the image, of several measurement areas to extract a pulse wave source signal representing luminance changes in time series in a first period; a step of extraction, by a pulse wave source signal extraction unit (14) for each of the measurement areas based on the luminance changes in the first period in the respective measurement areas of a pulse wave source signal; a step of generation, by a segment generation unit (15),For each of the measurement ranges, based on the pulse wave origin signal extracted from the corresponding measurement range, several pulse wave origin signal segments are extracted according to segment generation conditions, wherein the segment signals are where a pulse wave origin signal for a second period has been partially extracted from the pulse wave origin signal in the first period; a step of estimation, by a pulse wave estimation unit (16, 16a), of a pulse wave of the person based on the several pulse wave origin signal segments generated for each of the measurement ranges; and a step of calculation, by a parameter setting unit (18), based on the pulse wave of the person estimated by the pulse wave estimation unit (16, 16a), of the second period and for setting segment parameters that contain at least information representing a length of the second period.wherein the segment generation unit (15) sets the second period according to the segment parameters. Pulse wave estimation procedure comprising a step of acquisition, by an image acquisition unit (11), of an image depicting a person; a step of detection, by a skin area detection unit (12), of a skin area of ​​the person based on the image; a step of setting, by a measurement area setting unit (13), in an area corresponding to the skin area on the image, of several measurement areas to extract a pulse wave source signal representing luminance changes in time series in a first period; a step of extraction, by a pulse wave source signal extraction unit (14) for each of the measurement areas based on the luminance changes in the first period in the respective measurement areas of a pulse wave source signal; a step of generation, by a segment generation unit (15),For each of the measurement ranges, based on the pulse wave origin signal extracted from the corresponding measurement range, several pulse wave origin signal segments are generated according to segment generation conditions, wherein the segment signals are those in which a pulse wave origin signal for a second period was partially extracted from the pulse wave origin signal in the first period; a pulse wave estimation unit (16, 16a) estimates a person's pulse wave based on the several pulse wave origin signal segments generated for each of the measurement ranges; and a weighting coefficient calculation unit (19) calculates weighting coefficients of the respective several pulse wave origin signal segments; and wherein the pulse wave estimation unit (16, 16a) estimates the person's pulse wave based on the pulse wave origin signal segments and the weighting coefficients.Pulse wave estimation procedure comprising a step of acquisition, by an image acquisition unit (11), of an image depicting a person; a step of detection, by a skin area detection unit (12), of a skin area of ​​the person based on the image; a step of setting, by a measurement area setting unit (13), in an area corresponding to the skin area on the image, of several measurement areas to extract a pulse wave source signal representing luminance changes in time series in a first period; a step of extraction, by a pulse wave source signal extraction unit (14) for each of the measurement areas based on the luminance changes in the first period in the respective measurement areas of a pulse wave source signal; a step of generation, by a segment generation unit (15),for each of the measurement ranges, based on the pulse wave origin signal extracted from the corresponding measurement range, several pulse wave origin signal segments are generated according to segment generation conditions, wherein the segment signals are where a pulse wave origin signal for a second period has been partially extracted from the pulse wave origin signal in the first period; a pulse wave estimation step is performed by a pulse wave estimation unit (16, 16a) to estimate a person's pulse wave based on the several pulse wave origin signal segments generated for each of the measurement ranges, wherein the estimation step by the pulse wave estimation unit (16, 16a) includes: a generation step by a signal separation unit (161) based on the several pulse wave origin signal segments of separation signals representing multiple signal components; a recovery step by a recovery unit (162,162a) of the pulse wave origin signal for each of the measurement ranges based on the multiple separation signals, and a step of estimation, by an estimation unit (163), of the person's pulse wave based on the recovered pulse wave origin signal for each of the measurement ranges.

Images