Measurement device and measurement method

Abstract

Provided is a highly accurate biological information measurement device capable of calculating a period in which a biological phenomenon of a living body is reflected. A measurement device is provided with: a time sequence signal acquisition unit that acquires a biological signal, which is time sequence data of a value of the biological signal calculated on the basis of an image acquired by capturing an image of a living body, and a determination index relating to the image; a biological information calculation unit that divides the biological signal into pulse signals at predetermined times on the basis of a period of a biological phenomenon; and a pulse determination unit that uses a pulse signal comprising a value of a biological signal calculated from an image in which a determination index satisfying a determination condition can be calculated, and the biological information calculation unit calculates biological information using the pulse signal used by the pulse determination unit.

Description

Technical Field

[0001] The present disclosure relates to a measuring device and a measuring method. Background Art

[0002] Japanese Patent Application Laid-Open No. 2018-019882 discloses a life sign detection device comprising a human image detection unit for detecting a human image, a biometric signal detection unit for detecting a human biometric signal, and a control unit that outputs the detection results of the biometric signal unit. In the life sign detection device disclosed in Japanese Patent Application Laid-Open No. 2018-019882, if the biometric value represented by the biometric signal is within a predetermined normal range and if the detection area of ​​the human image detection unit is determined to be inactive and a human is present, the biometric signal detection unit is determined to be healthy and biometric information based on the biometric signal is output. Summary of the Invention

[0003] The cycle of life activities sometimes reflects the health status of an organism. For example, biological phenomena such as heartbeat and breathing have a cycle of repeated increase and decrease. In the biological signals obtained by measuring the organism, the cycle of such biological phenomena is sometimes reflected, and information on the health status of the organism can be obtained based on the analysis of the periodicity of the signal. For example, there are signals obtained by measuring a pulse sensor or a pulse oximeter. However, in the case of interference that adversely affects the measurement, the detected biological signal contains noise, and such a biological signal may not be able to properly display the periodicity based on the biological phenomenon. In addition, the "interference" here refers to unexpected influences from outside the system or unwanted interference received by the device from the outside, and does not refer to signals in a specific control system.

[0004] In the technology disclosed in Japanese Patent Application Laid-Open No. 2018-019882, a pulse sensor and an image detection device are used. When the image detection of the driver is performed, pulse information is output, and when the image detection of the driver is not performed, pulse information is not output. However, interference such as the movement of the driver's body sometimes occurs discontinuously in seconds. Since the pulse reflects the periodic temporal fluctuations of the heart, the pulse pulsation also periodically fluctuates in time according to this movement. Generally, this periodicity is detected and the pulse is calculated. Therefore, it is preferable to be able to detect based on the period of the pulsation. However, if the image detection fails in the middle of the cycle, the calculation of the pulsation is interrupted in the middle of the cycle, and the pulse is calculated based on the incomplete pulsation with the periodicity destroyed. As a result, in the technology disclosed in Japanese Patent Application Laid-Open No. 2018-019882, it may be impossible to calculate the biological information with high accuracy when interference that has a negative impact on the measurement occurs.

[0005] Therefore, one aspect of the present disclosure aims to provide a measuring device and a measuring method capable of calculating highly accurate biological information reflecting the cycle of a biological phenomenon of a living body.

[0006] A measuring device according to one aspect of the present invention comprises: a photographing unit for photographing a biological body and acquiring an image as time-series data; a time-series signal acquiring unit for acquiring a time-series biological signal representing a temporal change of the biological signal, and acquiring a judgment index temporally associated with the biological signal, wherein the biological signal is a value related to the biological body calculated based on the image; a biological information calculating unit for dividing the time-series biological signal into a prescribed time based on a cycle of a biological phenomenon of the biological body to extract a pulse signal; and a pulse judging unit for judging whether the judgment index satisfies a prescribed judgment condition, and adopting the pulse signal temporally associated with the judgment index that satisfies the judgment condition, wherein the biological information calculating unit uses the pulse signal adopted by the pulse judging unit to calculate biological information related to the biological body.

[0007] A measurement method according to one aspect of the present invention includes: a process of photographing a biological body and acquiring an image as time-series data; a process of acquiring a time-series biological signal representing the time change of the biological signal, and also acquiring a judgment index temporally associated with the biological signal, wherein the biological signal is a value related to the biological body calculated based on the image; a process of extracting a pulse signal by dividing the time-series biological signal according to a specified time based on the cycle of a biological phenomenon of the biological body; a process of judging whether the judgment index satisfies a specified judgment condition; and a process of calculating biological information related to the biological body using the pulse signal temporally associated with the judgment index that satisfies the judgment condition. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 This is a diagram showing an example of how the measuring device is used. Figure 2 This is a block diagram showing an example of the configuration of the measuring device according to the first embodiment. Figure 3 FIG. 1 is a diagram showing an example of a region of interest. Figure 4A This is a graph showing an example of a transition of a determination index indicating the number of pixels in a region of interest and a threshold value serving as a determination threshold value. Figure 4B It shows Figure 4A A graph showing an example of a pulse signal at a time point indicated by the transition of the illustrated judgment index. Figure 5 This is a flowchart showing an example of the operation of the measuring device according to the first embodiment. Figure 6 This is a block diagram showing an example of the configuration of a measuring device according to the second embodiment. Figure 7A This is a graph showing a transition of a judgment index indicating the amount of change in the measuring device and an example of a threshold value serving as a judgment threshold value. Figure 7B It shows Figure 7A A graph showing an example of a pulse signal at a time point indicating the transition of the amount of change of the exemplary measuring device. Figure 8 This is a flowchart showing an example of the operation of the measuring device according to the second embodiment. DETAILED DESCRIPTION

[0009] (First embodiment) Reference Figures 1 to 5 In the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant descriptions are omitted.

[0010] Figure 1 FIG. 1 is a diagram showing an example of how the measuring device 100 is used. Figure 1 As shown, the measuring device 100 includes a photographing unit 101 .

[0011] The measuring device 100 measures the time-series change of the state of the surface or the inside of the skin of the living body 102 based on the image obtained as time-series data by the imaging unit 101, and acquires the living body information 212 (see Figure 2 ). In addition, the image here is captured in a manner that allows biological signals to be acquired as time series data, such as a dynamic image or a still image acquired as time series data, etc., which is an image obtained by capturing the biological body 102 in at least multiple frames.

[0012] For example, the measuring device 100 is a PC (Personal Computer), a smartphone, a tablet terminal, a terminal dedicated to measuring biological information, or a monitoring robot equipped with an imaging unit 101. For example, the biological information 212 is blood pressure, pulse, respiratory rate, blood oxygen saturation, etc. Figure 1 The imaging unit 101 is shown capturing an image of the living body 102 while the living body 102 is holding the measuring device 100. However, the imaging unit 101 may also capture an image of the living body 102 while the living body 102 is not holding the measuring device 100.

[0013] The imaging unit 101 images a living subject 102 and acquires images as time-series data. The imaging unit 101 is located in a position where it can image exposed areas of the living subject 102. Exposed areas of the living subject 102 include the forehead, cheeks, fingertips, wrists, and palms. For example, the imaging unit 101 can be installed in a PC, smartphone, tablet computer, monitor, mirror, or sink.

[0014] The imaging unit 101 is a camera comprising a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and a lens. The imaging unit 101 may also be comprised of an image sensor for a camera that includes RGB filters. For example, the imaging unit 101 may include an RGB Bayer color filter to detect subtle changes in the skin color of the living subject 102. Alternatively, the imaging unit 101 may include RGBCy, RGBIR, or other color filters. RGBCy, RGBIR, or other color filters are suitable for observing increases or decreases in blood volume as indicated by reflected light from light that has passed through the skin.

[0015] Figure 2 This is a block diagram showing an example of the configuration of the measurement device 100 according to this embodiment.

[0016] The measuring device 100 includes an imaging unit 101 , an input unit 201 , an output unit 202 , a storage unit 203 , a control unit 204 , etc. The imaging unit 101 , the input unit 201 , the output unit 202 , and the storage unit 203 are electrically connected to the control unit 204 .

[0017] The imaging unit 101 captures the living body 102 and acquires an image 211, and transmits the acquired image 211 to the control unit 204. For example, the imaging unit 101 captures the living body 102 at 30 to 60 fps (frames per second) and acquires the image 211. The image 211 includes an image of the body surface of the living body 102.

[0018] The input unit 201 receives input of information required by the measurement device 100. For example, the input unit 201 is a keyboard, a mouse, a touch panel, or the like.

[0019] The output unit 202 outputs the image 211, the biological information 212, a message to the living body 102, the date and time when the image 211 was acquired, etc. For example, the output unit 202 is configured to include a display, a speaker, and the like.

[0020] The control unit 204 executes various processes according to the programs and data stored in the storage unit 203. The control unit 204 is composed of a processor such as a CPU (Central Processing Unit) or a GPU (Graphic Processing Unit).

[0021] The control unit 204 includes a time-series signal acquisition unit 205 , a biological information calculation unit 206 , a pulse determination unit 207 , and a signal correction unit 208 .

[0022] The time-series signal acquisition unit 205 calculates a biometric signal based on the image 211. Since the image 211 is acquired as time-series data representing temporal changes in each frame of an image of the living subject 102, the time-series signal acquisition unit 205 can acquire the time-series data representing the biometric signal value calculated from the image 211, namely, a time-series biometric signal 217. For example, the biometric signal value is a representative value or average value of the pixel values ​​of multiple pixels within a region of interest (ROI), and the time-series biometric signal 217 represents its temporal changes. The time-series biometric signal 217 associates the time-series biometric signal value, i.e., the biometric signal value calculated from the image 211, with the time at which the biometric signal value was acquired. The "time at which the biometric signal value was acquired" is defined as the time at which the original frame in the image 211 for which the biometric signal value was calculated was acquired by the imaging unit 101. In practice, the biometric signal value is obtained by the time-series signal acquisition unit 205 setting a region of interest (ROI) in a predetermined frame of the image 211 and calculating the biometric signal value from the pixel values ​​of the pixels within the ROI. However, since the moment the subject 102 is imaged is the time at which the biometric signal value reflecting the state of the subject 102 at that moment is obtained, the moment the imaging unit 101 acquires the frame from which the biometric signal value is calculated is defined as the time at which the biometric signal value is acquired from that frame. While biometric signal values ​​can be calculated without performing the setting of a ROI and calculation processing of the image within the ROI for each frame, performing the calculation at finer time intervals allows for more accurate acquisition of biometric signal data as time-series data, and is therefore preferably performed for each frame. Therefore, in this embodiment, the biometric signal value is acquired for each frame as time-series data.

[0023] Furthermore, the timing signal acquisition unit 205 acquires a judgment index for the image 211 at a predetermined timing, obtaining a judgment index signal 216 as timing data of the judgment index. The judgment index in this embodiment represents the number of pixels in the region of interest (ROI) included in the image 211. In this disclosure, the number of pixels is information indicating the number of pixels (pixels) included in the image 211, for example, the number of sub-pixels for each of R (Red), G (Green), and B (Blue) in the RGB space, or the number of pixels calculated from the sub-pixel values.

[0024] The biometric information calculation unit 206 obtains a pulse signal by time-dividing the time-series biometric signal 217 based on the cycle of the biometric phenomenon the user wishes to detect. Therefore, the pulse signal is temporally correlated with the moment at which the biometric signal value contained in the pulse signal was acquired. Examples of biometric phenomena that can be detected include pulse, heartbeat, respiration, blood pressure fluctuations, and blood enzymes, and the type of biometric information 212 to be detected is determined by the type of biometric information 212 desired to be detected.

[0025] The biological information calculation unit 206 calculates biological information 212 based on the correction signal 218 generated by the signal correction unit 208. The biological information 212 indicates information about the living body 102 that the user wants to detect. For example, the biological information 212 indicates blood pressure, heart rate, blood oxygen saturation, and the like.

[0026] The pulse determination unit 207 determines whether the pulse signal satisfies the determination condition based on the value of the determination indicator obtained from the determination indicator signal 216. The pulse determination unit 207 then uses the pulse signal that is temporally correlated with the determination indicator that satisfies the determination condition. The determination condition in this embodiment is that the number of pixels in the region of interest (ROI) exceeds the determination threshold 215 (first determination threshold) in this embodiment.

[0027] The signal correction unit 208 generates a correction signal 218 composed of the pulse signal used by the pulse determination unit 207 .

[0028] The storage unit 203 is a recording medium capable of storing various data, programs, and the like, and is comprised of, for example, a hard disk, an SSD (Solid State Drive), or a semiconductor memory. The storage unit 203 includes a measurement information storage unit 213, a biological information storage unit 214, and a judgment threshold value 215. The storage unit 203 may store multiple judgment threshold values ​​215.

[0029] The measurement information storage unit 213 stores pre-stored programs necessary for measuring the biological information 212, information registered by the user, and the like. For example, the measurement information storage unit 213 stores calculation formulas related to the biological information 212 and calculation formulas related to judgment indicators. For example, the user is the administrator of the biological body 102 or the measurement device 100.

[0030] The biometric information storage unit 214 stores information that allows identification of which pulse signal was used from among all pulse signals as adopted pulse information. For example, the adopted pulse information may include the adopted pulse signal itself, a unique number that identifies the adopted pulse signal, the time at which a judgment indicator temporally associated with the adopted pulse signal was obtained, or the value of the biometric signal contained in the adopted pulse signal. The adopted pulse information may also include other information related to the judgment indicator, such as the time at which the judgment indicator satisfies the judgment condition, or a unique number that identifies the time at which the judgment indicator satisfies the judgment condition.

[0031] At least one of the measurement information storage unit 213 and the biological information storage unit 214 stores information related to the living body 102 . For example, the information related to the living body 102 represents a log of the biological information 212 .

[0032] Figure 3 FIG. 1 is a diagram showing an example of a region of interest ROI.

[0033] The measurement device 100 of this embodiment calculates the biological information 212 based on the pixel values ​​of the pixels included in the region of interest ROI set in the image 211 .

[0034] Furthermore, to calculate the biometric information 212, it is preferable to select an image of a region suitable for calculation, such as a region where the body surface is exposed and changes in pixel values ​​of the body surface image due to changes in blood vessel volume, etc., can be easily observed as the region of interest (ROI). For example, an image of the cheek of the living subject 102 is suitable for calculating the biometric information 212.

[0035] For example, the timing signal acquisition unit 205 uses a face detection algorithm to set a region of interest ROI in the image 211. Specifically, the timing signal acquisition unit 205 uses a face detection algorithm based on pattern recognition, machine learning, etc. to extract feature quantities related to facial features from the image 211. For example, the feature quantities about facial features represent the positions and shapes of the eyes and nose. Then, the timing signal acquisition unit 205 sets the position of the region of interest ROI based on the extracted feature quantities. The timing signal acquisition unit 205 can obtain the position of the set region of interest ROI as coordinates in the image 211, for example, Figure 3The illustrated region of interest ROI is a rectangle, and the coordinates of its vertices are represented by (x1, y1), (x1, y2), (x2, y1), and (x2, y2) in the image 211 .

[0036] When the subject 102 holding the measurement device 100 experiences hand tremors or body movements, the size of the facial image of the subject 102 contained in the image 211 may change. Specifically, the size of the facial image contained in the image 211 changes depending on the distance between the measurement device 100 and the subject 102. For example, the further the subject 102 moves from the measurement device 100, the smaller the facial image contained in the image 211. As a result, the number of pixels in the region of interest (ROI) contained in the image 211 decreases.

[0037] If the number of pixels in the region of interest (ROI) is insufficient, the quality of the time-series biometric signal 217 may deteriorate. Specifically, a pulse signal acquired at a time when the number of pixels P within the region of interest (ROI) falls below the judgment threshold 215 (threshold Pth) may be unsuitable for accurately calculating the biometric information 212. Therefore, the biometric information calculation unit 206 preferably does not use this pulse signal to calculate the biometric information 212. For example, the judgment condition is that the number of pixels P within the region of interest (ROI) is greater than 25, that is, the number of pixels P is greater than the threshold Pth. In this case, the pulse determination unit 207 determines not to use a pulse signal containing a biometric signal value acquired at a time when the number of pixels P within the region of interest (ROI) falls below 25.

[0038] Reference Figure 4A and Figure 4B The processing of the pulse determination unit 207 will be described in detail. Figure 4A Graph 215 is a graph showing the transition of the determination index indicating the number of pixels P in the region of interest ROI and an example of the threshold value Pth serving as the determination threshold value 215. Figure 4A In FIG, the horizontal axis is time, and the vertical axis is the number of pixels P in the region of interest ROI which is used as a judgment indicator. Figure 4A Each point shown in the illustrated graph represents the number P of pixels in the region of interest ROI included in the image 211 acquired at different times.

[0039] Figure 4B It shows Figure 4A A graph showing an example of a pulse signal at a given moment as the number of pixels P in the region of interest ROI is shown. Figure 4B In FIG, the horizontal axis represents time and the vertical axis represents the value of the biological signal. For example, when the time series biological signal 217 represents a pulse wave signal, Figure 4B The value of the biological signal shown in the example represents the intensity of the pulse wave. Figure 4BIn FIG. 1 , as an example, 11 pulse signals are respectively marked with reference numerals tA to tK. Figure 4A The vertical axis shows the interval for obtaining the judgment index and Figure 4B The calculation interval of the value of the biological signal shown on the vertical axis is not limited to this, and at least one judgment index can be obtained for each pulse signal. For example, the value of the biological signal and the judgment index can be calculated for each frame of the image 211, or a plurality of judgment indexes can be obtained for each pulse signal and their average can be calculated. The finer the time interval for obtaining the judgment index, the more detailed the temporal observation of interference such as the body movement of the organism 102. However, if the time interval is refined beyond the required value, the calculation processing becomes complicated, so it is preferably determined according to the desired calculation accuracy of the biological signal. In addition, in the present embodiment, the value of the biological signal and the judgment index are calculated for each frame of the image 211, and the judgment index and the biological signal temporally associated with the judgment index are obtained at the same time, that is, the interval for obtaining the judgment index is the same as the interval for calculating the biological signal, but for the sake of explanation, Figure 4A The interval for obtaining the judgment index shown is the same as Figure 4B The calculation intervals of the values ​​of the biological signals shown are different.

[0040] In this embodiment, if Figure 4B As illustrated, the biological information calculation unit 206 divides a signal that varies temporally based on a biological phenomenon of the living subject 102, based on that period, into predetermined intervals. Biological phenomena include those that exhibit periodic fluctuations, such as those reflecting the state of the living subject, such as respiration, heartbeat, and blood pressure. For example, if the time-series biological signal 217 reflects information related to the blood vessels of the living subject 102, the intensity of the time-series biological signal 217 varies temporally based on the pulsation cycle of the blood vessels caused by the periodic contraction and dilation of the heart, repeating a cycle of increase and decrease. Because actual biological phenomena are sensitive to health or mental states, their periods are not necessarily constant; for example, the period of arrhythmia may be irregular. However, since biological phenomena generally vary temporally based on a certain periodicity, the biological information calculation unit 206 can divide the time-series biological signal 217 into predetermined intervals. In addition, the biological signal obtained at the moment when the biological signal is chaotic to the extent that it cannot be divided can be considered as having poor signal quality and not used, and the time-series biological signal 217 other than the moment can be used to divide it into predetermined time periods.

[0041] Thus, in this embodiment, the biological information calculation unit 206 divides the time series biological signal 217 into a plurality of pulse signals based on the cycle of the biological phenomenon of the living body 102. Specifically, Figure 4BAs shown in the example pulse signal tA to pulse signal tK, the cycle of increase and decrease of the value of the biological signal is divided into pulse signals at a predetermined time based on the cycle. Figure 4B The pulse signal shown in the example represents a pulse wave signal, and the type of biological information 212 is heart rate or blood pressure. Figure 4B As illustrated, the biological information calculation unit 206 sets the rise and fall of the pulse wave accompanying the contraction and expansion of the heart as one cycle. Figure 4B 3 shows a pulse signal divided into intervals divided into one period, that is, into a dotted rectangle.

[0042] As described above, the biometric information calculation unit 206 divides the temporally varying time-series biometric signal 217 based on the cycle of the biological phenomenon of the living subject 102 to obtain a plurality of pulse signals in order to measure the time-series biometric signal 217 and calculate the biometric information 212. Then, the biometric information calculation unit 206 selects a pulse signal that satisfies the determination condition and calculates the biometric information 212 using the selected pulse signal. This allows the biometric information calculation unit 206 to calculate the biometric information 212 with high precision based on the cycle of the biological phenomenon.

[0043] exist Figure 4A and Figure 4B In the illustrated graph, the number of pixels P, which is the judgment indicator before time t410 and after time t413, exceeds the threshold value Pth. Therefore, the pulse judgment unit 207 uses the pulse signals associated with the time before time t410 and after time t413. Specifically, the pulse judgment unit 207 uses the pulse signal tA and the pulse signal tF. Pulse signal tK.

[0044] On the other hand, the number of pixels P, which is the judgment indicator at the time after time t411 and before time t412, is less than the judgment threshold value 215, which is the threshold value Pth. Therefore, the pulse judgment unit 207 does not use the pulse signal composed of the value of the biological signal associated with the time after time t411 and before time t412. Specifically, the pulse judgment unit 207 does not use the pulse signal tB Pulse signal tE. Note that the pulse signal tE partially includes a time when the number of pixels P is less than or equal to the threshold value Pth. Therefore, the pulse determination unit 207 does not use the pulse signal tE.

[0045] Furthermore, information that can identify which pulse signal is used among all the pulse signals is stored as used pulse information. Based on the used pulse information, the signal correction unit 208 generates a correction signal 218. Thus, the signal correction unit 208 can generate a correction signal 218 composed of the used pulse signal.

[0046] The biometric information calculation unit 206 calculates the biometric information 212 based on the correction signal 218. Specifically, the biometric information calculation unit 206 calculates the biometric information 212 using the pulse signal that is temporally associated with the moment when the judgment index exceeds the threshold value Pth. Figure 4B The pulse signal tA and the pulse signal tF to the pulse signal tK shown as examples are used to calculate the biological information 212 from the image 211 .

[0047] Furthermore, the lower the threshold value Pth, the more likely the quality of the biometric information 212 is to decline. On the other hand, the higher the threshold value Pth, the fewer pixels in the region of interest (ROI) due to hand tremors or body movements of the subject 102 holding the measurement device 100. Consequently, the judgment indicator fails to meet the judgment criteria, and the pulse signal is more likely to be rejected by the pulse judgment unit 207. As a result, the time required from the imaging unit 101 starting the imaging process of the subject 102 to the output of the biometric information 212 may increase. Therefore, it is preferable to set the threshold value Pth based on the acceptable pixel quality for the biometric information 212 and the waiting time allowed by the subject 102.

[0048] Figure 5 This is a flowchart showing an example of the operation of the measurement device 100 according to this embodiment.

[0049] In step S501 , the following processing is started: the imaging unit 101 images the living body 102 and acquires the image 211 .

[0050] In step S502, the time-series signal acquisition unit 205 calculates a biometric signal from the image 211 acquired in step S501. Specifically, in step S502, the time-series signal acquisition unit 205 acquires a time-series biometric signal 217, which represents the temporal variation of a biometric signal that can be calculated from the image 211 acquired by the imaging unit 101. For example, the time-series signal acquisition unit 205 sets a region of interest (ROI) within the image 211 and calculates the time-series biometric signal 217 from the RGB pixel values ​​of the pixels within the ROI using a calculation formula pre-stored in the storage unit 203. Alternatively, if desired, the time-series signal acquisition unit 205 may acquire the time-series biometric signal 217, representing a pulse wave converted into absorbance, based on temporal variations in values ​​calculated by substituting the luminance values ​​of pixels within the ROI of the image 211 into a predetermined mathematical formula. Alternatively, the time-series signal acquisition unit 205 may acquire the time-series biological signal 217 representing the pulse wave using an independent component analysis method, a biological component separation method, or the like.

[0051] In step S503 , the time series signal acquisition unit 205 calculates a determination index indicating the number of pixels P in the region of interest ROI, and acquires a determination index signal 216 , which is time series data indicating temporal variation of the determination index calculated from the image 211 .

[0052] In step S504, the biological information calculation unit 206 divides the time-series biological signal 217 into pulse signals. Specifically, the biological information calculation unit 206 divides the time-series biological signal 217 into a plurality of pulse signals at predetermined times based on one cycle, which is the cycle of the biological phenomenon of the living body 102. The cycle based on the biological phenomenon of the living body 102 used to divide the pulse signals is set according to the type of biological information 212 being measured.

[0053] In step S505 , the pulse determination unit 207 determines whether the determination index satisfies a determination condition. The determination condition is that the number of pixels in the region of interest (ROI) indicated by the determination index exceeds a determination threshold 215 .

[0054] Specifically, the pulse determination unit 207 determines whether the number of pixels P exceeds the threshold value Pth. If the number of pixels P within the region of interest (ROI) exceeds the threshold value Pth, the pulse determination unit 207 determines that the determination indicator satisfies the determination condition. On the other hand, if the number of pixels P within the region of interest (ROI) is less than the threshold value Pth, the pulse determination unit 207 determines that the determination indicator does not satisfy the determination condition. The pulse determination unit 207 determines whether the determination indicator satisfies the determination condition at each moment of the determination indicator signal 216, which is time series data.

[0055] Furthermore, if multiple judgment thresholds 215 are stored in the storage unit 203, the pulse judgment unit 207 may select a judgment threshold 215 during measurement. Furthermore, the input unit 201 may receive an operation to change the judgment threshold 215 to be used based on the multiple judgment thresholds 215 stored in the biometric information storage unit 214, and the pulse judgment unit 207 may change the judgment threshold 215 to be used. For example, if there are multiple users, by storing the judgment thresholds set by each user in the biometric information storage unit 214 and using the optimal judgment threshold for each user, more accurate biometric information 212 can be obtained.

[0056] In step S506, the control unit 204 distinguishes the time when the judgment indicator satisfies the judgment condition from the time when the judgment indicator does not, and stores the information in the measurement information storage unit 213. For example, the information indicating the time is the time or identification information associated with the time. Furthermore, the measurement information storage unit 213 does not necessarily need to store the time when the judgment indicator does not satisfy the judgment condition.

[0057] In step S507, the pulse determination unit 207 adopts the pulse signal associated with the time at which the determination indicator satisfies the determination condition. Specifically, the pulse determination unit 207 adopts a pulse signal consisting of a biological signal value calculated from the pixel values ​​of the region of interest (ROI) of the image acquired at the time at which the determination indicator satisfies the determination condition, and stores adopted pulse information associated with the adopted pulse signal in the measurement information storage unit 213.

[0058] If the imaging unit 101 does not capture the subject 102 within the prescribed time required for appropriate calculation of the biometric information 212, the accuracy of the biometric information 212 calculated by the biometric information calculation unit 206 may not be guaranteed. Similarly, if the total number of pulse signals used is less than the prescribed number of pulses required for appropriate calculation of the biometric information 212, the accuracy of the biometric information 212 calculated by the biometric information calculation unit 206 may not be guaranteed.

[0059] Therefore, in step S508, the pulse determination unit 207 determines whether a pulse signal with a predetermined number of pulses or more is used. Specifically, the pulse determination unit 207 determines whether a pulse signal with a predetermined number of pulses or more is used to image the biological body 102. The predetermined number of pulses is pre-stored in the measurement information storage unit 213. The predetermined number of pulses may also vary depending on the type of biological information 212 used as the measurement target. Alternatively, in step S508, the pulse determination unit 207 may determine whether a number of pulse signals whose total duration satisfies a predetermined time or more is used. In this case, the predetermined time is pre-stored in the measurement information storage unit 213. That is, in step S508, the pulse determination unit 207 uses a plurality of pulse signals that satisfy at least one of the predetermined number of pulses or more and the predetermined time or more.

[0060] If a pulse signal that satisfies at least one of the predetermined number of pulses and the predetermined time period is not adopted in step S508, control unit 204 returns the process to step S502. In the process after step S502, pulse determination unit 207 adds newly adopted pulse signals after the time associated with the already adopted pulse signal to the already adopted pulse signal and stores the adopted pulse information in measurement information storage unit 213. This allows measurement device 100 to shorten the time required to adopt a number of pulse signals that satisfies at least one of the predetermined number of pulses and the predetermined time period.

[0061] On the other hand, if a pulse signal that satisfies at least one of the predetermined number of pulses or the predetermined time is used in step S508, the signal correction unit 208 generates a correction signal 218, which is time series data consisting only of the used pulse signal, in step S509. Thus, the signal correction unit 208 generates a correction signal 218 using a pulse signal consisting of the value of the biological signal associated with the time when the judgment indicator satisfies the judgment condition.

[0062] In step S510, the biometric information calculation unit 206 calculates the biometric information 212 based on the correction signal 218. Specifically, the biometric information calculation unit 206 calculates the biometric information 212 using the pulse signal used by the pulse determination unit 207. Specifically, the biometric information calculation unit 206 calculates the biometric information 212 using the pulse signal used by the pulse determination unit 207 that satisfies at least one of a predetermined number of pulses and a predetermined duration. Furthermore, the output unit 202 outputs the biometric information 212 as needed. For example, the output unit 202 outputs the biometric information 212 so that it is displayed on a display device (not shown) connected to the measurement device 100. The control unit 204 then terminates the processing.

[0063] As described above, the measurement device 100 according to this embodiment determines the suitability of each pulse signal included in the time-series biometric signal 217. Specifically, pulse signals associated with moments when the number of pixels P in the region of interest (ROI) decreases significantly and the judgment index does not exceed the judgment threshold 215, thereby failing to meet the judgment condition, are not used. In contrast, pulse signals of good quality associated with moments when the number of pixels P exceeds the judgment threshold 215 are used.

[0064] Furthermore, by calculating the biological information 212 using the values ​​of the biological signals included in the pulse signal of good quality, the calculation accuracy can be improved compared to calculating the biological information 212 based on the values ​​of all the biological signals included in the time-series biological signal 217 .

[0065] In this case, by performing a determination for each pulse signal, the biological information 212 can be calculated using a biological signal whose values ​​are completely complete based on one cycle of a biological phenomenon and suitable for measuring the biological information 212. As a result, the measurement device 100 of this embodiment can calculate the biological information 212 with high accuracy that reflects the cycle of the biological phenomenon of the living body 102.

[0066] Alternatively, the pulse determination unit 207 may determine whether the determination indicator satisfies the determination condition for a period of time or longer. Furthermore, if the determination indicator satisfies the determination condition for a period of time or longer, the pulse determination unit 207 may use a pulse signal representing the value of the biometric signal calculated from the image 211 acquired at the time the determination indicator satisfied the determination condition. Thus, the measurement device 100 according to this embodiment can calculate highly accurate biometric information 212 using a biometric signal that continuously reflects the cycle of a biometric phenomenon of the living body 102 over time.

[0067] Furthermore, information stored in the storage unit 203, other than information that needs to be saved, such as user logs, does not necessarily need to be stored long-term. For example, information such as identification information of a pulse signal used during calculation of the biometric information 212 may be temporarily stored in a variable at the moment a judgment indicator satisfies a judgment condition and then deleted after the calculation is completed. Alternatively, it may be stored after the calculation is completed. In the case of long-term storage, it can be used for data analysis, etc., to improve the accuracy of the calculation of the biometric information 212.

[0068] (Variation 1) Modification 1 of the first embodiment will be described. In the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant descriptions will be omitted.

[0069] For example, when the biological subject 102 holding the measuring device 100 experiences hand shaking, or when the biological subject 102 experiences body movement, the position of the region of interest ROI in the image 211 sometimes moves significantly. In more detail, there is a case where the coordinates of the region of interest ROI in the image 211 change significantly. In addition, the coordinates here are the positions in the image 211. Since the changes caused by such hand shaking, body movement, etc. become interference, the values ​​of the biological signal and the pulse signal may sometimes change. Therefore, the timing signal acquisition unit 205 involved in this modification example calculates the judgment index based on the displacement of the coordinates of the region of interest ROI in the image 211 acquired at different times. For example, the timing signal acquisition unit 205 involved in this modification example is for Figure 3 The coordinates of the vertices of the region of interest (ROI) illustrated in the example, namely, the coordinates (x1, y1), (x1, y2), (x2, y1), and (x2, y2), are used to calculate a judgment index based on the displacement of the respective coordinates between different frames. In other words, the judgment index involved in this variation represents the displacement of the coordinates of the region of interest set within image 211 acquired at different times. The judgment condition involved in this variation is that the displacement of the coordinates is less than the judgment threshold 215 (second judgment threshold) involved in this variation.

[0070] For example, let the coordinates of the vertex of the upper left corner of the region of interest ROI contained in the i-th frame image 211 be (x1i, y1i). i is a natural number greater than 1. In addition, let the coordinates of the vertex of the upper left corner of the region of interest ROI contained in the n-th frame image 211 be (x1n, y1n). n is a natural number greater than i+1. The displacement of the vertex of the upper left corner of the region of interest ROI between the i-th frame and the n-th frame is represented by (x1n-x1i, y1n-y1i). Furthermore, the judgment threshold 215 involved in this modification shows the threshold xth of the x coordinate in the region of interest ROI and the threshold yth of the y coordinate in the region of interest ROI. When x1n-x1i≤xth and y1n-y1i≤yth, the pulse judgment unit 207 judges that the judgment index satisfies the judgment condition and adopts the pulse signal associated with the moment of obtaining from the i-th frame to the n-th frame. On the other hand, when x1n-x1i>xth and y1n-y1i>yth, the pulse determination unit 207 determines that the determination index does not satisfy the determination condition and does not adopt the pulse signal associated with the time from the i-th frame to the n-th frame.

[0071] As described above, the measurement device 100 of this variation does not use the value of the biometric signal acquired at the time when the coordinate change of the region of interest (ROI) exceeds the judgment threshold 215 to calculate the biometric information 212. Therefore, the measurement device 100 of this variation can calculate the biometric information 212 that accurately reflects the period of the biometric phenomenon of the living subject 102, even if the hand holding the measurement device 100 of the living subject 102 shakes or the body of the living subject 102 moves while the imaging unit 101 is imaging the living subject 102.

[0072] In addition, the above-mentioned judgment condition is an example and is not intended to limit the judgment condition of this modified example. For example, the judgment condition may also be x1n-x1i≦xth, or y1n-y1i≦yth. In addition, the judgment condition may also be expressed only by the displacement of the x coordinate in the region of interest ROI. In addition, when the region of interest ROI is rectangular, the judgment condition may also be expressed by the displacement of the coordinates of the four corners. In addition, the judgment condition may also be set according to the type of biological information 212 used as the measurement purpose and the accuracy required for the biological information 212. In addition, in this modified example, the coordinates of the region of interest ROI in the image 211 are used to represent the displacement when the region of interest ROI moves, but this is not limited to this. As long as it is an indicator that represents the change of the region of interest ROI, it is preferable to use an indicator that captures the displacement as a numerical value like the coordinates.

[0073] (Variation 2) A second modification of the first embodiment will be described. In the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant descriptions will be omitted.

[0074] When the living subject 102 performs movements such as blinking, the facial skin of the living subject 102 moves, sometimes causing abrupt fluctuations in the value of the living signal, similar to spike noise. Due to this interference, the quality of the time-series living signal 217 acquired during the period when the living subject 102 is blinking is lower than the quality of the time-series living signal 217 acquired when the living subject 102 is not blinking or performing other movements. Therefore, the time-series signal acquisition unit 205 of this modified example calculates a judgment index representing the number of movements of the living subject 102 per unit time, calculated based on the image 211. For example, the number of movements represented by the judgment index is the number of blinks per unit time.

[0075] Specifically, the timing signal acquisition unit 205 of this modification example calculates the number of movements of the organism 102 per unit time for each pulse signal within the time period in which the biological signal included in the pulse signal is acquired. For example, the timing signal acquisition unit 205 of this modification example calculates the number of blinks per unit time of the image of the eyelid included in the image 211 as a judgment indicator. The judgment condition of this modification example is that the number of movements of the organism 102 per unit time is below the judgment threshold 215 (third judgment threshold) of this modification example. For example, the judgment condition of this modification example is that the number of blinks of the organism 102 per unit time is below the judgment threshold 215.

[0076] The judgment threshold 215 involved in this variation represents a threshold for the number of movements per unit time. For example, the judgment threshold 215 involved in this variation represents a threshold for the number of blinks per unit time. In this case, if the number of blinks represented by the judgment indicator exceeds the threshold for the number of blinks represented by the judgment threshold 215, the pulse judgment unit 207 involved in this variation determines that the judgment indicator does not meet the judgment condition. Therefore, the pulse signal calculated based on the image 211 acquired at the time when the number of blinks exceeded the threshold for the number of blinks represented by the judgment threshold 215 is not used.

[0077] As described above, the measurement device 100 of this variation, when obtaining a number of times that a living subject's movement, such as blinking, exceeds the judgment threshold 215, does not use the value of the biometric signal calculated from images obtained during the time of the movement in calculating the biometric information 212. Thus, even when the living subject 102 performs movement, such as blinking, a number of times exceeding the judgment threshold 215 per unit time, the measurement device 100 of this variation can calculate the biometric information 212 that accurately reflects the period of the biological phenomenon of the living subject 102 by excluding the value of the biometric signal obtained during the time of the blinking.

[0078] (Second embodiment) Reference Figures 6 to 8 In the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant descriptions are omitted.

[0079] Figure 6 This is a block diagram showing an example of the configuration of the measurement device 600 according to this embodiment. Figure 6 The exemplary measuring device 600 is Figure 2 The exemplary measuring device 100 is different in that it has Figure 6 An example interference measurement unit 601 is shown.

[0080] When the imaging unit 101 captures the image of the living subject 102 while the living subject 102 is holding the measurement device 600 and the living subject 102 is moving, the measurement device 600 moves in conjunction with the movement of the living subject 102. When the living subject 102 is holding the measurement device 600, body movement such as hand tremors, swaying, coughing, or shaking of the living subject 102 can cause interference, potentially mixing noise or undesirable fluctuations into the biometric signal calculated from the image 211.

[0081] Therefore, the measuring device 600 includes a disturbance measurement unit 601. When the measuring device 600 spatially varies due to, for example, physical movement or violent physical movement of the living subject 102, the disturbance measurement unit 601 acquires disturbance information 611 indicating the amount of variation. Disturbance information 611 indicates at least one element selected from the group consisting of the displacement of the measuring device 600 per unit time, the amount of variation in the velocity of the measuring device 600, the amount of variation in the acceleration of the measuring device 600, and the amount of variation in the angular velocity of the measuring device 600. The measuring device 600 includes a device capable of measuring the amount of spatial variation of the measuring device 600 as time-series data, and acquires disturbance information 611 as time-series data of the amount of spatial variation measured by the device. Examples of devices capable of measuring the amount of spatial variation of the measuring device 600 include devices capable of measuring position, acceleration, angular velocity, and the like, such as acceleration sensors and gyroscope sensors.

[0082] The time series signal acquisition unit 205 of this embodiment calculates a judgment index derived from the interference information 611 and acquires a judgment index signal 612 of time series data serving as the judgment index. Specifically, the judgment index of this embodiment represents the spatial variation of the measurement device 600 and is temporally correlated with the pulse signal composed of the value of the biological signal obtained from the image 211. The judgment condition of this embodiment is that the spatial variation of the measurement device 600 is equal to or less than the judgment threshold 215 (fourth judgment threshold) of this embodiment.

[0083] Reference Figure 7A and Figure 7B , the processing of the pulse determination unit 207 involved in this embodiment is described in detail.

[0084] Figure 7A : is a graph showing the transition of the judgment index representing the change amount M of the measuring device 600 and an example of the threshold value Mth as the judgment threshold value 215. Figure 7A In FIG, the horizontal axis is time, and the vertical axis is the change amount M represented by the judgment index. Figure 7A Each point shown in the illustrated graph represents the amount of change M indicated by the interference information 611 acquired at different times.

[0085] Figure 7B It shows Figure 7A A graph showing an example of a pulse signal at a given moment showing the change in the amount M of change in the measuring device. Figure 7B In the example, the horizontal axis is time and the vertical axis is the value of the biological signal. Figure 7B In the above example, the pulse signal mG is also measured, but for the sake of convenience, it is omitted below. Figure 4B The same, detailed description is omitted.

[0086] exist Figure 7A and Figure 7B In the illustrated graph, the change M indicated by the disturbance information 611 at times before t710 and after t713 in the pulse signals mA through mG is greater than the threshold value Mth serving as the determination threshold 215. Therefore, the pulse determination unit 207 does not adopt the pulse signals containing the values ​​of the biological signal calculated from the images 211 acquired before t710 and after t713. Specifically, the pulse determination unit 207 determines that the pulse signals mA and mE through mG are not to be adopted from the pulse signals mA through mG.

[0087] On the other hand, the change amount M represented by the judgment index at the time after time t711 and before time t712 is less than the judgment threshold value 215, that is, the threshold value Mth. Therefore, the pulse signal composed of the value of the biological signal calculated from the image 211 acquired at the time after time t711 and before time t712 is used. Specifically, the pulse judgment unit 207 uses the pulse signal mB Pulse signal mD. In addition, the pulse determination unit 207 determines to use a pulse signal including only the value of the biological signal calculated from the image acquired at the time when the change amount M indicated by the interference information 611 is less than the threshold value Mth. Figure 7B The pulse signal mE includes both the moment t712 when the change amount M represented by the interference information 611 is below the threshold Mth and the moment t713 when it is above the threshold Mth. However, since at least one moment is included when the change amount M is above the threshold Mth, the pulse judgment unit 207 does not adopt the pulse signal mE.

[0088] Therefore, the biological information calculation unit 206 uses Figure 7B The pulse signals mB to mD are used to calculate the biometric information 212 from the image 211. Specifically, the biometric information calculation unit 206 calculates the biometric information 212 using pulse signals composed of biometric signal values ​​calculated from images at times when the determination index is below the threshold value Mth.

[0089] Figure 8 is a flowchart showing an example of the operation of the measuring device 600 according to this embodiment. Figure 5 The processing of the illustrated steps is the same as that of the steps, and detailed description thereof will be omitted.

[0090] Figure 8 Flowchart of the example and Figure 5 The illustrated flowchart is different in that the spatial variation of the measurement device 600 indicated by the interference information 611 acquired by the interference measurement unit 601 is used as a determination indicator.

[0091] In step S801 , the imaging unit 101 starts a process of acquiring the image 211 , and the interference measurement unit 601 starts a process of acquiring the amount of change in the measurement device.

[0092] In step S802 , the time-series signal acquisition unit 205 acquires the time-series biological signal 217 from the image 211 acquired by the imaging unit 101 .

[0093] In step S803 , the time series signal acquisition unit 205 calculates a determination index based on the amount of change indicated by the interference information 611 , and acquires the determination index signal 216 as the time series data.

[0094] In step S804, the biological information calculation unit 206 divides the time series biological signal 217 into pulse signals. Figure 5 The illustrated step S504 is the same, so detailed description is omitted.

[0095] In step S805, the pulse judgment unit 207 judges whether the judgment index calculated in step S803 satisfies the judgment condition. The judgment condition of this embodiment is that the change amount M represented by the interference information 611 is below the threshold value Mth. That is, the pulse judgment unit 207 judges whether the change amount M represented by the interference information 611 is ≤ the threshold value Mth. When the change amount M represented by the interference information 611 is below the threshold value Mth, the pulse judgment unit 207 judges that the judgment index satisfies the judgment condition. On the other hand, when the change amount M represented by the interference information 611 is greater than the threshold value Mth, the pulse judgment unit 207 judges that the judgment index does not satisfy the judgment condition. The pulse judgment unit 207 judges whether the judgment index satisfies the judgment condition at each moment of the time series data, i.e., the judgment index signal 216. Then, the control unit 204 transfers the processing to step S806. The processing after step S806 is the same as Figure 5 The processing of step S506 shown is the same, so detailed description is omitted.

[0096] In addition, the embodiment is not limited to this, and the information of the interference obtained from the image acquired by the shooting unit 101 can also be used to determine whether it is the interference that is originally unexpected. That is, if the interference obtained from the image is generated at approximately the same time as the interference detected by the interference measurement unit 601, it can also be determined that the unexpected interference has occurred, and the pulse signal including that moment may not be used. Specifically, when the time difference between the moment of large body movement such as the large facial movement of the organism 102 that can be detected from the image contained in the image 211 and the moment when the change M represented by the interference information 611 does not meet the judgment condition is smaller than a predetermined range, the pulse signal including that moment is not used. In this case, by comparing the information obtained from the image with the information obtained from the interference measurement unit 601, it can be determined whether it is truly unexpected interference, and the influence of the interference can be removed more accurately.

[0097] As described above, the measurement device 600 according to this embodiment can more accurately eliminate the influence of the body movement of the living subject 102 , and can calculate the highly accurate biological information 212 reflecting the cycle of the biological phenomenon of the living subject 102 .

[0098] The various processes performed in the above embodiments are not limited to the processing methods illustrated in the respective embodiments. The aforementioned functional blocks may be implemented using either a logic circuit (hardware) formed in an integrated circuit or software using a CPU. The various processes performed in the above embodiments may also be performed by multiple computers. For example, the processes performed by the control unit 204 may be partially performed by other computers, or the entire process may be shared among multiple computers.

[0099] The present invention is not limited to the above-described embodiments, and can be replaced with configurations that are substantially the same as those shown in the above-described embodiments, configurations that achieve the same effects, or configurations that can achieve the same purpose. The technical scope of the present disclosure also includes embodiments obtained by appropriately combining the technical solutions disclosed in different embodiments. Furthermore, new technical features can be formed by combining the technical methods disclosed in various embodiments.

Claims

1. A measuring device, characterized in that: It has: an imaging unit that images a living body and acquires images as time-series data; a time-series signal acquisition unit that acquires a time-series biological signal indicating a temporal change of a biological signal, and acquires a determination index temporally associated with the biological signal, the biological signal being a value related to the biological body calculated based on the image; a biological information calculation unit that divides the time-series biological signal into predetermined time intervals based on a cycle of a biological phenomenon of the organism and extracts a pulse signal; as well as a pulse determination unit for determining whether the determination indicator satisfies a predetermined determination condition, using the pulse signal that is temporally associated with the determination indicator that satisfies the determination condition; The biological information calculation unit calculates biological information related to the living body using the pulse signal used by the pulse determination unit.

2. The measuring device according to claim 1, wherein At the time indicating the time when the image at which the value of the biological signal is calculated is captured, the value of the biological signal is calculated based on the pixel values ​​of pixels within the region of interest included in the image captured at each of the time points.

3. The measuring device according to claim 2, wherein The judgment index represents the number of pixels in the region of interest, The judgment condition is that the number of pixels exceeds a first judgment threshold.

4. The measuring device according to claim 1 or 2, characterized in that The judgment index represents the displacement between the regions of interest included in the images acquired at different times. The judgment condition is that the displacement is below a second judgment threshold.

5. The measuring device according to claim 1 or 2, characterized in that The judgment index represents the number of movements of the living body per unit time calculated based on the image. The judgment condition is that the number of exercises is below a third judgment threshold.

6. The measuring device according to claim 5, characterized in that The number of movements is the number of blinks per unit time.

7. The measuring device according to claim 1 or 2, characterized in that The measuring device further includes an interference measuring unit configured to obtain interference information indicating a spatial variation of the measuring device. The judgment index represents the amount of change, The judgment condition is that the change amount is below a fourth judgment threshold.

8. The measuring device according to claim 1 or 2, characterized in that The biological information calculation unit calculates the biological information based on a plurality of pulse signals that satisfy at least one of a predetermined number of pulses or more and a predetermined time or more.

9. A measurement method, characterized in that: include: The process of photographing a living organism and acquiring images as time-series data; a step of acquiring a time-series biological signal representing a temporal change of a biological signal, and further acquiring a determination index temporally associated with the biological signal, wherein the biological signal is a value related to the biological body calculated based on the image; a step of dividing the time-series biological signal into predetermined time intervals based on a cycle of a biological phenomenon of the biological body and extracting a pulse signal; A process for determining whether the judgment indicator satisfies a prescribed judgment condition; as well as A step of calculating biological information related to the living body using the pulse signal temporally associated with the judgment index that satisfies the judgment condition.

Images