Measuring device and measuring method

Abstract

To provide a measuring device capable of calculating biological information with a high degree of precision reflecting a cycle of a biological phenomenon of a living body.SOLUTION: A measuring device includes: a time-series signal acquisition unit for acquiring biological signals, which are time-series data of values of biological signals calculated from images obtained by imaging a living body and a determination index for the images; a biological information calculation unit for dividing the biological signals into pulse signals for each predetermined time on the basis of a cycle of a biological phenomenon; and a pulse determination unit for adopting a pulse signal consisting of values of the biological signal calculated from the image from which a determination index satisfying a determination condition can be calculated. The biological information calculation unit calculates biological information using the pulse signal adopted by the pulse determination unit.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a measurement device and a measurement method. [Background technology]

[0002] Patent Document 1 includes a human image detection unit that detects images of people, a biological signal detection unit that detects biological signals from people, and a control unit that outputs the detection results of the biological signal unit. If the biological value indicated by the biological signal is within a predetermined normal range and it is determined that a person is present and not moving in the detection area of ​​the human image detection unit, the vital sign detection device disclosed in Patent Document 1 determines that the biological signal detection unit is not malfunctioning and outputs biological information based on the biological signal. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-019882 Summary of the Invention [Problem to be solved by the invention]

[0004] The cycles of biological activity can reflect the health of an organism. For example, biological phenomena such as heart rate and breathing have cycles that increase and decrease repeatedly. Some biological signals obtained by measuring an organism reflect the cycles of these biological phenomena, and analyzing the periodicity of these signals can provide information about the organism's health. Examples include signals obtained by pulse sensors and pulse oximeters. However, if disturbances that adversely affect the measurement occur, the detected biological signals may contain noise, and these biological signals may not accurately reflect the periodicity based on biological phenomena. Note that "disturbances" here refer to unexpected influences from outside the system or undesirable interference that the device receives from outside, and do not refer to signals in a specific control system.

[0005] The technology disclosed in Patent Document 1 uses a pulse sensor and an image detection device, which outputs pulse information when an image of the driver is detected and does not output pulse information when an image of the driver is not detected. However, disturbances such as the driver's body movement can occur discontinuously on a second-by-second basis. Because the pulse reflects the periodic fluctuations of the heart, the pulse also fluctuates periodically over time in accordance with the movement. Generally, this periodicity is detected to calculate the pulse, so it is desirable to be able to detect it based on the pulse period. However, if image detection fails midway through the period, the pulse calculation will be interrupted, resulting in the pulse being calculated based on an incomplete pulse with a disrupted periodicity. As a result, the technology disclosed in Patent Document 1 may not be able to accurately calculate biometric information if a disturbance that adversely affects the measurement occurs. Therefore, one aspect of the present disclosure aims to provide a measurement device and measurement method that can accurately calculate biometric information that reflects the cycle of biological phenomena of a living organism. [Means for solving the problem]

[0006] A measurement device according to one aspect of the present disclosure includes an imaging unit that images a living organism and acquires images as time-series data; a time-series signal acquisition unit that acquires a time-series biosignal that indicates a temporal change in the biosignal, which is a value related to the living organism calculated from the image, and further acquires a judgment index that is temporally linked to the biosignal; a bioinformation calculation unit that divides the time-series biosignal into predetermined time intervals based on the period of a biological phenomenon of the living organism and extracts pulse signals; and a pulse determination unit that determines whether the judgment index satisfies predetermined judgment conditions and adopts the pulse signal that is temporally linked to the judgment index that satisfies the judgment condition, and the bioinformation calculation unit calculates bioinformation related to the living organism using the pulse signal adopted by the pulse determination unit.

[0007] A measurement method according to one aspect of the present disclosure includes the steps of imaging a living organism to obtain images as time-series data, obtaining a time-series biological signal indicating a temporal change in the biological signal, which is a value related to the living organism calculated from the image, and further obtaining a judgment index linked to the biological signal over time, dividing the time-series biological signal into predetermined time intervals based on the cycle of a biological phenomenon of the living organism to extract pulse signals, determining whether the judgment index satisfies predetermined judgment conditions, and calculating biological information related to the living organism using the pulse signals linked to the judgment index in a time-corresponding manner that satisfies the judgment conditions. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of how the measurement device is used. [Figure 2] 1 is a block diagram showing an example of the configuration of a measurement device according to a first embodiment. [Figure 3] FIG. 2 is a diagram illustrating an example of a region of interest. [Figure 4A] 10 is a graph showing the transition of a determination index indicating the number of pixels in a region of interest, and an example of a threshold value that is a determination threshold value. [Figure 4B] 4B is a graph showing an example of a pulse signal at a time point indicated by the transition of the determination index shown in FIG. 4A. [Figure 5] 4 is a flowchart showing an example of the operation of the measurement device according to the first embodiment. [Figure 6] FIG. 10 is a block diagram showing an example of the configuration of a measurement device according to a second embodiment. [Figure 7A] 10 is a graph showing an example of the transition of a determination index indicating the amount of change in a measurement device and a threshold value that is a determination threshold value. [Figure 7B] 7B is a graph showing an example of a pulse signal at a time point indicated by the transition of the amount of change in the measurement device illustrated in FIG. 7A. [Figure 8] 10 is a flowchart showing an example of the operation of the measurement device according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] (Embodiment 1) A first embodiment will be described with reference to Figures 1 to 5. In the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant explanations will be omitted.

[0010] 1 is a diagram showing an example of a usage mode of the measurement device 100. As shown in the example of FIG.

[0011] The measuring device 100 measures time-series changes in the state of the surface or inside of the skin of the living body 102 from images acquired as time-series data by the imaging unit 101 capturing images of the living body 102, and acquires biological information 212 (see FIG. 2). Note that the images here are images captured so that biological signals can be acquired as time-series data, and are images of the living body 102 captured in at least multiple frames, such as moving images or still images acquired as time-series data.

[0012] For example, the measuring device 100 is a PC (Personal Computer), a smartphone, a tablet terminal, a terminal dedicated to measuring biological information, or a monitoring robot equipped with an imaging unit 101. For example, the biological information 212 is blood pressure, pulse rate, respiratory rate, blood oxygen saturation, etc. Note that FIG. 1 shows a state in which the imaging unit 101 images the living organism 102 while the living organism 102 is holding the measuring device 100. However, the imaging unit 101 may also image the living organism 102 while the living organism 102 is not holding the measuring device 100.

[0013] The imaging unit 101 captures an image of a living body 102 and acquires the image as time-series data. The imaging unit 101 is installed in a position where it can capture an image of an exposed part of the body surface of the living body 102. The exposed part of the body surface of the living body 102 is the forehead, cheeks, fingertips, wrist, palm, etc. of the living body 102. For example, the imaging unit 101 is installed in a PC, a smartphone, a tablet, a display, a mirror, a washbasin, etc.

[0014] The image capturing unit 101 is a camera including a CCD (Charged Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and a lens. The image capturing unit 101 may be configured by an image sensor for a camera including an RGB filter. For example, the image capturing unit 101 captures an image of the skin of the living body 102. color In order to detect minute changes in blood volume, the imaging unit 101 is provided with a color filter in an RGB Bayer array. Alternatively, the imaging unit 101 may be provided with color filters such as RGBCy and RGBIR. Color filters such as RGBCy and RGBIR are suitable for observing increases and decreases in blood volume indicated by reflected light of light that has passed through the inside of the skin.

[0015] FIG. 2 is a block diagram showing an example of the configuration of the measurement device 100 according to this embodiment.

[0016] The measuring device 100 includes an imaging unit 101, an input unit 201, an output unit 202, a storage unit 203, and a control unit 204. The imaging unit 101, the input unit 201, the output unit 202, and the storage unit 203 are electrically connected to the control unit 204.

[0017] The imaging unit 101 captures an image of the living body 102 to obtain an image 211, and transmits the obtained image 211 to the control unit 204. For example, the imaging unit 101 captures an image of the living body 102 at 30 to 60 fps (frames per second) to obtain the image 211. The image 211 includes an image of the body surface of the living body 102.

[0018] The input unit 201 accepts input of information required for the measurement device 100. For example, the input unit 201 is a keyboard, a mouse, a touch panel, or the like.

[0019] The output unit 202 outputs the image 211, the biometric information 212, a message for the living body 102, the date and time when the image 211 was acquired, etc. For example, the output unit 202 is configured to include a display, a speaker, etc.

[0020] The control unit 204 executes various processes in accordance with the programs and data stored in the storage unit 203. The control unit 204 is configured by a processor such as a CPU (Central Processing Unit) or a GPU (Graphic Processing Unit), for example.

[0021] The control unit 204 includes a time-series signal acquisition unit 205 , a biological information calculation unit 206 , a pulse determination unit 207 , and a signal correction unit 208 .

[0022] The time-series signal acquisition unit 205 calculates a biosignal from the image 211. The image 211 is acquired as time-series data representing temporal changes in each frame of the image of the living body 102, so the time-series signal acquisition unit 205 can acquire a time-series biosignal 217, which is time-series data of the values ​​of the biosignal calculated from the image 211. For example, the value of the biosignal is a representative value or average value of pixel values ​​of multiple pixels included in the region of interest (ROI), and the time-series biosignal 217 is its temporal change. The time-series biosignal 217 associates the value of the time-series biosignal, i.e., the value of the biosignal calculated from the image 211, with the time point at which the value of the biosignal was acquired. Note that, here, the "time point at which the value of the biosignal was acquired" is defined as the time point at which the original frame from which the value of the biosignal was calculated in the image 211 was acquired by the imaging unit 101. In practice, the time-series signal acquisition unit 205 sets a region of interest (ROI) in a predetermined frame of the image 211 and calculates the value of the biosignal from the pixel values ​​of the pixels included in the region of interest (ROI). However, since the time when the living organism 102 is photographed is the timing at which the value of the biosignal reflecting the state of the living organism 102 at that time is obtained, the timing at which the frame from which the value of the biosignal is calculated is acquired by the imaging unit 101 is defined as the time at which the value of the biosignal is acquired from that frame. Note that the setting of the region of interest (ROI) and calculation processing using the image within the region of interest (ROI) can be performed to calculate the value of the biosignal without performing it for each frame, but performing it at shorter time intervals for each frame is preferable because it allows for more accurate acquisition of the biosignal as time-series data. Therefore, in this embodiment, the value of the biosignal is assumed to be obtained for each frame as time-series data.

[0023] Furthermore, the time-series signal acquisition unit 205 acquires a determination index for the image 211 at a predetermined timing, and acquires a determination index signal 216 that is time-series data of the determination index. The determination index according to this embodiment indicates the number of pixels in the region of interest ROI included in the image 211. In the present disclosure, the number of pixels is information indicating the number of pixels included in the image 211, and is, for example, the number of sub-pixels for each of R (Red), G (Green), and B (Blue) in the RGB space, or the number of pixels calculated from the sub-pixel values.

[0024] The biological information calculation unit 206 acquires pulse signals by dividing the time-series biological signal 217 in time based on the cycle of the biological phenomenon that the user wants to detect. Therefore, the pulse signal is linked in time to the time point when the value of the biological signal contained in the pulse signal was acquired. Examples of biological phenomena that can be detected include pulse, heart rate, breathing, blood pressure fluctuations, blood enzymes, etc., and are determined according to the type of biological information 212 that the user wants to detect.

[0025] The biological information calculation unit 206 calculates biological information 212 based on the corrected signal 218 generated by the signal correction unit 208. The biological information 212 indicates information about the living body 102 that the user wants to detect. For example, the biological information 212 indicates blood pressure, heart rate, blood oxygen saturation, etc.

[0026] The pulse determination unit 207 determines whether or not the pulse signal satisfies the determination condition based on the value of the determination index obtained from the determination index signal 216. Then, the pulse determination unit 207 adopts the pulse signal that is temporally linked to the determination index that satisfies the determination condition. The determination condition according to this embodiment is that the number of pixels in the region of interest ROI exceeds the determination threshold 215 (first determination threshold) according to this embodiment.

[0027] The signal corrector 208 generates a corrected signal 218 that is made up of the pulse signal adopted by the pulse determiner 207 .

[0028] The storage unit 203 is a recording medium capable of recording various data, programs, etc., and is configured by a hard disk, an SSD (Solid State Drive), a semiconductor memory, etc. The storage unit 203 includes a measurement information storage unit 213, a biological information storage unit 214, and a determination threshold 215. The storage unit 203 may store a plurality of determination thresholds 215.

[0029] The measurement information storage unit 213 stores pre-stored programs, information registered by the user, and the like, which are necessary for measuring the biological information 212. For example, the measurement information storage unit 213 stores a calculation formula for the biological information 212 and a calculation formula for the determination index. For example, the user is the administrator of the living body 102 or the measuring device 100.

[0030] The biological information storage unit 214 stores, as adopted pulse information, information that can identify which pulse signal has been adopted from among all pulse signals. For example, the adopted pulse information may include the adopted pulse signal itself, a unique number that can identify the adopted pulse signal, information such as the time when a determination index temporally linked to the adopted pulse signal was obtained, or the value of a biological signal included in the adopted pulse signal. The adopted pulse information may also include information related to the determination index, such as the time when the determination index satisfied the determination condition, or a unique number that can identify the time when the determination index satisfied the determination condition.

[0031] At least one of the measurement information storage unit 213 and the biological information storage unit 214 may store information related to the living body 102. For example, the information related to the living body 102 indicates a log of the biological information 212.

[0032] FIG. 3 is a diagram showing an example of the region of interest ROI.

[0033] The measuring device 100 according to this embodiment calculates biological information 212 from pixel values ​​of pixels included in a region of interest ROI set in an image 211.

[0034] In order to calculate the biometric information 212, it is desirable to select and use, as the region of interest ROI, an image of a part suitable for calculation, such as a part where the body surface is exposed and it is easy to observe changes in pixel values ​​of the image of the body surface due to changes in the volume of blood vessels, etc. For example, an image of the cheek of the living body 102 is suitable for calculating the biometric information 212.

[0035] For example, the time-series signal acquisition unit 205 sets a region of interest ROI in the image 211 using a face detection algorithm. Specifically, the time-series signal acquisition unit 205 extracts feature quantities related to facial features from the image 211 using a face detection algorithm based on pattern recognition, machine learning, or the like. For example, the feature quantities related to facial features indicate the positions and shapes of the eyes and nose. Then, the time-series signal acquisition unit 205 sets the position of the region of interest ROI based on the extracted feature quantities. The time-series signal acquisition unit 205 can acquire the position of the set region of interest ROI as coordinates within the image 211. For example, the region of interest ROI illustrated in FIG. 3 is rectangular, and the coordinates of its vertices are indicated by (x1, y1), (x1, y2), (x2, y1), and (x2, y2) within the image 211.

[0036] When the living body 102 holding the measuring device 100 shakes or when the living body 102 moves, the size of the image of the face of the living body 102 included in the image 211 may change. Specifically, the size of the image of the face included in the image 211 changes depending on the distance between the measuring device 100 and the living body 102. For example, the farther the living body 102 is from the measuring device 100, the smaller the image of the face included in the image 211 becomes. As a result, the number of pixels of the region of interest ROI included in the image 211 decreases.

[0037] If the number of pixels in the region of interest ROI is insufficient, the quality of the time-series biological signal 217 may deteriorate. In other words, a pulse signal acquired at a time point when the number of pixels P in the region of interest ROI is equal to or less than the threshold value Pth, which is the determination threshold 215, may not be suitable for accurately calculating the biological information 212. Therefore, it is desirable that the biological information calculation unit 206 does not use the pulse signal to calculate the biological information 212. For example, the determination condition is that the number of pixels P in the region of interest ROI is greater than the threshold value Pth, which is 25. In this case, the pulse determination unit 207 determines not to use a pulse signal including a value of a biological signal acquired at a time point when the number of pixels P in the region of interest ROI is equal to or less than 25.

[0038] The processing of pulse determination unit 207 will be described in detail with reference to Figures 4A and 4B. Figure 4A is a graph showing the transition of a determination index indicating the number of pixels P in a region of interest ROI, and an example of threshold value Pth, which is determination threshold 215. In Figure 4A, the horizontal axis represents time, and the vertical axis represents the number of pixels P in region of interest ROI, which is the determination index. Each point on the graph shown in Figure 4A represents the number of pixels P in region of interest ROI included in image 211 acquired at a different time point.

[0039] FIG. 4B is a graph showing an example of pulse signals at points indicated by the transition of the number of pixels P in the region of interest ROI shown in FIG. 4A. In FIG. 4B, the horizontal axis represents time, and the vertical axis represents the value of the biological signal. For example, when the time-series biological signal 217 represents a pulse wave signal, the value of the biological signal shown in FIG. 4B represents the intensity of the pulse wave. In FIG. 4B, 11 pulse signals are given the symbols tA to tK, respectively, as an example. Note that FIG. 4 A The vertical axis indicates the interval at which the judgment index is acquired. The vertical axis of Figure 4B showsThe calculation interval for the value of the biosignal is not limited to this, and it is sufficient that at least one judgment index is obtained for each pulse signal. For example, the value of the biosignal and the judgment index may be calculated for each frame of the image 211, or multiple judgment indexes may be obtained for each pulse signal and their average may be calculated. The shorter the time interval for obtaining the judgment index, the more detailed the disturbances such as the body movement of the living body 102 can be seen in time. However, if the time interval is made shorter than necessary, the calculation process becomes complicated, so it is desirable to determine the time interval according to the desired calculation accuracy of the biosignal. In this embodiment, both the value of the biosignal and the judgment index are calculated for each frame of the image 211, and the judgment index and the biosignal linked to the judgment index in time are obtained at the same time point, i.e., the judgment index is obtained at the same time point. index The acquisition interval and the calculation interval of the biological signal are the same. A The intervals for obtaining the judgment indicators shown in Figure 4 B The intervals at which the values ​​of the biological signals are calculated are not the same as those shown in FIG.

[0040] In this embodiment, as shown in FIG. 4B, the biological information calculation unit 206 divides a signal that varies over time based on the cycle of a biological phenomenon of the living body 102 into predetermined time intervals based on the cycle. Some biological phenomena have periodic fluctuations that reflect the state of the living body, such as breathing, heart rate, and blood pressure. For example, if the time-series biological signal 217 contains information about the blood vessels of the living body 102, reflection When the heart is contracting and expanding periodically, the intensity of the time-series biological signal 217 fluctuates over time in accordance with the pulsation period of the blood vessels caused by the heart's periodic contraction and expansion, resulting in a cycle of repeated increase and decrease. Actual biological phenomena are sensitive to health or mental states, so their periods are not necessarily constant, and may be disrupted, for example, as in arrhythmia. However, since biological phenomena generally change over time according to a certain period, the biological information calculation unit 206 can divide the time-series biological signal 217 at predetermined time intervals. Note that a biological signal obtained at a time point when the biological signal is disrupted to the extent that it cannot be divided may not be used as it is deemed to have poor signal quality, and the time-series biological signal 217 other than that time point may be used to divide the signal at predetermined time intervals.

[0041] In this manner, in this embodiment, the biological information calculation unit 206 divides the time-series biological signal 217 into a plurality of pulse signals based on the cycle of the biological phenomenon of the living organism 102. Specifically, as shown by pulse signals tA to tK illustrated in FIG. 4B, the time-series biological signal 217 is divided into pulse signals at predetermined times based on one cycle of the increase and decrease in the value of the biological signal. For example, assume that the pulse signal illustrated in FIG. 4B represents a pulse wave signal, and the type of biological information 212 is heart rate or blood pressure. In this case, as illustrated in FIG. 4B, the biological information calculation unit 206 sets one cycle as the period from the rising edge to the falling edge of the pulse wave accompanying the contraction and expansion of the heart. FIG. 4B shows each pulse signal divided into sections separated by dotted rectangles, which represent one cycle.

[0042] As described above, in order to measure the time-series biological signal 217 and calculate the biological information 212, the biological information calculation unit 206 acquires a plurality of pulse signals by dividing the time-series biological signal 217, which has temporal fluctuations, in time based on the cycle of a biological phenomenon of the living body 102. Then, it adopts a pulse signal that satisfies a determination condition from the pulse signals and calculates the biological information 212 using the adopted pulse signal. This allows the biological information calculation unit 206 to accurately calculate the biological information 212 based on the cycle of the biological phenomenon.

[0043] 4A and 4B, the number of pixels P, which is the determination index, exceeds the threshold value Pth before time t410 and after time t413. Therefore, pulse determination unit 207 employs pulse signals associated with time points before time t410 and after time t413. Specifically, pulse determination unit 207 employs pulse signals tA and pulse signals tF to tK.

[0044] On the other hand, the number of pixels P, which is the determination index from time t411 onwards and before time t412, is equal to or less than threshold Pth, which is determination threshold 215. Therefore, pulse determination unit 207 does not adopt pulse signals made up of values ​​of biological signals linked to time points from time t411 onwards and before time t412. Specifically, pulse determination unit 207 does not adopt pulse signals tB to tE. Note that pulse signal tE includes a portion of time points at which the number of pixels P is equal to or less than threshold Pth. Therefore, pulse determination unit 207 does not adopt pulse signal tE.

[0045] Then, information that can identify which pulse signals have been adopted from among all the pulse signals is stored as adopted pulse information. The signal correction unit 208 generates the correction signal 218 based on the adopted pulse information. This allows the signal correction unit 208 to generate the correction signal 218 that is composed of the adopted pulse signals.

[0046] The biometric information calculation unit 206 calculates the biometric information 212 from the corrected signal 218. Specifically, the biometric information calculation unit 206 calculates the biometric information 212 using a pulse signal that is linked in time to the point in time at which the determination index exceeds the threshold value Pth. That is, the biometric information calculation unit 206 calculates the biometric information 212 from the image 211 using pulse signal tA and pulse signals tF to tK illustrated in FIG. 4B.

[0047] Note that the lower the threshold value Pth, the more likely it is that the quality of the biometric information 212 will be degraded. On the other hand, the higher the threshold value Pth, the more likely it is that the number of pixels in the region of interest ROI will be reduced due to hand shake of the living body 102 holding the measuring device 100 or body movement of the living body 102, causing the determination index to not satisfy the determination condition, and the pulse signal will be rejected by the pulse determination unit 207. As a result, there is a risk that the time required from when the imaging unit 101 starts processing to capture an image of the living body 102 until the biometric information 212 is output will be longer. For this reason, it is preferable to set the threshold value Pth according to the pixel quality acceptable for the biometric information 212 and the waiting time acceptable by the living body 102.

[0048] FIG. 5 is a flowchart showing an example of the operation of the measurement device 100 according to this embodiment.

[0049] In step S501, the image capturing unit 101 starts processing to capture an image 211 by capturing an image of the living body 102.

[0050] In step S502, the time-series signal acquisition unit 205 calculates a biosignal from the image 211 acquired in step S501. Specifically, in step S502, the time-series signal acquisition unit 205 acquires a time-series biosignal 217, which is a temporal change in the biosignal that can be calculated from the image 211 acquired by the imaging unit 101. For example, the time-series signal acquisition unit 205 sets a region of interest (ROI) within the image 211 and calculates the time-series biosignal 217 from the RGB pixel values ​​of the pixels included in the region of interest (ROI) using a calculation formula pre-stored in the storage unit 203. Furthermore, as necessary, the time-series signal acquisition unit 205 may acquire the time-series biosignal 217 indicating the pulse wave converted into absorbance or the like from the temporal change in the value calculated by substituting the luminance values ​​of the pixels in the region of interest (ROI) of the image 211 into a predetermined formula. Alternatively, the time-series signal acquisition unit 205 may acquire the time-series biosignal 217 indicating the pulse wave using an independent component analysis method, a bioelement component separation method, or the like.

[0051] In step S503, the time-series signal acquisition unit 205 calculates a judgment index indicating the number of pixels P in the region of interest ROI, and acquires a judgment index signal 216, which is time-series data indicating the temporal fluctuation of the judgment index calculated from the image 211.

[0052] In step S504, the biological information calculation unit 206 divides the time-series biological signal 217 into pulse signals. Specifically, the biological information calculation unit 206 divides the time-series biological signal 217 into a plurality of pulse signals for each predetermined time based on one cycle of a biological phenomenon of the living organism 102. The cycle based on the biological phenomenon of the living organism 102 into which the pulse signal is divided is set according to the type of biological information 212 to be measured.

[0053] In step S505, the pulse determination unit 207 determines whether the determination index satisfies a determination condition, which is that the number of pixels in the region of interest ROI indicated by the determination index exceeds the determination threshold 215.

[0054] That is, the pulse determination unit 207 determines whether the number of pixels P is greater than the threshold value Pth. If the number of pixels P in the region of interest ROI exceeds the threshold value Pth, the pulse determination unit 207 determines that the determination index satisfies the determination condition. On the other hand, if the number of pixels P in the region of interest ROI is equal to or less than the threshold value Pth, the pulse determination unit 207 determines that the determination index does not satisfy the determination condition. The pulse determination unit 207 determines whether the determination index satisfies the determination condition for each time point of the determination index signal 216, which is time-series data.

[0055] When multiple determination thresholds 215 are stored in storage unit 203, pulse determination unit 207 may select a determination threshold 215 during measurement. Furthermore, input unit 201 may accept an operation to change determination threshold 215 to be used from multiple determination thresholds 215 stored in biological information storage unit 214, thereby causing pulse determination unit 207 to change determination threshold 215 to be used. For example, when there are multiple users, determination thresholds set according to the users may be stored in biological information storage unit 214, and an optimum determination threshold may be used for each user, thereby enabling more accurate biological information 212 to be obtained.

[0056] In step S506, the control unit 204 stores the time points at which the judgment indexes satisfied the judgment conditions and the time points at which the judgment indexes did not satisfy the judgment conditions separately in the measurement information storage unit 213. For example, the information indicating the time points is the time of day, identification information linked to the time points, etc. Note that the measurement information storage unit 213 does not necessarily need to store the time points at which the judgment indexes did not satisfy the judgment conditions.

[0057] In step S507, pulse determination unit 207 adopts a pulse signal linked to the time point at which the determination index satisfies the determination condition. Specifically, pulse determination unit 207 adopts a pulse signal consisting of biological signal values ​​calculated from pixel values ​​of the region of interest ROI of the image acquired at the time point at which the determination index satisfies the determination condition, and stores adopted pulse information related to the adopted pulse signal in measurement information storage unit 213.

[0058] If the imaging unit 101 does not capture an image of the living body 102 for a predetermined time required to properly calculate the biometric information 212, the accuracy of the biometric information 212 calculated by the biometric information calculation unit 206 may not be guaranteed. Similarly, if the total number of pulse signals used is less than the predetermined number of pulses required to properly calculate the biometric information 212, the accuracy of the biometric information 212 calculated by the biometric information calculation unit 206 may not be guaranteed.

[0059] Therefore, in step S508, pulse determination unit 207 determines whether or not a pulse signal having a predetermined number of pulses or more has been adopted. Specifically, pulse determination unit 207 determines whether or not a pulse signal having a predetermined number of pulses or more has been adopted after imaging of living body 102. The predetermined number of pulses is assumed to be stored in measurement information storage unit 213 in advance. The predetermined number of pulses may vary depending on the type of biological information 212 to be measured. Alternatively, in step S508, pulse determination unit 207 may determine whether or not a number of pulse signals has been adopted whose total duration satisfies a predetermined time or more. In this case, the predetermined time is assumed to be stored in measurement information storage unit 213 in advance. That is, in step S508, pulse determination unit 207 adopts a plurality of pulse signals that satisfy at least one of the conditions of a predetermined number of pulses or more and a predetermined time or more.

[0060] If a pulse signal that satisfies at least one of the predetermined number of pulses or the predetermined time or more is not adopted in step S508, control unit 204 returns the process to step S502. In the processes from step S502 onwards, pulse determination unit 207 adds a pulse signal that has been newly adopted after being linked to the already adopted pulse signal to the already adopted pulse signal, and stores the adopted pulse information in measurement information storage unit 213. This allows measurement device 100 to shorten the time required to adopt a number of pulse signals that satisfy at least one of the predetermined number of pulses or the predetermined time or more.

[0061] On the other hand, if pulse signals satisfying at least one of the predetermined number of pulses or more and the predetermined time or more are adopted in step S508, the signal correction unit 208 generates the corrected signal 218, which is time-series data consisting of only the adopted pulse signals, in step S509. As a result, the signal correction unit 208 generates the corrected signal 218 configured using pulse signals consisting of the values ​​of the biological signals linked to the time points at which the judgment index satisfies the judgment condition.

[0062] In step S510, the biological information calculation unit 206 calculates the biological information 212 from the corrected signal 218. That is, the biological information calculation unit 206 calculates the biological information 212 using the pulse signal adopted by the pulse determination unit 207. Specifically, the biological information calculation unit 206 calculates the biological information 212 using the pulse signal adopted by the pulse determination unit 207 that satisfies at least one of the conditions of a predetermined number of pulses or more and a predetermined time or more. Then, the output unit 202 outputs the biological information 212 as necessary. For example, the output unit 202 outputs the biological information 212 so that the biological information 212 is displayed on a display device (not shown) connected to the measurement device 100. Then, the control unit 204 ends the processing.

[0063] As described above, the measurement device 100 according to this embodiment judges the suitability of each pulse signal included in the time-series biological signal 217. Specifically, the measurement device 100 does not adopt a pulse signal associated with a time point where the number of pixels P in the region of interest ROI has significantly decreased and the judgment index does not exceed the judgment threshold 215, thereby not satisfying the judgment condition, but adopts a pulse signal of good quality associated with a time point where the number of pixels P exceeds the judgment threshold 215.

[0064] By calculating the biological information 212 using the values ​​of the biological signals contained in the pulse signal with good quality, the calculation accuracy can be improved compared to the biological information 212 calculated from the values ​​of all the biological signals contained in the time-series biological signal 217.

[0065] At this time, by making a determination for each pulse signal, the biological information 212 can be calculated using a biological signal whose values ​​based on one cycle of the biological phenomenon are completely consistent and which is suitable for measuring the biological information 212. As a result, the measuring device 100 according to this embodiment can calculate accurate biological information 212 that reflects the cycle of the biological phenomenon of the living body 102.

[0066] The pulse determination unit 207 may determine whether the time points at which the determination index satisfies the determination condition continue for a certain period of time or more. If the time points at which the determination index satisfies the determination condition continue for a certain period of time or more, the pulse determination unit 207 may employ a pulse signal indicating the value of a biosignal calculated from an image 211 acquired at the time point at which the determination index satisfies the determination condition. In this way, the measurement device 100 according to this embodiment can calculate accurate bioinformation 212 using a biosignal that continuously reflects the cycle of a biological phenomenon of the living organism 102 over time.

[0067] Furthermore, among the information stored in the storage unit 203, information other than that which needs to be saved, such as a user log, does not necessarily need to be stored long-term, and for example, the time when the judgment index satisfied the judgment condition or the identification information of the adopted pulse signal may be stored in a form temporarily stored as a variable during calculation of the biometric information 212 and deleted after the calculation is completed. Alternatively, the information may be stored even after the calculation is completed, and if stored long-term, it can be used for data analysis to improve the calculation accuracy of the biometric information 212, etc.

[0068] (Variation 1) A first modification of the first embodiment will be described. In the drawings, the same or similar elements are denoted by the same reference numerals, and redundant explanations will be omitted.

[0069] For example, when the living body 102 holding the measuring device 100 shakes or moves, the position of the region of interest ROI in the image 211 may move significantly. More specifically, the coordinates of the region of interest ROI in the image 211 may fluctuate significantly. Note that the coordinates here refer to the position in the image 211. Such fluctuations due to hand shake, body movement, etc. may act as disturbances, causing fluctuations in the values ​​of the biosignals and pulse signals. Therefore, the time-series signal acquisition unit 205 according to this modification calculates a determination index from the displacement of the coordinates of the region of interest ROI in the image 211 acquired at different times. For example, the time-series signal acquisition unit 205 according to this modification calculates a determination index from the displacement of some or all of the coordinates (x1, y1), (x1, y2), (x2, y1), and (x2, y2), which are the coordinates of the vertices of the region of interest ROI illustrated in FIG. 3, between different frames. That is, the determination index according to this modification indicates the displacement of the coordinates of the region of interest set in the images 211 acquired at different times. The determination condition according to this modification is that the displacement of the coordinates is equal to or less than the determination threshold value 215 (second determination threshold value) according to this modification.

[0070] For example, assume that the coordinates of the upper left vertex of the region of interest ROI included in the i-th frame image 211 are (x1i, y1i), where i is a natural number equal to or greater than 1. Also assume that the coordinates of the upper left vertex of the region of interest ROI included in the n-th frame image 211 are (x1n, y1n), where n is a natural number equal to or greater than i+1. The displacement of the upper left vertex of the region of interest ROI between the i-th frame and the n-th frame is represented by (x1n-x1i, y1n-y1i). Furthermore, assume that the determination threshold 215 according to this modification indicates xth, which is a threshold for the x-coordinate of the region of interest ROI, and yth, which is a threshold for the y-coordinate of the region of interest ROI. If x1n-x1i≦xth and y1n-y1i≦yth, the pulse determination unit 207 determines that the determination index satisfies the determination condition and employs a pulse signal associated with the time when the i-th frame to the n-th frame were acquired. On the other hand, if x1n-x1i>xth and y1n-y1i>yth, the pulse determination unit 207 determines that the determination index does not satisfy the determination condition, and does not adopt the pulse signal associated with the time point from the i-th frame to the n-th frame.

[0071] As described above, the measuring device 100 according to this modification does not use the value of the biological signal acquired at the time when the coordinates of the region of interest ROI have changed beyond the determination threshold 215 to calculate the biological information 212. As a result, the measuring device 100 according to this modification can calculate accurate biological information 212 that reflects the cycle of the biological phenomenon of the living body 102, even if the living body 102 holding the measuring device 100 shakes or the body of the living body 102 moves while the imaging unit 101 is imaging the living body 102.

[0072] The above-described determination conditions are merely examples and are not intended to limit the determination conditions according to this modification. For example, the determination condition may be x1n-x1i≦xth or y1n-y1i≦yth. The determination condition may be expressed only by the displacement of the x-coordinate in the region of interest ROI. If the region of interest ROI is rectangular, the determination condition may be expressed by the displacement of the coordinates of the four corners. The determination condition may be set according to the type of biometric information 212 to be measured and the required accuracy of the biometric information 212. In this modification, the displacement when the region of interest ROI moves is expressed using the coordinates of the region of interest ROI in the image 211. However, this is not a limitation. Any index that represents the change in the region of interest ROI may be used. However, it is preferable to use an index that can represent the displacement as a numerical value, such as a coordinate.

[0073] (Variation 2) A description will be given of Modification 2 of Embodiment 1. In the drawings, the same or similar elements are given the same reference numerals, and redundant explanations will be omitted.

[0074] When the living organism 102 makes a movement such as blinking, the skin on the face of the living organism 102 moves, which can cause a sudden fluctuation in the value of the biological signal, like spike noise. Due to such disturbances, the quality of the time-series biological signal 217 acquired while the living organism 102 is blinking is lower than the quality of the time-series biological signal 217 acquired when the living organism 102 is not blinking. Therefore, the time-series signal acquisition unit 205 according to this modification calculates a determination index indicating the number of movements of the living organism 102 per unit time, which is calculated from the image 211. For example, the number of movements indicated by the determination index is the number of blinks per unit time.

[0075] Specifically, the time-series signal acquiring unit 205 according to this modification calculates, for each pulse signal, the number of movements of the living organism 102 per unit time during the time that the biological signal included in one pulse signal is acquired. For example, the time-series signal acquiring unit 205 according to this modification calculates, as a determination index, the number of blinks per unit time of the image of the eyelid included in the image 211. The determination condition according to this modification is that the number of movements of the living organism 102 per unit time is equal to or less than the determination threshold 215 (third determination threshold) according to this modification. For example, the determination condition according to this modification is that the number of blinks of the living organism 102 per unit time is equal to or less than the determination threshold 215.

[0076] The determination threshold 215 according to this modification indicates a threshold for the number of movements per unit time. For example, the determination threshold 215 according to this modification indicates a threshold for the number of blinks per unit time. In this case, if the number of blinks indicated by the determination index exceeds the threshold for the number of blinks indicated by the determination threshold 215, the pulse determination unit 207 according to this modification determines that the determination index does not satisfy the determination condition. Therefore, the pulse signal calculated from the image 211 acquired at a time point included in the time when the number of blinks exceeds the threshold for the number of blinks indicated by the determination threshold 215 is not used.

[0077] As described above, when the measurement device 100 according to this modification acquires a number of times of movement, such as blinking, of the living organism 102 that exceeds the judgment threshold 215, the measurement device 100 according to this modification does not use the value of the biosignal calculated from the image acquired during the time when the movement was made, to calculate the bioinformation 212. As a result, even when the living organism 102 makes a movement, such as blinking, a number of times that exceeds the judgment threshold 215 per unit time, the measurement device 100 according to this modification can calculate accurate bioinformation 212 that reflects the cycle of the biological phenomenon of the living organism 102 by excluding the value of the biosignal acquired during the time when the movement, such as blinking, was made.

[0078] (Embodiment 2) A second embodiment will be described with reference to Figures 6 to 8. In the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant explanations will be omitted.

[0079] Fig. 6 is a block diagram showing an example of the configuration of a measurement device 600 according to this embodiment. The measurement device 600 shown in Fig. 6 differs from the measurement device 100 shown in Fig. 2 in that it includes a disturbance measurement unit 601 shown in Fig. 6.

[0080] If the imaging unit 101 images the living organism 102 when the living organism 102 moves while holding the measuring device 600, the measuring device 600 moves in accordance with the movement of the living organism 102. When the living organism 102 is holding the measuring device 600, body movements such as hand shaking, sneezing, coughing, or shaking of the living organism 102 by the living organism 102 can cause disturbances, which can cause noise or undesirable fluctuations in the biosignals calculated from the image 211.

[0081] Therefore, the measuring device 600 is equipped with a disturbance measurement unit 601 that acquires disturbance information 611 indicating the amount of change when the measuring device 600 fluctuates spatially based on the body movement or the intensity of the body movement of the living organism 102. The disturbance information 611 indicates at least one selected from the group consisting of the displacement of the measuring device 600 per unit time, the amount of change in the velocity of the measuring device 600, the amount of change in the acceleration of the measuring device 600, and the amount of change in the angular velocity of the measuring device 600. The measuring device 600 is equipped with a device that can measure the amount of spatial change of the measuring device 600 as time-series data, and acquires the disturbance information 611, which is time-series data of the amount of spatial change measured by the device. Examples of devices that can measure the amount of spatial change of the measuring device 600 include acceleration sensors, gyro sensors, etc. that can measure position, acceleration, angular velocity, etc.

[0082] The time-series signal acquiring unit 205 according to this embodiment calculates a determination index obtained from disturbance information 611, and acquires a determination index signal 612 that is time-series data of the determination index. That is, the determination index according to this embodiment indicates the amount of spatial change in the measurement device 600, and is temporally linked to a pulse signal made up of the value of a biological signal obtained from the image 211. The determination condition according to this embodiment is that the amount of spatial change in the measurement device 600 is equal to or less than a determination threshold 215 (fourth determination threshold) according to this embodiment.

[0083] The processing of pulse determination section 207 according to this embodiment will be described in detail with reference to FIGS. 7A and 7B.

[0084] Fig. 7A is a graph showing an example of the transition of the judgment index indicating the amount of change M of the measurement device 600, and the threshold value Mth, which is the judgment threshold 215. In Fig. 7A, the horizontal axis represents time, and the vertical axis represents the amount of change M indicated by the judgment index. Each point on the graph shown in Fig. 7A represents the amount of change M indicated by the disturbance information 611 acquired at a different time point.

[0085] 7B is a graph showing an example of a pulse signal at a time point indicated by the transition of the amount of change M shown in FIG. 7A. In FIG. 7B, the horizontal axis represents time, and the vertical axis represents the value of the biosignal. Note that in FIG. 7B, measurement continues after the pulse signal mG, but for the sake of explanation, this will be omitted below. Other than that, it is the same as FIG. 4B, and detailed explanation will be omitted.

[0086] 7A and 7B, the amount of change M indicated by disturbance information 611 before time t710 and after time t713 is greater than threshold Mth, which is determination threshold 215. Therefore, pulse determination section 207 does not adopt pulse signals including values ​​of biological signals calculated from images 211 acquired before time t710 and after time t713. Specifically, pulse determination section 207 determines not to adopt pulse signal mA and pulse signals mE to mG for pulse signals mA to mG.

[0087] On the other hand, since the amount of change M indicated by the determination index from time t711 onward and before time t712 is equal to or less than threshold value Mth, which is determination threshold 215, pulse determination unit 207 adopts pulse signals consisting of biosignal values ​​calculated from images 211 acquired from time t711 onward and before time t712. Specifically, pulse determination unit 207 adopts pulse signals mB to mD. Pulse determination unit 207 determines to adopt pulse signals including only biosignal values ​​calculated from images acquired at times when the amount of change M indicated by disturbance information 611 is equal to or less than threshold value Mth. For example, pulse signal mE in FIG. 7B includes both time t712 when the amount of change M indicated by disturbance information 611 is equal to or less than threshold value Mth and time t713 when the amount of change M is equal to or greater than threshold value Mth. However, pulse determination unit 207 does not adopt pulse signal mE because pulse signal mE includes at least one time point when the amount of change M is equal to or greater than threshold value Mth.

[0088] 7B, the biological information calculation unit 206 calculates the biological information 212 from the image 211. That is, the biological information calculation unit 206 calculates the biological information 212 using pulse signals consisting of biological signal values ​​calculated from the image at the time when the determination index is equal to or less than the threshold value Mth.

[0089] 8 is a flowchart showing an example of the operation of the measurement device 600 according to this embodiment. Note that detailed explanations of steps similar to the steps illustrated in FIG.

[0090] The difference between the flowchart illustrated in FIG. 8 and the flowchart illustrated in FIG. 5 is that the spatial change amount of the measuring device 600 indicated by the disturbance information 611 acquired by the disturbance measuring unit 601 is used as a judgment index.

[0091] In step S801, the image capturing unit 101 starts the process of acquiring the image 211, and the disturbance measuring unit 601 starts the process of acquiring the amount of change in the measuring device.

[0092] In step S802 , the time-series signal acquisition unit 205 acquires the time-series biological signal 217 from the image 211 acquired by the imaging unit 101 .

[0093] In step S803, the time-series signal acquisition unit 205 calculates a decision index from the amount of change indicated by the disturbance information 611, and acquires the decision index signal 216 as time-series data.

[0094] In step S804, the biological information calculation unit 206 divides the time-series biological signal 217 into pulse signals. The process of step S804 is the same as step S504 exemplified in FIG.

[0095] In step S805, the pulse determination unit 207 determines whether the determination index calculated in step S803 satisfies the determination condition. The determination condition according to this embodiment is that the amount of change M indicated by the disturbance information 611 is equal to or less than the threshold value Mth. That is, the pulse determination unit 207 determines whether the amount of change M indicated by the disturbance information 611 is equal to or less than the threshold value Mth. If the amount of change M indicated by the disturbance information 611 is equal to or less than the threshold value Mth, the pulse determination unit 207 determines that the determination index satisfies the determination condition. On the other hand, if the amount of change M indicated by the disturbance information 611 is greater than the threshold value Mth, the pulse determination unit 207 determines that the determination index does not satisfy the determination condition. The pulse determination unit 207 determines whether the determination index satisfies the determination condition for each time point of the determination index signal 216, which is time-series data. Then, the control unit 204 proceeds to step S806. The processing from step S806 onwards is the same as step S506 illustrated in FIG. 5, and therefore detailed description thereof will be omitted.

[0096] However, the method is not limited to this, and it is also possible to determine whether a disturbance is truly undesirable by also using disturbance information obtained from an image acquired by the imaging unit 101. In other words, if a disturbance obtained from an image occurs at approximately the same timing as a disturbance detected by the disturbance measurement unit 601, it may be determined that an undesirable disturbance has occurred, and the pulse signal including that time point may not be used. Specifically, if the time difference between a time point at which there is significant body movement, such as a significant movement of the face of the living body 102 detectable from an image included in the image 211, and a time point at which the amount of change M indicated by the disturbance information 611 does not satisfy the determination condition is smaller than a predetermined range, the pulse signal including that time point is not used. In this case, by comparing the information obtained from the image with the information obtained from the disturbance measurement unit 601, it is possible to determine whether a disturbance is truly undesirable, and more accurately remove the effects of the disturbance.

[0097] As described above, the measuring device 600 according to this embodiment can more accurately remove the influence of the body movement of the living body 102 and calculate accurate biological information 212 that reflects the cycle of the biological phenomenon of the living body 102.

[0098] The processes executed in the above embodiments are not limited to the processing modes exemplified in the above embodiments. The above-described functional blocks may be realized using either a logic circuit (hardware) formed in an integrated circuit or the like, or software using a CPU. The processes executed in the above embodiments may be executed by multiple computers. For example, some of the processes executed by the control unit 204 may be executed by another computer, or all of the processes may be shared and executed by multiple computers.

[0099] The present disclosure is not limited to the above-described embodiments, and may be replaced with a configuration that is substantially the same as the configuration shown in the above-described embodiments, a configuration that achieves the same effect, or a configuration that can achieve the same purpose. The present disclosure also includes within its technical scope embodiments obtained by appropriately combining the technical means disclosed in different embodiments. Furthermore, new technical features can be formed by combining the technical means disclosed in each embodiment. [Explanation of symbols]

[0100] 100 measuring device, 101 imaging unit, 102 living body, 201 input unit, 202 output unit, 203 storage unit, 204 control unit, 205 time series signal acquisition unit, 206 living body information calculation unit, 207 pulse determination unit, 208 signal correction unit, 211 image, 212 living body information, 213 measurement information storage unit, 214 living body information storage unit, 215 determination threshold, 216 determination index signal, 217 time series living body signal, 218 correction signal, 600 measuring device, 601 disturbance measurement unit, 611 disturbance information, 612 determination index signal

Claims

1. an imaging unit that captures images of a living body and acquires images as time-series data; a time-series signal acquisition unit that acquires a time-series biosignal indicating a temporal change in a biosignal, which is a value related to the living body calculated from the image, and further acquires a determination index temporally linked to the biosignal; a biological information calculation unit that divides the time-series biological signal into predetermined time intervals based on a cycle of a biological phenomenon of the living body and extracts pulse signals; a pulse determination unit that determines whether the judgment index satisfies a predetermined judgment condition and uses the pulse signal that is linked in time to the judgment index that satisfies the judgment condition for a certain period of time or more; Equipped with the biological information calculation unit calculates biological information related to the living body using the pulse signal adopted by the pulse determination unit; The certain period of time is a period of time that continuously reflects the period. Measuring equipment.

2. At a time point indicating a time when the image at which the value of the biological signal was calculated was captured, the value of the biological signal is calculated from pixel values ​​of pixels within a region of interest included in the image captured at each of the time points. The measuring device according to claim 1 .

3. the determination index indicates the number of pixels in the region of interest; The determination condition is that the number of pixels exceeds a first determination threshold. The measuring device according to claim 2 .

4. the determination index indicates a displacement between the regions of interest included in the images acquired at different times, The determination condition is that the displacement is equal to or less than a second determination threshold. The measuring device according to claim 2 .

5. the determination index indicates the number of movements of the living body per unit time calculated from the image, The determination condition is that the number of exercises is equal to or less than a third determination threshold.

3. The measuring device according to claim 1 or 2.

6. The number of movements is the number of blinks per unit time. The measuring device according to claim 5 .

7. a disturbance measuring unit for acquiring disturbance information indicating a spatial change amount of the measuring device; Further provided with the determination index indicates the amount of change, The determination condition is that the amount of change is equal to or less than a fourth determination threshold.

3. The measuring device according to claim 1 or 2.

8. a step of capturing an image of a living body and acquiring images as time-series data; a step of acquiring a time-series biosignal indicating a temporal change in a biosignal, which is a value related to the living body calculated from the image, and further acquiring a determination index linked to the biosignal over time; a step of extracting pulse signals by dividing the time-series biological signal into predetermined time intervals based on a cycle of a biological phenomenon of the living body; determining whether the judgment index satisfies a predetermined judgment condition; calculating biological information related to the living body using the pulse signal linked in time to the determination index, the determination index having a time point where the determination condition is satisfied for a certain period of time or more; Including, The certain period of time is a period of time that continuously reflects the period. Measurement method.

Images