Measurement device and measurement method

Abstract

Provided is a measurement device capable of appropriately determining the date and time of photographing without missing a photographed image. A measurement device is provided with: an imaging unit that captures an image of a living body and sequentially acquires a plurality of images; and a time estimation unit that estimates the time at which each of the plurality of images is acquired, on the basis of a periodic variation occurring in a column of a plurality of pixel values acquired from each of the plurality of images.

Description

Technical Field

[0001] The present disclosure relates to a measuring device and a measuring method. Background Art

[0002] Japanese Unexamined Patent Application Publication No. 2021-048527 discloses a technique for outputting an image in which a string indicating the photographed date and time is superimposed on a frame image. Summary of the Invention

[0003] In the technique disclosed in Japanese Unexamined Patent Application Publication No. 2021-048527, the camera generates a frame image on which a string indicating the shooting date and time is superimposed, and presents the shooting date and time of each frame image to the user. Therefore, in the technique disclosed in Japanese Unexamined Patent Application Publication No. 2021-048527, an image of a part of the field of view angle of the camera in the frame image is missing due to the string indicating the shooting date and time.

[0004] Therefore, an object of one aspect of the present disclosure is to provide a measuring device and a measuring method capable of appropriately determining the shooting date and time without missing the captured image.

[0005] A measuring device according to one aspect of the present disclosure includes: a photographing unit that photographs a living body and sequentially acquires a plurality of images; and a time estimation unit that estimates the times at which the plurality of images are respectively acquired based on periodic variations that appear in columns of a plurality of pixel values respectively acquired from the plurality of images.

[0006] A measuring method according to one aspect of the present disclosure includes: a step of photographing a living body and sequentially acquiring a plurality of images; and a step of estimating the times at which the plurality of images are respectively acquired based on periodic variations that appear in columns of a plurality of pixel values respectively acquired from the plurality of images. Brief Description of the Drawings

[0007] Figure 1 is a diagram showing an example of a usage mode of the measuring device. Figure 2 is a block diagram showing an example of the configuration of the measuring device according to the first embodiment. Figure 3 is a flowchart showing an example of the operation of the measuring device according to the first embodiment. Figure 4 is a continuation of the measuring device according to the first embodiment Figure 3 is a flowchart showing an example of the subsequent operation. Figure 5 is a diagram showing an example of a case where a frame drop occurs due to a deviation in processing time. Figure 6 is a diagram showing an example of a case where a frame drop occurs because the processing time is longer than the exposure time. Figure 7 It is a diagram showing an example of the change over time of the representative value of the pixel values of the pixels within the first region of interest. Figure 8 It is a diagram showing an example of the change over time of the representative value of the pixel values of the pixels within the first region of interest and an example of the assumed change signal. Figure 9 It is a diagram showing an example of the situation of assigning the frame numbers of the captured images to the images of the representative values calculated Figure 8 as exemplified. Figure 10 It is a block diagram showing an example of the configuration of the measuring device according to the second embodiment. Figure 11 It is a flowchart showing an example of the operation of the measuring device according to the second embodiment. Figure 12 It is a flowchart showing an example of the subsequent Figure 11 operation of the measuring device according to the second embodiment. Detailed Embodiment

[0008] (First Embodiment) Refer to Figures 1 to 9 to describe the first embodiment. In addition, for the drawings, the same or equivalent elements are given the same reference numerals, and repeated explanations are omitted.

[0009] Figure 1 It is a diagram showing an example of the usage mode of the measuring device 100. As Figure 1 shown, the measuring device 100 includes a photographing unit 101.

[0010] The measuring device 100 calculates a biological signal based on a plurality of images obtained by photographing the living body 102 by the photographing unit 101. For example, the measuring device 100 is a PC (Personal Computer), a smart phone, a tablet terminal, a dedicated pulse estimation terminal, or the like. For example, the biological signal is a pulse signal. In this specification, the pulse refers to a time-series signal representing the change in the volume of blood vessels, which is calculated based on the time-series signal representing the pixel values of the pixels included in the image, for the same position on the body surface. In this specification, the pixel value is information representing the brightness of the pixel included in the image, and is, for example, the pixel value of each pixel of R (Red), G (Green), and B (Blue) or the brightness value of the pixel.

[0011] For example, the imaging unit 101 is composed of a CCD (Charge Coupled Device) and a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The imaging unit 101 may also be composed of an image sensor for a camera including a filter containing RGB (Red Green Blue).

[0012] Figure 2 FIG. is a block diagram showing an example of the configuration of the measurement device 100 according to the present embodiment. The measurement device 100 includes an imaging unit 101, a storage unit 201, and a control unit 202.

[0013] The imaging unit 101 images the living body 102 and sequentially acquires a plurality of images. Specifically, the imaging unit 101 images the living body 102 and sequentially acquires a plurality of captured images at a frame rate Fr. The frame rate Fr is set to generate aliasing distortion caused by the regular flicker of illumination. In addition, the imaging unit 101 may acquire each of the plurality of captured images at regular time intervals that are not the same time intervals. The captured images include an image of the body surface of the living body 102. For example, the image of the body surface is an image of the face, an image of the cheek, an image of the forehead, an image of the palm, an image of the wrist, an image of the sole, etc. For example, the regular flicker of illumination is generated by the flicker of illumination.

[0014] The storage unit 201 is a recording medium capable of recording various data, programs, etc., and is composed of, for example, a hard disk, an SSD (SolidState Drive), a semiconductor memory, etc.

[0015] The control unit 202 performs various processes according to the programs and data stored in the storage unit 201. The control unit 202 is implemented by a processor such as a CPU (Central Processing Unit), for example. The control unit 202 includes a pixel value calculation unit 203, a time estimation unit 204, and a biological signal calculation unit 205.

[0016] The pixel value calculation unit 203 calculates a first representative pixel value 212 of two or more pixels within a first region of interest from each of a plurality of processed images IMG that are at least a part of the plurality of captured images acquired by the imaging unit 101. Specifically, the pixel value calculation unit 203 sequentially acquires the latest captured image among the plurality of captured images acquired by the imaging unit 101 as the processed image IMG. Then, the pixel value calculation unit 203 calculates the first representative pixel value 212 of two or more pixels within the first region of interest based on the processed image IMG. Thereby, the pixel value calculation unit 203 calculates a plurality of first representative pixel values 212 from each of the plurality of processed images IMG. The first region of interest in the present embodiment includes an image of the body surface of the living body 102. After calculating the first representative pixel value 212 from the processed image IMG, the pixel value calculation unit 203 acquires the next processed image IMG from the imaging unit 101.

[0017] The time estimation unit 204 estimates the times 213 at which the plurality of processed images IMG are respectively acquired by the imaging unit 101 based on the periodic variations in the occurrences of the columns of the plurality of pixel values respectively obtained from the plurality of processed images IMG. Specifically, the columns of the plurality of pixel values are the columns of the plurality of first representative pixel values 212.

[0018] The biological signal calculation unit 205 calculates a biological signal based on the plurality of first representative pixel values 212.

[0019] Figure Figure 4 is a flowchart showing an example of the operation of the measurement device 100 of the present embodiment.

[0020] In step S301, the imaging unit 101 captures the living body 102 at a frame rate Fr and starts acquiring a moving image. That is, the imaging unit 101 captures the living body 102 at a frame rate Fr and sequentially starts acquiring a plurality of captured images in time series. The captured images include an image of the body surface of the living body 102.

[0021] In step S302, the pixel value calculation unit 203 acquires the latest captured image among the multiple captured images acquired by the imaging unit 101 as the processing image IMG. Specifically, the pixel value calculation unit 203 requests the imaging unit 101 to acquire the latest captured image. In response to the request from the pixel value calculation unit 203, the imaging unit 101 sends the latest captured image to the pixel value calculation unit 203. The pixel value calculation unit 203 acquires the captured image sent from the imaging unit 101 as the processing image IMG. Then, the pixel value calculation unit 203 sequentially stores the acquired processing image IMG in the storage unit 201. Additionally, in the case where the measuring device 100 includes an internal clock (not shown) that measures time represented by hours, minutes, and seconds, the pixel value calculation unit 203 may also associate the processing image IMG of the first frame with the time measured by the internal clock when the processing image IMG of the first frame is acquired, and store the processing image IMG of the first frame in the storage unit 201.

[0022] In step S303, the pixel value calculation unit 203 determines a first region of interest based on the processing image IMG acquired in step S302. The first region of interest is a region that includes an image of a region of the body surface and includes a plurality of pixels. For example, in the case where the processing image IMG includes an image of the face of the living body 102, the first region of interest includes an image of the cheek, an image of the forehead, or an image between the eyebrows. The number of first regions of interest may be one or more. The shape of the first region of interest may be a polygon surrounded by straight lines, or a shape surrounded by curves. Alternatively, the first region of interest may also be a closed region formed by straight lines and curves.

[0023] In step S304, the pixel value calculation unit 203 calculates a first representative pixel value 212 of the pixel values of the pixels within the first region of interest determined in step S303. For example, the first representative pixel value 212 is the average value, median value, or mode value of the pixel values of the pixels in the first region of interest. For example, in the case where the imaging unit 101 is composed of an image sensor for a camera including a filter of RGB (Red Green Blue), the pixel value calculation unit 203 may also calculate the first representative pixel value 212 of the pixel values of R, G, and B of the pixels within the first region of interest.

[0024] In step S305, the pixel value calculation unit 203 determines whether processing images IMG of a specified number of frames or more have been acquired. The specified number of frames is the number of frames sufficient to calculate a biological signal in step S408 exemplified below. Figure 4

[0025] When the control unit 202 fails to obtain the processed image IMG for more than a specified number of frames in step S305, the control unit 202 returns the process to step S302 and continues the process. That is, when the control unit 202 fails to obtain the processed image IMG for more than a specified number of frames in step S305, the pixel value calculation unit 203 obtains the latest captured image acquired by the imaging unit 101 in step S302 as a new processed image IMG. In addition, during the period when the control unit 202 executes the process of step S305, the imaging unit 101 continues the process of capturing the living body 102 at the frame rate Fr and sequentially acquiring a plurality of captured images. On the other hand, when the control unit 202 obtains the processed image IMG for more than a specified number of frames in step S305, the control unit 202 transfers the process to Figure 4 step S401 exemplified.

[0026] Refer to Figure 4 and continue to describe the operation of the measuring device 100 of the present embodiment.

[0027] In an environment where regular flicker of illumination occurs, when the imaging unit 101 captures the living body 102 to obtain a captured image, the pixel values of the pixels included in the captured image change periodically. In an environment where regular flicker of illumination occurs, when a periodic signal is observed at a frequency different from the signal, a signal called aliasing distortion is observed. Therefore, in an environment where regular flicker of illumination occurs and the frame rate Fr is different from the frequency of the regular flicker of illumination, aliasing distortion is periodically observed in the signal representing the pixel values in time series.

[0028] Specifically, in an environment where the illumination regularly flickers at a frequency of Fq [Hz], the imaging unit 101 observes aliasing distortion of Fq - n × Fr [Hz] in the nth frame image acquired at the frame rate Fr [fps]. Assume that, by periodically generating aliasing distortion, the pixel values of the pixels included in the captured image acquired by the imaging unit 101 change periodically in time series according to the frequency of the aliasing distortion. As a result, it can be envisioned that when the processed image IMG is obtained in the case where aliasing distortion occurs in the pixel value calculation unit 203, the first representative pixel value 212 changes periodically. That is, when the pixel value calculation unit 203 obtains the processed image IMG in the case where aliasing distortion occurs, the periodic change includes a frequency component based on the regular flicker of the illumination.

[0029] Therefore, when the imaging unit 101 captures an image of the living body 102 at a frame rate Fr capable of observing folding distortion and obtains a captured image, the first representative pixel value 212 varies due to folding distortion. At this time, the time estimation unit 204 can estimate the time 213 when the image obtained by the imaging unit 101 as the processed image IMG is captured, based on the period of variation of the first representative pixel value 212 in the time series. In addition, the frame rate Fr is set such that the frequency component of folding distortion is different from the frequency component included in the biological signal. The frequency component included in the biological signal refers to the frequency component in which the pixel value changes due to the change in the volume of the blood vessels of the living body 102.

[0030] For example, in a region where the AC power supply is 100 Hz, 100 Hz flicker is generated in the lighting and the lighting flickers. Under the lighting where 100 Hz flicker is generated, when the frame rate Fr is 50 fps, no folding distortion occurs and the first representative pixel value 212 does not vary. Therefore, the frame rate Fr of 50 fps is not a value suitable for the time estimation unit 204 to estimate the time 213 when the imaging unit 101 captures the processed image IMG.

[0031] Therefore, in step S401, the time estimation unit 204 calculates a hypothetical variation period, which is the period hypothesized for the variation of the first representative pixel value 212. For example, the time estimation unit 204 calculates the period in the periodic variation of the pixel values of multiple pixels included in multiple processed images IMS respectively, based on the frame rate Fr and the regular flicker frequency of the lighting, as the hypothetical variation period. Or, for example, the time estimation unit 204 may also calculate the hypothetical variation period in a manner suitable for the variation in the time series of the first representative pixel value 212 calculated in step S304 as exemplified. Figure 3 In step S402, the time estimation unit 204 calculates a hypothetical variation signal 801 (refer to

[0032] ), and this hypothetical variation signal 801 is a signal having the hypothetical variation period calculated in step S401. For example, the hypothetical variation signal 801 is a sine wave, which has the calculated hypothetical variation period and has a phase and amplitude suitable for the variation of the first representative pixel value 212 calculated in step S304 as exemplified. Figure 8 In step S403, the time estimation unit 204 compares the hypothetical variation signal 801 calculated in step S402 with the variation of the first representative pixel value 212 in the time series to determine whether a frame drop has occurred. Figure 3 For example, when the pixel value calculation unit 203 executes step S303

[0033]

[0034] At the time of the processing in step S304, the processing time is likely to deviate according to the shooting method of the living body 102. In addition, for example, when the measuring device 100 is a PC (Personal Computer), a smart phone, etc., the control unit 202 executes a plurality of processes in parallel through a plurality of different application programs, etc. Therefore, regarding step S303 For the processing in step S304, the processing time may vary for each processed image IMG.

[0035] Regarding the processing from step S303 to step S304, when there is a deviation in the processing time, as Figure 5 illustrated, the pixel value calculation unit 203 cannot obtain all the captured images obtained by the imaging unit 101 as the processed image IMG, and the processing executed by the pixel value calculation unit 203 sometimes experiences dropped frames.

[0036] Moreover, regarding step the processing in step S304, even when the processing time is constant for each processed image IMG, as Figure 6 illustrated, step the processing time of step S305 may also be longer than the exposure time in the imaging unit 101. In this case, the pixel value calculation unit 203 sometimes generates captured images that are not obtained as the processed image IMG and experiences dropped frames.

[0037] Therefore, for example, the time estimation unit 204 determines, in the order of the processed image IMG obtained by the pixel value calculation unit 203, whether the minimum value of the difference between the assumed change signal 801 and the first representative pixel value 212 calculated based on the processed image IMG exceeds a threshold. When the minimum value of the difference between the assumed change signal 801 and the first representative pixel value 212 does not exceed the threshold, the time estimation unit 204 determines that no dropped frame has occurred for the processed image IMG for which the first representative pixel value 212 is calculated. On the other hand, when the minimum value of the difference between the assumed change signal 801 and the first representative pixel value 212 exceeds the threshold, the time estimation unit 204 determines that a dropped frame has occurred before the processed image IMG for which the first representative pixel value 212 is calculated.

[0038] For example, when a 100 Hz flicker is generated and the illumination regularly flickers, and the frame rate Fr is 55 fps, when the first representative pixel value 212 changes in a 5.5-frame period as the assumed change period, in step S403, the time estimation unit 204 determines that no dropped frame has occurred for the plurality of processed images IMG from the plurality of captured images obtained by the imaging unit 101.

[0039] On the other hand, in the case of generating a 100 Hz flicker and the illumination flickering regularly, when the frame rate Fr is 55 fps, if the first representative pixel value 212 does not change at the assumed change period, that is, the 5.5-frame period, in step S403, the time estimation unit 204 determines that a frame drop has occurred in the plurality of processed images IMG from the plurality of captured images acquired by the imaging unit 101. That is, the time estimation unit 204 calculates the assumed change period based on the frame rate Fr and the frequency of the regular flicker of the illumination. Then, when the assumed change signal with the calculated assumed change period does not match the plurality of first representative pixel values 212, it is determined that a frame drop has occurred.

[0040] When it is determined in step S403 that no frame drop has occurred, in step S404, the time estimation unit 204 assigns frame numbers to the plurality of processed images IMG in the order of acquisition. Then, the control unit 202 transfers the process to step S406.

[0041] On the other hand, when it is determined in step S403 that a frame drop has occurred, in step S405, the time estimation unit 204 assigns the frame numbers of the captured images to each of the plurality of processed images IMG including the frames where frame drops have occurred and for which the first representative pixel value 212 frame numbers have not been calculated. Then, the control unit 202 transfers the process to step S406.

[0042] In step S406, the time estimation unit 204 estimates the time 213 at which the imaging unit 101 acquires the plurality of processed images IMG based on the frame numbers assigned to the respective plurality of processed images IMG and the frame rate Fr.

[0043] The time estimation unit 204 estimates the time 213 at which each of the plurality of images acquired by the imaging unit 101 is acquired based on the period of the aliasing distortion. Therefore, when the imaging unit 101 acquires captured images at a frame rate Fr at which aliasing distortion can be observed, the time estimation unit 204 can estimate the time 213 at which the imaging unit 101 acquires the processed image IMG based on the periodic change of the first representative pixel value 212.

[0044] For example, when the time measured by the internal clock is associated with the first-frame processed image IMG1, the time estimation unit 204 estimates the absolute time starting from the absolute time associated with the first-frame processed image IMG, based on the product of the exposure time determined from the frame rate Fr and the frame number n, where the absolute time represents the time at which the imaging unit 101 acquires the nth-frame processed image IMG. n is a natural number of 2 or more.

[0045] Alternatively, the time estimation unit 204 may also estimate a relative time indicating the elapsed time until the time when the imaging unit 101 acquires the processing image IMG of the n-th frame, starting from the time when the processing image IMG of the first frame is acquired, based on the product of the exposure time determined from the frame rate and the frame number n.

[0046] In step S407, the time estimation unit 204 associates the time 213 estimated in step S406 with the first representative pixel value 212 calculated from each processing image IMG. Therefore, through the processing of steps S401 to S407, the measuring device 100 according to the present embodiment can determine the appropriate date and time of shooting for each processing image without missing the captured image.

[0047] In step S408, the biological signal calculation unit 205 calculates a biological signal based on a plurality of first representative pixel values 212 associated with the time 213. For example, the biological signal calculation unit 205 processes a signal representing the temporal change of the first representative pixel value 212 by independent component analysis or pigment component separation, and calculates the processed result as the biological signal. The temporal change of the first representative pixel value 212 includes information on the volume change of blood vessels. For example, the biological signal represents a pulse wave signal.

[0048] Further, in order to remove noise from the calculated biological signal, when the biological signal calculation unit 205 performs a digital filter, trend removal, etc. on the calculated biological signal, it is preferable to associate the time representing an equal time interval with the value represented by the signal representing the temporal change of the first representative pixel value 212. Therefore, the biological signal calculation unit 205 can also interpolate the values of the dropped frame portions in the biological signal based on the times 213 associated with the respective plurality of processing images IMG. For example, the biological signal calculation unit 205 interpolates the first representative pixel values 212 of the dropped frame portions and processes the signal obtained by interpolating the first representative pixel values 212 of the dropped frame portions, thereby interpolating the values of the dropped frame portions in the biological signal.

[0049] Alternatively, the biological signal calculation unit 205 can also correct the signal representing the temporal change of the first representative pixel value 212 to have values at equal time intervals by upsampling or downsampling the value represented by the signal representing the temporal change of the first representative pixel value 212 based on the times 213 associated with the respective plurality of processing images IMG. Through the above, the biological signal calculation unit 205 can calculate the biological signal more accurately by interpolating the values of the dropped frame portions in the biological signal.

[0050] In addition, the biological signal output by the biological signal calculation unit 205 may also show the association between the time 213 estimated by the time estimation unit 204 and the value represented by the biological signal. In this case, the biological signal output by the biological signal calculation unit 205 may also be in a state of dropping frames at unequal time intervals.

[0051] Figure 5 It represents for Figure 3 the steps exemplified An example of a diagram when the process of step S304 causes frame drops due to deviation in processing time.

[0052] At time t100, in Figure 3 the step S301 exemplified, the imaging unit 101 starts the process of acquiring a moving image at a frame rate Fr. At each time point from time t101 to time t106, the imaging unit 101 images the living body 102 to sequentially obtain captured images I11 to I16. Each time interval from time t100 to time t106 is the exposure time determined by the frame rate Fr.

[0053] For example, the pixel value calculation unit 203 acquires the image I11 as the processed image IMG of the first frame. Then, after the pixel value calculation unit 203 calculates the first representative pixel value 212 from the processed image IMG of the first frame, it acquires the latest captured image I12 as the processed image IMG of the second frame.

[0054] Time Time T114 respectively represent the processing time required for each processed image IMG to perform the process of step S304. For example, when the time T111 is shorter than the exposure time determined by the frame rate Fr, the pixel value calculation unit 203 calculates the first representative pixel value 212 from the processed image IMG of the first frame which is the captured image 111. After the imaging unit 101 acquires the captured image II2, the pixel value calculation unit 203 acquires the latest captured image I12 obtained at time t102 as the processed image IMG of the second frame.

[0055] Similarly, for example, when the time T112 is shorter than the exposure time determined by the frame rate Fr, the pixel value calculation unit 203 calculates the first representative pixel value 212 from the processed image IMG of the second frame which is the captured image II2. After the imaging unit 101 acquires the captured image I13, the pixel value calculation unit 203 acquires the latest captured image I13 obtained at time t103 as the processed image IMG of the third frame.

[0056] On the other hand, for example, when the time T113 is longer than the exposure time determined by the frame rate Fr, before the pixel value calculation unit 203 calculates the first representative pixel value 212 based on the processed image IMG which is the third frame of the captured image I13 obtained at time t103, the imaging unit 101 acquires the captured image I14 and the captured image I15. Therefore, the pixel value calculation unit 203 does not acquire the captured image I14 obtained at time point t104 as the processed image IMG, but acquires the latest captured image I15 obtained at time point t105 as the processed image IMG of the fourth frame.

[0057] Therefore, regarding step In the process of step S304, when there is a deviation in the processing time, the pixel value calculation unit 203 cannot acquire all the captured images acquired by the imaging unit 101 as the processed image IMG, and frame drops may occur in the process performed by the pixel value calculation unit 203.

[0058] Figure 6 It represents for Figure 3 The exemplified step In the process of step S304, when the processing time is longer than the exposure time determined by the frame rate Fr and frame drops occur, this is an example diagram. Regarding time points t100 to t106 and captured images I11 to I16, they are the same as the Figure 5 Exemplified time points t100 to t106 and captured images I11 to I16, so the detailed description is omitted. Time Time T124 respectively represent that, for the process of Step S304, the processing time of each processed image IMG is longer than the exposure time determined by the frame rate Fr.

[0059] For example, since each of the times T121 to T123 is longer than the exposure time determined by the frame rate Fr, during the time T123, the imaging unit 101 acquires the captured image I14 and the captured image I15. In this case, after the pixel value calculation unit 203 calculates the first representative pixel value 212 based on the processed image IMG which is the third frame of the captured image I13 obtained at time t103, the pixel value calculation unit 203 cannot acquire the captured image I14 obtained at time point t104 as the processed image IMG, and acquires the latest captured image I15 obtained at time point t105 as the processed image IMG of the fourth frame.

[0060] Therefore, regarding step In the process of step S304, when the processing time is longer than the exposure time, the pixel value calculation unit 203 cannot obtain all the captured images obtained by the imaging unit 101 as the processing image IMG, and frame drops sometimes occur in the process executed by the pixel value calculation unit 203.

[0061] Figure 7 is a diagram showing an example of the change over time of the first representative pixel value 212. In Figure 7 the horizontal axis represents the frame number assigned to the processing image IMG, and the vertical axis represents the first representative pixel value 212.

[0062] For example, when 100 Hz flickering occurs and the illumination flickers regularly, when the frame rate Fr is 55 fps, 10 Hz aliasing occurs. In this case, as Figure 7 illustrated, the first representative pixel value 212 changes with an assumed change period of 5.5 frame periods. When the first representative pixel value 212 changes with the assumed change period, no frame drops occur in the process executed by the pixel value calculation unit 203.

[0063] Figure 8 is a diagram showing an example of the change and the assumed change signal 801 in the time series of the first representative pixel value 212. In Figure 8 the horizontal axis represents the frame number assigned to the processing image IMG, and the vertical axis represents the first representative pixel value 212.

[0064] When frame drops occur in the process executed by the pixel value calculation unit 203, the assumed change signal 801 does not match the multiple first representative pixel values 212. For example, in an environment where 100 Hz flickering occurs and the illumination flickers regularly, when the frame rate Fr is 55 fps, the assumed change signal 801 is a sine wave with an assumed change period, and this assumed change period is 5.5 frame periods.

[0065] When frame drops occur, as Figure 8 illustrated, the assumed change signal 801 does not match the multiple first representative pixel values 212. Specifically, when the pixel value calculation unit 203 does not obtain the captured image of the eighth frame as the processing image IMG and frame drops occur for the captured image of the eighth frame, as Figure 8 illustrated, the first representative pixel value 212 calculated based on the processing image IMG of the eighth frame differs from the assumed change signal 801.

[0066] Figure 9 is a diagram showing an example when the frame number of the captured image is assigned to the processing image IMG for the first representative pixel value 212 Figure 8 illustrated.

[0067] For example, in the case where frame drops occur in the captured image of the eighth frame and the captured image of the sixteenth frame, as Figure 9 illustrated, the frame numbers of the eighth frame and the sixteenth frame are added as the frame numbers for which the first representative pixel value 212 has not been calculated, and a frame number is assigned to each of the plurality of processed images IMG. Then, in Figure 4 the step S407 illustrated, the time estimation unit 204 estimates the time at which the imaging unit 101 acquired each of the plurality of processed images IMG based on the frame numbers assigned to the respective plurality of processed images IMG and the frame rate Fr. Then, the biological signal calculation unit 205 interpolates the values of the eighth frame and the sixteenth frame in which frame drops have occurred among the values represented by the biological signal based on the time estimated by the time estimation unit 204. Thereby, the biological signal calculation unit 205 can calculate the biological signal more accurately.

[0068] Through the above, the measuring device 100 according to the present embodiment can appropriately determine the captured date and time without missing the captured image, and thus can suppress the influence of frame drops and calculate the biological signal even in the case of frame drops.

[0069] (Second Embodiment) Refer to Figures 10 to 12 to describe the second embodiment. Also, for the drawings, the same or equivalent elements are given the same reference numerals, and redundant descriptions are omitted. Structures and processes having substantially the same functions as those of other embodiments are referred to with common reference numerals and the descriptions are omitted, and the differences from other embodiments are described.

[0070] Figure 10 is a block diagram showing an example of the configuration of the measuring device 100 according to the present embodiment. Figure 10 The measuring device 100 illustrated is different from the measuring device 100 illustrated in Figure 2 that, in the measuring device 100 illustrated in Figure 10 the pixel value calculation unit 203 further calculates a second representative pixel value 1001 of the pixel values of the pixels in the second region of interest.

[0071] The pixel value calculation unit 203 of the present embodiment calculates a first representative pixel value 212 of the pixel values of two or more pixels in the first region of interest that does not include the image of the body surface of the living body 102 from each of the plurality of processed images IMG. Thus, the pixel value calculation unit 203 calculates a plurality of first representative pixel values 212 from each of the plurality of processed images IMG.

[0072] Further, the pixel value calculation unit 203 of the present embodiment calculates a second representative pixel value 1001 of two or more pixels in a second region of interest including an image of the body surface of the living body 102 from each of the plurality of processed images IMG. Thus, the pixel value calculation unit 203 calculates a plurality of second representative pixel values 1001 from each of the plurality of processed images IMG.

[0073] The biological signal calculation unit 205 of the present embodiment calculates a biological signal from the plurality of second representative pixel values 1001.

[0074] Figure Figure 12 is a flowchart showing an example of the operation of the measurement device 100 of the present embodiment. Figure 11 The processes of steps S1101 to S1102 shown are the same as Figure 3 the processes of steps S301 to S302 shown, and thus detailed description thereof is omitted.

[0075] In step S1103, the pixel value calculation unit 203 determines a first region of interest included in the processed image IMG and not including an image of the body surface of the living body 102 and a second region of interest including an image of the body surface of the living body 102. The first region of interest of the present embodiment represents an image of a portion that does not move and does not change in color within the field of view captured by the imaging unit 101. For example, the first region of interest represents an image such as a wall or a ceiling. The second region of interest is the same as the first region of interest of the first embodiment, and thus detailed description thereof is omitted.

[0076] In the image of the body surface of the living body 102, the pixel value periodically changes due to folding distortion caused by regular flickering of illumination, and the pixel value also periodically changes due to changes in the volume of blood vessels of the living body 102. Therefore, in an image of a portion that does not move and does not change in color, compared with the image of the body surface of the living body 102, the periodic change in the pixel value caused by folding distortion due to the influence of regular flickering of illumination is more clearly manifested.

[0077] In step S1104, the pixel value calculation unit 203 calculates a first representative pixel value 212 of the pixel values of the pixels in the first region of interest determined in step S1103. The process of step S1104 is the same as Figure 3 the process of step S304 shown, and thus detailed description thereof is omitted.

[0078] In step S1105, the pixel value calculation unit 203 calculates a second representative pixel value 1001 of the pixel values of the pixels within the second region of interest determined in step S1103. For example, the second representative pixel value 1001 is the average value, median value, or mode value of the pixel values of the pixels in the second region of interest. For example, when the imaging unit 101 is composed of an image sensor for a camera including an RGB (Red Green Blue) filter, the pixel value calculation unit 203 may also calculate the second representative pixel value 1001 of the pixel values of the R, G, and B pixels respectively within the second region of interest.

[0079] In step S1106, the pixel value calculation unit 203 determines whether a processed image IMG of a specified number of frames or more has been acquired. The process of step S1106 is the same as the process of step S305 Figure 3 shown, and thus detailed description thereof is omitted.

[0080] If a processed image IMG of a specified number of frames or more has not been acquired in step S1106, the control unit 202 returns the process to step S1102 and continues the process. On the other hand, if a processed image IMG of a specified number of frames or more has been acquired in step S1106, the control unit 202 transfers the process to Figure 12 step S1201 illustrated.

[0081] Referring to Figure 12 , the operation of the measuring device 100 of the present embodiment will be further described. Figure 12 The processes of steps S1201 to S1202 shown are the same as the processes of steps S401 to S402 Figure 4 shown, and thus detailed description thereof is omitted.

[0082] In step S1203, the time estimation unit 204 compares the assumed change signal 801 calculated in step S1202 with the change in the first representative pixel value 212 in the time series to determine whether a frame drop has occurred. That is, the time estimation unit 204 compares the assumed change signal 801 with the change in the first representative pixel value 212 calculated based on the first region of interest, which includes an image of a portion that does not move and whose color does not change, to determine whether a frame drop has occurred. Since the image of the body surface of the living body 102 is not included in the first region of interest, but an image of a portion that does not move and whose color does not change is included, the pixel values do not change periodically due to the change in the volume of the blood vessels of the living body 102. Therefore, by clearly showing the periodic change in the pixel values caused by the folding distortion due to the influence of the regular flicker of the illumination, compared with the time estimation unit 204 of the first embodiment, the time estimation unit 204 of the present embodiment can more accurately determine whether a frame drop has occurred.

[0083] When no frame drop occurs in step S1203, the control unit 202 transfers the process to step S1204. On the other hand, when a frame drop occurs in step S1203, the control unit 202 transfers to step S1205. The processes of steps S1204 to S1206 are the same as the processes of steps S404 to S406 shown in Figure 4 and thus the detailed description is omitted.

[0084] The first region of interest according to the present embodiment includes an image of a portion that does not move and whose color does not change in the field of view captured by the imaging unit 101, and does not include an image of the body surface of the living body 102. Therefore, the time estimation unit 204 according to the present embodiment can estimate the time 213 at which the imaging unit 101 acquires the processed image IMG more accurately based on the change in the time series of the first representative pixel values 212 compared to the time estimation unit 204 according to the first embodiment.

[0085] In step S1207, the time estimation unit 204 associates the second representative pixel value 1001 regarding the second region of interest with the time 213 at which the imaging unit 101 acquires the processed image IMG for which the second representative pixel value 1001 is calculated. Then, the control unit 202 transfers the process to step S1208. The process of step S1208 is the same as the process of step S408 shown in Figure 4 and thus the detailed description is omitted.

[0086] Through the above, the measuring device 100 according to the present embodiment associates a more accurate time with the second representative pixel value 1001 than the measuring device 100 according to the first embodiment, and thus can calculate the biological signal more accurately than the measuring device 100 according to the first embodiment.

[0087] (Modification example) As a modification example of the measuring device 100 of the present embodiment, the first region of interest may include all the pixels of each of the processed images IMG. That is, the pixel value calculation unit 203 of the present embodiment calculates the first representative pixel value 212 of the pixel values of all the pixels in the processed image IMG for each of the plurality of processed images IMG. Therefore, the measuring device 100 according to this modification example does not perform the process of determining the region that does not include the image of the body surface of the living body 102 and calculates the first representative pixel value 212. The measuring device 100 according to this modification example can reduce the processing time required for the process of determining the first region of interest.

[0088] Each process executed in the above-described embodiments is not limited to the processing methods illustrated in the respective embodiments. The above-described functional blocks can be implemented by either a logic circuit (hardware) formed in an integrated circuit or the like or software using a CPU. Each process executed in the above-described embodiments can also be executed by a plurality of computers. For example, with respect to the process executed by the control unit 202, part of the process can be executed by another computer, or all of the process can be executed by being shared among a plurality of computers.

[0089] The present invention is not limited to the above-described embodiments, and can be replaced with a configuration that is substantially the same as the configuration shown in the above-described embodiments, a configuration that achieves the same effects, or a configuration that can achieve the same object. In the present invention, embodiments obtained by appropriately combining technical solutions respectively disclosed in different embodiments are also included in the technical scope of the present invention. Moreover, new technical features can be formed by combining the technical methods respectively disclosed in the respective embodiments.

Claims

1. A measuring device, characterized in that: have: an imaging unit that images a living body and sequentially acquires a plurality of images; and A time estimation unit estimates the time at which each of the plurality of images is acquired based on a periodic variation appearing in a sequence of a plurality of pixel values ​​acquired from each of the plurality of images.

2. The measuring device according to claim 1, characterized in that The imaging unit sequentially acquires a plurality of captured images at a frame rate set to produce aliasing distortion caused by regular flickering of the illumination, The plurality of images are at least a portion of the plurality of captured images.

3. The measuring device according to claim 2, characterized in that The measuring device further includes a pixel value calculation unit that calculates a plurality of first representative pixel values ​​from the plurality of images by calculating first representative pixel values ​​of pixel values ​​of two or more pixels in the first region of interest from the plurality of images, respectively. The plurality of pixel values ​​are the plurality of first representative pixel values.

4. The measuring device according to claim 3, characterized in that The timing estimation unit calculates a period in the periodic variation based on the frame rate and the frequency of the flicker, and determines that a frame drop has occurred when a signal having the period does not match a plurality of first representative pixel values.

5. The measuring device according to claim 2, characterized in that The periodic variation includes a frequency component caused by the flicker.

6. The measuring device according to claim 3, characterized in that The first region of interest includes an image of the body surface of the living body, The measuring device further includes a biological signal calculation unit that calculates a biological signal based on the plurality of first representative pixel values.

7. The measuring device according to claim 3, characterized in that The first region of interest does not include an image of the body surface of the living body, The pixel value calculation unit calculates a plurality of second representative pixel values ​​from the plurality of images by calculating a second representative pixel value of pixel values ​​of two or more pixels in the second region of interest from each of the plurality of images, thereby calculating a plurality of second representative pixel values ​​from the plurality of images respectively. The measuring device further includes a biological signal calculation unit that calculates a biological signal based on the plurality of second representative pixel values.

8. The measuring device according to claim 6 or 7, characterized in that The biological signal is a pulse wave signal.

9. A measurement method, characterized in that: have: an imaging step of imaging the biological body to sequentially acquire a plurality of images; and The time estimation step estimates the time at which each of the plurality of images is acquired based on a periodic variation appearing in a sequence of a plurality of pixel values ​​acquired from each of the plurality of images.

Images