Apparatus and method for measurement

Abstract

To provide a measuring apparatus that can properly identify a date and time at which an image is filmed without missing the filmed image.SOLUTION: A measuring apparatus includes: an imaging part for imaging a living body and acquiring a plurality of images sequentially; and a time estimation part for estimating time at which each of the images is acquired on the basis of periodic changes appearing in an array of a plurality of pixel values respectively acquired from the plurality of images.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to a measuring device and a measuring method.

Background Art

[0002] Patent Document 1 discloses a technique for outputting a video in which a character string indicating the date and time imaged in a frame image is superimposed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the technique disclosed in Patent Document 1, the camera generates a frame image in which a character string indicating the imaging date and time is superimposed, and presents the imaging date and time of each frame image to the user. Therefore, in the technique disclosed in Patent Document 1, in the frame image, an image of a part of the camera's angle of view is missing due to the character string indicating the imaging date and time. Therefore, an aspect of the present disclosure aims to provide a measuring device and a measuring method capable of appropriately specifying the imaging date and time without missing the captured image.

Means for Solving the Problems

[0005] A measuring device according to an aspect of the present disclosure includes an imaging unit that captures a living body and sequentially acquires a plurality of images, and a time estimation unit that estimates the time at which each of the plurality of images was acquired based on a periodic variation appearing in a sequence of a plurality of pixel values respectively acquired from the plurality of images.

[0006] A measurement method according to one embodiment of the present disclosure includes a step of imaging a living body to sequentially acquire a plurality of images, and a step of estimating the time at which each of the plurality of images is acquired based on a periodic variation appearing in a sequence of a plurality of pixel values respectively acquired from the plurality of images.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Embodiments for Carrying Out the Invention

[0008] (First Embodiment) Referring to FIGS. 1 to 9, the first embodiment will be described. In the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant descriptions are omitted.

[0009] FIG. 1 is a diagram showing an example of the usage mode of the measuring device 100. As illustrated in FIG. 1, the measuring device 100 includes an imaging unit 101.

[0010] The measuring device 100 calculates a biological signal from a plurality of images acquired by the imaging unit 101 when the imaging unit 101 images the living body 102. For example, the measuring device 100 is a PC (Personal Computer), a smartphone, a tablet terminal, a dedicated pulse wave estimation terminal, or the like. For example, the biological signal is a pulse wave signal. In this specification, the pulse wave is a time-series signal indicating a change in the volume of blood vessels, which is calculated from a time-series signal indicating the pixel value of a pixel included in an image with respect to the same position on the body surface. In this specification, the pixel value is information indicating the brightness of a pixel included in an image, and is, for example, the pixel value of each pixel of R (Red), G (Green), B (Blue) or the luminance value of the pixel.

[0011] For example, the imaging unit 101 is composed of a CCD (Charged Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The imaging unit 101 may be composed of an image sensor for a camera including an RGB (Red Green Blue) filter.

[0012] FIG. 2 is a block diagram showing an example of the configuration of the measuring device 100 according to the present embodiment. The measuring device 100 includes an imaging unit 101, a storage unit 201, and a control unit 202.

[0013] The imaging unit 101 captures the living body 102 and sequentially acquires a plurality of images. Specifically, the imaging unit 101 captures the living body 102 and sequentially acquires a plurality of captured images at a frame rate Fr. The frame rate Fr is set so that aliasing distortion caused by regular brightening and dimming of the illumination occurs. Note that the imaging unit 101 may acquire each of the plurality of captured images at regular time intervals that are not the same time interval. The captured images include an image of the body surface of the living body 102. For example, the image of the body surface is an image of the face, cheek, forehead, palm, wrist, sole of the foot, etc. For example, the regular brightening and dimming of the illumination is caused by illumination flicker.

[0014] The storage unit 201 is a recording medium capable of recording various data, programs, etc., and is composed of, for example, a hard disk, SSD (Solid State Drive), semiconductor memory, etc.

[0015] The control unit 202 executes various processes according to the programs and data stored in the storage unit 201. The control unit 202 is realized by a processor such as a CPU (Central Processing Unit), for example. The control unit 202 includes a pixel value calculation unit 203, a time estimation unit 204, and a biological signal calculation unit 205.

[0016] The pixel value calculation unit 203 calculates a first representative pixel value 212 of pixel values of two or more pixels in the first region of interest from each of a plurality of processed images IMG that are at least a part of the plurality of captured images acquired by the imaging unit 101. Specifically, the pixel value calculation unit 203 sequentially acquires the latest captured image as the processed image IMG from among the plurality of captured images acquired by the imaging unit 101. Then, the pixel value calculation unit 203 calculates the first representative pixel value 212 of pixel values of two or more pixels in the first region of interest from the processed image IMG. Thereby, the pixel value calculation unit 203 calculates a plurality of first representative pixel values 212 from the plurality of processed images IMG, respectively. The first region of interest according to the present embodiment includes an image of the body surface of the living body 102. After calculating the first representative pixel value 212 from the processed image IMG, the pixel value calculation unit 203 acquires the next processed image IMG from the imaging unit 101.

[0017] The time estimation unit 204 estimates the time 213 at which each of the plurality of processed images IMG was acquired by the imaging unit 101 based on the periodic variations appearing in the columns of the plurality of pixel values respectively acquired from the plurality of processed images IMG. Specifically, the columns of the plurality of pixel values are columns of a plurality of first representative pixel values 212.

[0018] The biological signal calculation unit 205 calculates a biological signal from the plurality of first representative pixel values 212.

[0019] FIGS. 3 to 4 are flowcharts showing an example of the operation of the measuring device 100 according to the present embodiment.

[0020] In step S301, the imaging unit 101 starts acquiring a moving image by imaging the living body 102 at the frame rate Fr. That is, the imaging unit 101 starts acquiring a plurality of imaging images in time series by imaging the living body 102 at the frame rate Fr. The imaging images include images of the surface of the living body 102.

[0021] In step S302, the pixel value calculation unit 203 acquires the latest imaging image among the plurality of imaging images acquired by the imaging unit 101 as the processed image IMG. Specifically, the pixel value calculation unit 203 requests the imaging unit 101 for the latest acquired imaging image. The imaging unit 101 transmits the latest imaging image to the pixel value calculation unit 203 in response to the request from the pixel value calculation unit 203. The pixel value calculation unit 203 acquires the imaging image transmitted from the imaging unit 101 as the processed image IMG. Then, the pixel value calculation unit 203 stores the acquired processed image IMG in the storage unit 201 in order. When the measuring device 100 includes an internal clock (not shown) that measures the time indicated by hours, minutes, and seconds, the pixel value calculation unit 203 may associate the time measured by the internal clock when the first-frame processed image IMG was acquired with the first-frame processed image IMG and store the first-frame processed image IMG in the storage unit 201.

[0022] In step S303, the pixel value calculation unit 203 determines a first region of interest from the processed image IMG acquired in step S302. The first region of interest includes an image of a region of the body surface and is a region including a plurality of pixels. For example, when the processed image IMG includes an image of the face of the living body 102, the first region of interest includes an image of the cheek, an image of the forehead, or an image between the eyebrows. The number of first regions of interest may be one or more. The shape of the first region of interest may be a polygon surrounded by straight lines or a shape surrounded by curves. Alternatively, the first region of interest may be a closed region composed of straight lines and curves.

[0023] In step S304, the pixel value calculation unit 203 calculates a first representative pixel value 212 of the pixel values of the pixels within the first region of interest determined in step S303. For example, the first representative pixel value 212 is the average value, the median value, or the mode value of the pixel values of the pixels in the first region of interest. For example, when the imaging unit 101 is configured by an image sensor for a camera including an RGB (Red Green Blue) filter, the pixel value calculation unit 203 may calculate the first representative pixel value 212 of the pixel values of the R, G, and B pixels respectively within the first region of interest.

[0024] In step S305, the pixel value calculation unit 203 determines whether or not it has acquired the processed image IMG for a predetermined number of frames or more. The predetermined number of frames is the number of frames necessary and sufficient for calculating a biological signal in step S408 illustrated in FIG. 4 described later.

[0025] If no processed image IMG of a predetermined number of frames or more has been acquired in step S305, the control unit 202 returns the process to step S302 and continues the process. That is, if no processed image IMG of a predetermined number of frames or more has been acquired in step S305, the pixel value calculation unit 203 acquires the latest captured image acquired by the imaging unit 101 in step S302 as a new processed image IMG. While the control unit 202 executes the processes of steps S302 to S305, the imaging unit 101 continues the process of imaging the living body 102 at the frame rate Fr and sequentially acquiring a plurality of captured images. On the other hand, if a processed image IMG of a predetermined number of frames or more has been acquired in step S305, the control unit 202 shifts the process to step S401 illustrated in FIG. 4.

[0026] With reference to FIG. 4, the operation of the measuring device 100 according to the present embodiment will be continuously described.

[0027] When the imaging unit 101 images the living body 102 and acquires a captured image in an environment where regular light-on and light-off of illumination occurs, the pixel values of the pixels included in the captured image change periodically. When a periodic signal is observed at a frequency different from that of the signal in an environment where regular light-on and light-off of illumination occurs, a signal called aliasing distortion is observed. Therefore, when regular light-on and light-off of illumination occurs and the frame rate Fr is different from the frequency of the regular light-on and light-off of illumination, aliasing distortion is periodically observed in the signal indicating the time-series pixel values.

[0028] Specifically, in an environment where the illumination regularly turns on and off at a frequency of Fq [Hz], in the image of the n-th frame acquired by the imaging unit 101 at a frame rate of Fr [fps], an aliasing distortion of Fq - n×Fr [Hz] is observed. Due to the periodic occurrence of the aliasing distortion, it is assumed that the pixel values of the pixels included in the captured image acquired by the imaging unit 101 vary periodically in time series according to the frequency of the aliasing distortion. As a result, when the pixel value calculation unit 203 acquires the processed image IMG when the aliasing distortion occurs, it is assumed that the first representative pixel value 212 varies periodically. That is, when the pixel value calculation unit 203 acquires the processed image IMG when the aliasing distortion occurs, the periodic variation includes a frequency component due to the regular on and off of the illumination.

[0029] Therefore, when the imaging unit 101 captures the living body 102 at a frame rate Fr at which aliasing distortion can be observed and acquires a captured image, the time estimation unit 204 can estimate the time 213 at which the image, which is the processed image IMG, was acquired by the imaging unit 101 from the period of variation of the first representative pixel value 212 in time series due to the variation of the first representative pixel value 212 caused by the aliasing distortion. Note that the frame rate Fr is set so that the frequency component of the aliasing distortion is different from the frequency components included in the biological signal. The frequency components included in the biological signal are the frequency components indicating the change in pixel value due to the change in the volume of the blood vessels of the living body 102.

[0030] For example, in an area where the AC power supply is 100 Hz, flicker of 100 Hz occurs in the illumination and the illumination turns on and off. When the frame rate Fr is 50 fps under the illumination where 100 Hz flicker occurs, no aliasing distortion occurs and the first representative pixel value 212 does not vary. Therefore, the frame rate Fr of 50 fps is not a suitable value for the time estimation unit 204 to estimate the time 213 at which the processed image IMG was acquired by the imaging unit 101.

[0031] Therefore, in step S401, the time estimation unit 204 calculates an assumed fluctuation period, which is a period assumed for the fluctuation of the first representative pixel value 212. For example, the time estimation unit 204 calculates an assumed fluctuation period, which is a period in the periodic fluctuation of the pixel values of a plurality of pixels respectively included in a plurality of processed images IMG, based on the frequency of regular lighting on and off, the frame rate Fr, and the frequency of regular lighting on and off. Alternatively, for example, the time estimation unit 204 may calculate the assumed fluctuation period so as to conform to the fluctuation in the time series of the first representative pixel value 212 calculated in step S304 illustrated in FIG. 3.

[0032] In step S402, the time estimation unit 204 calculates an assumed fluctuation signal 801 (see FIG. 8), which is a signal having the assumed fluctuation period calculated in step S401. For example, the assumed fluctuation signal 801 is a sine wave having the calculated assumed fluctuation period and having a phase and an amplitude conforming to the fluctuation of the first representative pixel value 212 calculated in step S304 illustrated in FIG. 3.

[0033] In step S403, the time estimation unit 204 determines whether or not frame drop has occurred by comparing the assumed fluctuation signal 801 calculated in step S402 with the fluctuation in the time series of the first representative pixel value 212.

[0034] For example, when the pixel value calculation unit 203 executes the processes of steps S303 to S304, variations in the processing time are likely to occur depending on the way the living body 102 is imaged. Also, for example, when the measuring device 100 is a PC (Personal Computer), a smartphone, or the like, the control unit 202 executes a plurality of processes in parallel by a plurality of different application programs or the like. Therefore, for the processes of steps S303 to S304, the processing time may change for each processed image IMG.

[0035] When there are variations in the processing time for the processing from step S303 to step S304, as illustrated in FIG. 5, the pixel value calculation unit 203 may not be able to obtain all the captured images acquired by the imaging unit 101 as the processed image IMG, and frame drops may occur in the processing executed by the pixel value calculation unit 203.

[0036] Also, regarding the processing from step S303 to step S304, even when the processing time is constant for each processed image IMG, as illustrated in FIG. 6, the processing time from step S302 to step S305 may be longer than the exposure time in the imaging unit 101. In that case, the pixel value calculation unit 203 may generate a captured image that is not acquired as the processed image IMG, and frame drops may occur.

[0037] Therefore, for example, the time estimation unit 204 determines whether the minimum value of the difference between the assumed fluctuation signal 801 and the first representative pixel value 212 calculated from the processed image IMG exceeds a threshold value in the order in which the processed image IMG is acquired by the pixel value calculation unit 203. When the minimum value of the difference between the assumed fluctuation signal 801 and the first representative pixel value 212 does not exceed the threshold value, the time estimation unit 204 determines that no frame drop has occurred for the processed image IMG for which the first representative pixel value 212 is calculated. On the other hand, when the minimum value of the difference between the assumed fluctuation signal 801 and the first representative pixel value 212 exceeds the threshold value, the time estimation unit 204 determines that a frame drop has occurred immediately before the processed image IMG for which the first representative pixel value 212 is calculated.

[0038] For example, when a 100 Hz flicker occurs and the illumination regularly turns on and off, and when the frame rate Fr is 55 fps, if the first representative pixel value 212 varies in a 5.5 frame period, which is the assumed fluctuation period, in step S403, the time estimation unit 204 determines that no frame drop has occurred in the plurality of processed images IMG from the plurality of captured images acquired by the imaging unit 101.

[0039] On the other hand, when the flicker of 100 Hz occurs and the illumination regularly turns on and off, if the first representative pixel value 212 does not vary in a 5.5-frame period, which is the assumed fluctuation period, when the frame rate Fr is 55 fps, in step S403, the time estimation unit 204 determines that frame drops have occurred in the plurality of processed images IMG obtained from the plurality of captured images by the imaging unit 101. That is, the time estimation unit 204 calculates the assumed fluctuation period based on the frame rate Fr and the frequency of the regular on-off of the illumination. Then, when the assumed fluctuation signal having the calculated assumed fluctuation period does not match the plurality of first representative pixel values 212, it is determined that frame drops have occurred.

[0040] When it is determined in step S403 that no frame drops have occurred, in step S404, the time estimation unit 204 assigns a frame number to each of the plurality of processed images IMG in the order of acquisition. Then, the control unit 202 shifts the process to step S406.

[0041] On the other hand, when it is determined in step S403 that frame drops have occurred, in step S405, the time estimation unit 204 assigns the frame number at which the frame drop has occurred, including the frame numbers of the captured images that are each of the plurality of processed images IMG for which the first representative pixel value 212 is not calculated. Then, the control unit 202 shifts the process to step S406.

[0042] In step S406, the time estimation unit 204 estimates the time 213 at which the plurality of processed images IMG were obtained by the imaging unit 101 from the frame numbers assigned to each of the plurality of processed images IMG and the frame rate Fr.

[0043] The time estimation unit 204 estimates the time 213 at which each of the plurality of images acquired by the imaging unit 101 was acquired, based on the period of the folding strain. Therefore, when the imaging unit 101 acquires imaging images at a frame rate Fr at which the folding strain can be observed, the time estimation unit 204 can estimate the time 213 at which the processed image IMG was acquired by the imaging unit 101 from the periodic variation of the first representative pixel value 212.

[0044] For example, when the time measured by the internal clock is associated with the processed image IMG1 of the first frame, the time estimation unit 204 starts from the absolute time associated with the processed image IMG of the first frame, and the absolute time indicating the time at which the processed image IMG of the n-th frame was acquired by the imaging unit 101 is estimated from the product of the exposure time determined from the frame rate Fr and the frame number n. n is a natural number of 2 or more.

[0045] Alternatively, the time estimation unit 204 may estimate the relative time indicating the elapsed time from the time when the processed image IMG of the first frame was acquired to the time when the processed image IMG of the n-th frame was acquired by the imaging unit 101, from the product of the exposure time determined from the frame rate and the frame number n.

[0046] In step S407, the time estimation unit 204 associates the time 213 estimated in step S406 with the first representative pixel value 212 calculated from each processed image IMG. Therefore, by the processing of steps S401 to S407, the measuring device 100 according to the present embodiment can specify the date and time at which each processed image was appropriately imaged without missing the captured image.

[0047] In step S408, the biological signal calculation unit 205 calculates a biological signal from the plurality of first representative pixel values 212 associated with the time 213. For example, the biological signal calculation unit 205 processes a signal indicating the temporal change of the first representative pixel value 212 by independent component analysis and pigment component separation, and calculates the processed result as a biological signal. The temporal change of the first representative pixel value 212 includes information on the change in the volume of blood vessels. For example, the biological signal indicates a pulse wave signal.

[0048] In addition, when the biological signal calculation unit 205 performs digital filtering, trend removal, etc. on the calculated biological signal to remove noise from the calculated biological signal, it is preferable to associate the values indicated by the signal indicating the temporal change of the first representative pixel value 212 with time points at equal time intervals. Therefore, the biological signal calculation unit 205 may interpolate the values of the portions of the biological signal where frame drops have occurred based on the time 213 associated with each of the plurality of processed images IMG. For example, the biological signal calculation unit 205 interpolates the first representative pixel value 212 of the portion where frame drops have occurred, and processes the signal in which the first representative pixel value 212 of the portion where frame drops have occurred is interpolated, thereby interpolating the values of the portions of the biological signal where frame drops have occurred.

[0049] Alternatively, the biological signal calculation unit 205 may correct the signal indicating the temporal change of the first representative pixel value 212 to have values at equal time intervals by upsampling or downsampling the values indicated by the signal indicating the temporal change of the first representative pixel value 212 based on the time 213 associated with each of the plurality of processed images IMG. As described above, the biological signal calculation unit 205 can calculate the biological signal more accurately by interpolating the values of the portions of the biological signal where frame drops have occurred.

[0050] Note that the biological signal output by the biological signal calculation unit 205 may be indicated by associating the time 213 estimated by the time estimation unit 204 with the value indicated by the biological signal. In that case, the biological signal output by the biological signal calculation unit 205 may be in a state where frame drops occur at non-equal time intervals.

[0051] FIG. 5 is a diagram showing an example in which frame drops occur due to variations in processing time in the processing of steps S303 to S304 illustrated in FIG. 3.

[0052] At time t100, in step S301 illustrated in FIG. 3, the imaging unit 101 starts the process of acquiring a moving image at frame rate Fr. At each of the times t101 to t106, the imaging unit 101 images the living body 102 and sequentially acquires imaging images I11 to I16. The interval between each of the times from t100 to t106 is the exposure time determined by the frame rate Fr.

[0053] For example, the pixel value calculation unit 203 acquires the image I11 as the processing image IMG of the first frame. Then, after calculating the first representative pixel value 212 from the processing image IMG of the first frame, the pixel value calculation unit 203 acquires the latest imaging image I12 as the processing image IMG of the second frame.

[0054] Each of the times T111 to T114 indicates the processing time for each processing image IMG required for the processing in steps S303 to S304. For example, when the time T111 is shorter than the exposure time determined by the frame rate Fr, the pixel value calculation unit 203 calculates the first representative pixel value 212 from the processing image IMG of the first frame, which is the imaging image I11. After the imaging unit 101 acquires the imaging image I12, the pixel value calculation unit 203 acquires the latest imaging image I12 acquired at time t102 as the processing image IMG of the second frame.

[0055] Similarly, for example, when the time T112 is shorter than the exposure time determined by the frame rate Fr, the pixel value calculation unit 203 calculates the first representative pixel value 212 from the processing image IMG of the second frame, which is the imaging image I12. After the imaging unit 101 acquires the imaging image I13, the pixel value calculation unit 203 acquires the latest imaging image I13 acquired at time t103 as the processing image IMG of the third frame.

[0056] On the other hand, for example, when the time T113 is longer than the exposure time determined by the frame rate Fr, before the pixel value calculation unit 203 calculates the first representative pixel value 212 from the processed image IMG of the third frame, which is the captured image I13 acquired at the time point t103, the imaging unit 101 acquires the captured image I14 and the captured image I15. Therefore, the pixel value calculation unit 203 does not acquire the captured image I14 acquired at the time point t104 as the processed image IMG, but acquires the latest captured image I15 acquired at the time point t105 as the processed image IMG of the fourth frame.

[0057] Therefore, when there is a variation in the processing time for the processing from step S303 to step S304, the pixel value calculation unit 203 may not be able to acquire all the captured images acquired by the imaging unit 101 as the processed image IMG, and frame drops may occur in the processing executed by the pixel value calculation unit 203.

[0058] FIG. 6 is a diagram showing an example of a case where frame drops occur because the processing time for the processing from step S303 to step S304 illustrated in FIG. 3 is longer than the exposure time determined by the frame rate Fr. Regarding the time points t100 to t106 and the captured images I11 to I16, since they are the same as those illustrated in FIG. 5 for the time points t100 to t106 and the captured images I11 to I16, detailed descriptions are omitted. Each of the times T121 to T124 indicates the processing time for each processed image IMG that is longer than the exposure time determined by the frame rate Fr for the processing from step S303 to step S304.

[0059] For example, if each of time T121 to time T123 is longer than the exposure time determined by the frame rate Fr, it is assumed that the imaging unit 101 acquires the captured image I14 and the captured image I15 during the period of time T123. In that case, after the pixel value calculation unit 203 calculates the first representative pixel value 212 from the processed image IMG of the third frame, which is the captured image I13 acquired at time t103, the pixel value calculation unit 203 cannot acquire the captured image I14 acquired at time t104 as the processed image IMG, and acquires the latest captured image I15 acquired at time t105 as the processed image IMG of the fourth frame.

[0060] Therefore, when the processing time for the processing in steps S303 to S304 is longer than the exposure time, the pixel value calculation unit 203 cannot acquire all the captured images acquired by the imaging unit 101 as the processed image IMG, and frame drops may occur in the processing executed by the pixel value calculation unit 203.

[0061] FIG. 7 is a diagram showing an example of the variation in the time series of the first representative pixel value 212. In FIG. 7, the horizontal axis indicates the frame number assigned to the processed image IMG, and the vertical axis indicates the first representative pixel value 212.

[0062] For example, when 100 Hz flicker occurs and the illumination regularly turns on and off, if the frame rate Fr is 55 fps, a 10 Hz aliasing distortion occurs. In that case, as illustrated in FIG. 7, the first representative pixel value 212 varies at an assumed variation period that is a 5.5-frame period. When the first representative pixel value 212 varies at the assumed variation period, no frame drops occur in the processing executed by the pixel value calculation unit 203.

[0063] FIG. 8 is a diagram showing an example of the variation in the time series of the first representative pixel value 212 and the assumed variation signal 801. In FIG. 8, the horizontal axis indicates the frame number assigned to the processed image IMG, and the vertical axis indicates the first representative pixel value 212.

[0064] When frame drops occur in the process executed by the pixel value calculation unit 203, the assumed fluctuation signal 801 does not match the plurality of first representative pixel values 212. For example, in an environment where flicker at 100 Hz occurs and the illumination regularly turns on and off, when the frame rate Fr is 55 fps, the assumed fluctuation signal 801 is a sine wave having an assumed fluctuation period that is 5.5 frame periods.

[0065] When frame drops occur, as illustrated in FIG. 8, the assumed fluctuation signal 801 does not match the plurality of first representative pixel values 212. Specifically, when the pixel value calculation unit 203 does not acquire the captured image of the 8th frame as the processed image IMG and a frame drop occurs in the captured image of the 8th frame, as illustrated in FIG. 8, a difference occurs between the first representative pixel value 212 calculated from the processed image IMG of the 8th frame and the assumed fluctuation signal 801.

[0066] FIG. 9 is a diagram showing an example when frame numbers of the captured images are assigned to the processed image IMG from which the first representative pixel value 212 illustrated in FIG. 8 is calculated.

[0067] For example, when frame drops occur in the captured image of the 8th frame and the captured image of the 16th frame, as illustrated in FIG. 9, the frame numbers of the 8th frame and the 16th frame, which are the frames where frame drops occur, are added as frame numbers for which the first representative pixel value 212 is not calculated, and frame numbers are assigned to each of the plurality of processed images IMG. Then, in step S407 illustrated in FIG. 4, the time estimation unit 204 estimates the time at which each of the plurality of processed images IMG was acquired by the imaging unit 101 from the frame numbers assigned to each of the plurality of processed images IMG and the frame rate Fr. Then, based on the time estimated by the time estimation unit 204, the biological signal calculation unit 205 interpolates the values of the 8th frame and the 16th frame where frame drops occur among the values indicated by the biological signal. Thereby, the biological signal calculation unit 205 can calculate the biological signal more accurately.

[0068] As described above, the measuring device 100 according to the present embodiment can appropriately identify the date and time at which the captured image was captured without missing the captured image. Therefore, even when frame drops occur, the influence of the frame drops can be suppressed and the biological signal can be calculated.

[0069] (Second Embodiment) With reference to FIGS. 10 to 12, the second embodiment will be described. Regarding the drawings, the same or similar elements are denoted by the same reference numerals, and redundant descriptions are omitted. Configurations and processes having substantially the same functions as those of other embodiments are referred to by the same reference numerals and the descriptions thereof are omitted, and differences from other embodiments will be described.

[0070] FIG. 10 is a block diagram showing an example of the configuration of the measuring device 100 according to the present embodiment. The difference between the measuring device 100 illustrated in FIG. 10 and the measuring device 100 illustrated in FIG. 2 is that in the measuring device 100 illustrated in FIG. 10, the pixel value calculation unit 203 further calculates a second representative pixel value 1001 of the pixel values of the pixels in the second region of interest.

[0071] The pixel value calculation unit 203 according to the present embodiment calculates a first representative pixel value 212 of the pixel values of two or more pixels in a first region of interest that does not include an image of the body surface of the living body 102 from each of the plurality of processed images IMG. Thereby, the pixel value calculation unit 203 calculates a plurality of first representative pixel values 212 from the plurality of processed images IMG, respectively.

[0072] Furthermore, the pixel value calculation unit 203 according to the present embodiment calculates a second representative pixel value 1001 of the pixel values of two or more pixels in a second region of interest that includes an image of the body surface of the living body 102 from each of the plurality of processed images IMG. Thereby, the pixel value calculation unit 203 calculates a plurality of second representative pixel values 1001 from the plurality of processed images IMG, respectively.

[0073] The biological signal calculation unit 205 according to the present embodiment calculates a biological signal from the plurality of second representative pixel values 1001.

[0074] FIGS. 11 to 12 are flowcharts showing an example of the operation of the measuring device 100 according to the present embodiment. Since the processes of steps S1101 to S1102 illustrated in FIG. 11 are the same as those of steps S301 to S302 illustrated in FIG. 3, detailed description thereof will be omitted.

[0075] In step S1103, the pixel value calculation unit 203 determines a first region of interest included in the processing image IMG and not including an image of the body surface of the living body 102, and a second region of interest including an image of the body surface of the living body 102. The first region of interest according to the present embodiment indicates an image of a portion that does not move and whose color does not change within the imaging angle imaged by the imaging unit 101. For example, the first region of interest indicates an image of a wall, a ceiling, or the like. Since the second region of interest is the same as the first region of interest according to the first embodiment, detailed description thereof will be omitted.

[0076] In the image of the body surface of the living body 102, the pixel value periodically changes due to the aliasing distortion caused by the regular light and dark of the illumination, and the pixel value also periodically changes due to the change in the volume of the blood vessels of the living body 102. Therefore, in the image of the portion that does not move and whose color does not change, the periodic change in the pixel value due to the aliasing distortion caused by the regular light and dark of the illumination appears more clearly than in the image of the body surface of the living body 102.

[0077] In step S1104, the pixel value calculation unit 203 calculates a first representative pixel value 212 of the pixel values of the pixels in the first region of interest determined in step S1103. Since the process of step S1104 is the same as the process of step S304 illustrated in FIG. 3, detailed description thereof will be omitted.

[0078] In step S1105, the pixel value calculation unit 203 calculates the second representative pixel value 1001 of the pixel values of the pixels within the second region of interest determined in step S1103. For example, the second representative pixel value 1001 is the average value, median value, or mode value of the pixel values of the pixels in the second region of interest. For example, when the imaging unit 101 is configured by an image sensor for a camera including an RGB (Red Green Blue) filter, the pixel value calculation unit 203 may calculate the second representative pixel value 1001 of the pixel values of the R, G, and B pixels respectively within the second region of interest.

[0079] In step S1106, the pixel value calculation unit 203 determines whether or not a processed image IMG of a predetermined number of frames or more has been acquired. Since the process of step S1106 is the same as step S305 illustrated in FIG. 3, detailed description thereof is omitted.

[0080] If a processed image IMG of a predetermined number of frames or more has not been acquired in step S1106, the control unit 202 returns the process to step S1102 and continues the process. On the other hand, if a processed image IMG of a predetermined number of frames or more has been acquired in step S1106, the control unit 202 shifts the process to step S1201 illustrated in FIG. 12.

[0081] Referring to FIG. 12, the operation of the measuring device 100 according to the present embodiment will be continued. Since the processes of steps S1201 to S1202 illustrated in FIG. 12 are the same as the processes of steps S401 to S402 illustrated in FIG. 4, detailed description thereof is omitted.

[0082] In step S1203, the time estimation unit 204 determines whether a frame drop has occurred by comparing the assumed fluctuation signal 801 calculated in step S1202 with the fluctuation in the time series of the first representative pixel value 212. That is, the time estimation unit 204 determines whether a frame drop has occurred by comparing the assumed fluctuation signal 801 with the fluctuation of the first representative pixel value 212 calculated from the first region of interest including the image of the portion that does not move and whose color does not change. Since the image of the surface of the living body 102 is not included in the first region of interest and the image of the portion that does not move and whose color does not change is included, the pixel value does not change periodically due to the change in the volume of the blood vessels of the living body 102. Thus, the periodic change in the pixel value of the aliasing distortion due to the regular light and dark of the illumination clearly appears, and the time estimation unit 204 according to the present embodiment can determine whether a frame drop has occurred more accurately than the time estimation unit 204 in the first embodiment.

[0083] If no frame drop has occurred in step S1203, the control unit 202 proceeds to step S1204. On the other hand, if a frame drop has occurred in step S1203, the control unit 202 proceeds to step S1205. The processes of steps S1204 to S1206 are the same as the processes of steps S404 to S406 illustrated in FIG. 4, and thus detailed description thereof is omitted.

[0084] Since the first region of interest according to the present embodiment includes the image of the portion that does not move and whose color does not change within the imaging angle imaged by the imaging unit 101 and does not include the image of the surface of the living body 102, the time estimation unit 204 according to the present embodiment can estimate the time 213 at which the processed image IMG was acquired by the imaging unit 101 more accurately from the fluctuation in the time series of the first representative pixel value 212 than the time estimation unit 204 according to the first embodiment.

[0085] In step S1207, the time estimation unit 204 associates the second representative pixel value 1001 for the second region of interest with the time 213 at which the processing image IMG from which the second representative pixel value 1001 is calculated is acquired by the imaging unit 101. Then, the control unit 202 proceeds with the process to step S1208. Since the process of step S1208 is the same as step S408 illustrated in FIG. 4, a detailed description thereof is omitted.

[0086] As described above, in the measuring device 100 according to the present embodiment, a more accurate time than that of the measuring device 100 according to the first embodiment is associated with the second representative pixel value 1001, so that a biological signal can be calculated more accurately than the measuring device 100 according to the first embodiment.

[0087] (Modification example) As a modification example of the measuring device 100 according to the present embodiment, the first region of interest may include all the pixels of each of the processing images IMG. That is, the pixel value calculation unit 203 according to the present embodiment calculates the first representative pixel value 212 of the pixel values of all the pixels in each of the plurality of processing images IMG. Therefore, the measuring device 100 according to this modification example calculates the first representative pixel value 212 without executing a process of specifying a region that does not include an image of the surface of the living body 102. The measuring device 100 according to this modification example can reduce the processing time required for the process of specifying the first region of interest.

[0088] Each process executed in the above embodiment is not limited to the processing modes exemplified in each embodiment. The above-described functional blocks may be realized by either a logic circuit (hardware) formed in an integrated circuit or the like, or software using a CPU. Each process executed in the above embodiment may be executed by a plurality of computers. For example, for the process executed by the control unit 202, part of the process may be executed by another computer, or all the processes may be executed in a shared manner by a plurality of computers.

[0089] The present disclosure is not limited to the above-described embodiments, and may be replaced with a configuration that is substantially the same as the configuration shown in the above-described embodiments, a configuration that exhibits the same operational effects, or a configuration that can achieve the same purpose. The present disclosure also includes embodiments obtained by appropriately combining the technical means disclosed in different embodiments within the technical scope of the present disclosure. Furthermore, by combining the technical means disclosed in each embodiment, new technical features can be formed.

Explanation of Reference Numerals

[0090] 100 Measuring device, 101 Imaging unit, 102 Living body, 201 Storage unit, 202 Control unit, 203 Pixel value calculation unit, 204 Time estimation unit, 205 Biosignal calculation unit, 212 First representative pixel value, 213 Time, 801 Assumed fluctuation signal, 1001 Second representative pixel value

Claims

1. An imaging unit that captures a living body to sequentially acquire a plurality of images; A time estimation unit that estimates the time at which each of the plurality of images was acquired based on a periodic variation appearing in a sequence of a plurality of pixel values respectively acquired from the plurality of images; Comprising A measuring device.

2. The imaging unit sequentially acquires a plurality of captured images at a frame rate set so that aliasing distortion caused by regular on / off of illumination occurs, The plurality of images are at least a part of the plurality of captured images The measuring device according to Claim 1.

3. A pixel value calculation unit that calculates a plurality of first representative pixel values from the plurality of images by calculating a first representative pixel value of two or more pixels within a first region of interest from each of the plurality of images; Further comprising The plurality of pixel values are the plurality of first representative pixel values The measuring device according to Claim 2.

4. The time estimation unit calculates a period in the periodic variation based on the frame rate and the frequency of the on / off, and determines that a frame drop has occurred when a signal having the period does not match a plurality of first representative pixel values. The measuring device according to Claim 3.

5. The periodic variation includes a frequency component due to the on / off. The measuring device according to Claim 2.

6. The first region of interest includes an image of the surface of the living body, A biological signal calculation unit that calculates a biological signal from the plurality of first representative pixel values Further comprising The measuring device according to Claim 3.

7. The first region of interest does not include an image of the surface of the living body, The pixel value calculation unit calculates a plurality of second representative pixel values from the plurality of images by calculating a second representative pixel value of two or more pixels within a second region of interest from each of the plurality of images, A biological signal calculation unit that calculates a biological signal from the plurality of second representative pixel values Further comprising The measuring device according to Claim 3.

8. The biological signal indicates a pulse wave signal. The measuring device according to Claim 6 or 7.

9. A step of capturing a living body to sequentially acquire a plurality of images; A step of estimating the time at which each of the plurality of images was acquired based on a periodic variation appearing in a sequence of a plurality of pixel values respectively acquired from the plurality of images; Including A measuring method.

Images