Measuring device and measuring method

Abstract

To provide a measuring apparatus that can properly identify a date and time at which an image is filmed without missing the filmed image.SOLUTION: A measuring apparatus includes: an imaging part for imaging a living body and acquiring a plurality of images sequentially; and a time estimation part for estimating time at which each of the images is acquired on the basis of periodic changes appearing in an array of a plurality of pixel values respectively acquired from the plurality of images.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a measurement device and a measurement method. [Background technology]

[0002] Patent Document 1 discloses a technique for outputting a video in which a character string indicating the date and time of capture is superimposed on a frame image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-048527 Summary of the Invention [Problem to be solved by the invention]

[0004] In the technology disclosed in Patent Document 1, a camera generates frame images on which a character string indicating the capture date and time is superimposed, thereby presenting the capture date and time of each frame image to a user. Therefore, in the technology disclosed in Patent Document 1, part of the image within the camera's angle of view is missing from the frame image due to the character string indicating the capture date and time. Therefore, one aspect of the present disclosure aims to provide a measurement device and a measurement method that can appropriately identify the capture date and time without missing any captured image. [Means for solving the problem]

[0005] A measurement device according to one embodiment of the present disclosure includes an imaging unit that images a living body and sequentially acquires multiple images, and a time estimation unit that estimates the time at which each of the multiple images was acquired based on periodic fluctuations that appear in a sequence of multiple pixel values ​​acquired from each of the multiple images.

[0006] A measurement method according to one embodiment of the present disclosure includes the steps of imaging a living body to sequentially obtain a plurality of images, and estimating the time at which each of the plurality of images was obtained based on periodic fluctuations that appear in a sequence of a plurality of pixel values ​​obtained from each of the plurality of images. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing an example of how the measurement device is used. [Figure 2] 1 is a block diagram showing an example of the configuration of a measurement device according to a first embodiment. [Figure 3] 4 is a flowchart showing an example of the operation of the measurement device according to the first embodiment. [Figure 4] 4 is a flowchart showing an example of the operation of the measurement device according to the first embodiment, following FIG. 3. [Figure 5] FIG. 10 is a diagram illustrating an example of a case where frame dropping occurs due to variations in processing time. [Figure 6] 10A and 10B are diagrams illustrating an example of a case where a frame drop occurs because the processing time is longer than the exposure time. [Figure 7] FIG. 10 is a diagram showing an example of fluctuations in time series of representative values ​​of pixel values ​​of pixels in a first region of interest. [Figure 8] 10A and 10B are diagrams showing an example of fluctuations in a time series of representative values ​​of pixel values ​​of pixels in a first region of interest and an expected fluctuation signal. [Figure 9] 9 is a diagram showing an example in which frame numbers of captured images are assigned to images for which representative values ​​are calculated, as exemplified in FIG. 8. FIG. [Figure 10] FIG. 10 is a block diagram showing an example of the configuration of a measurement device according to a second embodiment. [Figure 11] 10 is a flowchart showing an example of the operation of the measurement device according to the second embodiment. [Figure 12] 12 is a flowchart showing an example of the operation of the measurement device according to the second embodiment, following FIG. 11. DETAILED DESCRIPTION OF THE INVENTION

[0008] (First embodiment) The first embodiment will be described with reference to Figures 1 to 9. In the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant explanations will be omitted.

[0009] 1 is a diagram showing an example of a usage mode of the measurement device 100. As shown in the example of FIG.

[0010] In the measuring device 100, an imaging unit 101 captures images of a living body 102, and calculates a biosignal from multiple images acquired by the imaging unit 101. For example, the measuring device 100 is a PC (Personal Computer), a smartphone, a tablet terminal, a terminal dedicated to pulse wave estimation, or the like. For example, the biosignal is a pulse wave signal. In this specification, a pulse wave is a time-series signal that indicates changes in the volume of blood vessels, calculated from time-series signals that indicate pixel values ​​of pixels included in an image, for the same position on the body surface. In this specification, a pixel value is information that indicates the brightness of a pixel included in an image, and is, for example, the pixel value or luminance value of each R (Red), G (Green), and B (Blue) pixel.

[0011] For example, the imaging unit 101 is configured by a CCD (Charged Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) image sensor. The imaging unit 101 may be configured by an image sensor for a camera including RGB (Red Green Blue) filters.

[0012] 2 is a block diagram showing an example of the configuration of the measurement device 100 according to this embodiment. The measurement device 100 includes an imaging unit 101, a storage unit 201, and a control unit 202.

[0013] The imaging unit 101 captures an image of the living body 102 and sequentially acquires a plurality of images. Specifically, the imaging unit 101 captures an image of the living body 102 and sequentially acquires a plurality of captured images at a frame rate Fr. The frame rate Fr is set so that aliasing distortion caused by regular flickering of lighting occurs. Note that the imaging unit 101 may acquire each of the plurality of captured images at regular time intervals that are not the same time intervals. The captured images include images of the body surface of the living body 102. For example, the images of the body surface include images of the face, cheeks, forehead, palms, wrists, soles, etc. For example, regular flickering of lighting occurs due to lighting flicker.

[0014] The storage unit 201 is a recording medium capable of recording various data, programs, etc., and is configured by, for example, a hard disk, an SSD (Solid State Drive), a semiconductor memory, etc.

[0015] The control unit 202 executes various processes in accordance with the programs and data stored in the storage unit 201. The control unit 202 is realized by a processor such as a CPU (Central Processing Unit). The control unit 202 includes a pixel value calculation unit 203, a time estimation unit 204, and a biological signal calculation unit 205.

[0016] The pixel value calculation unit 203 calculates a first representative pixel value 212 of two or more pixels in a first region of interest from each of a plurality of processed images IMG, which are at least a portion of the plurality of captured images acquired by the imaging unit 101. Specifically, the pixel value calculation unit 203 sequentially acquires the most recent captured images from the plurality of captured images acquired by the imaging unit 101 as processed images IMG. Then, the pixel value calculation unit 203 calculates a first representative pixel value 212 of two or more pixels in the first region of interest from the processed images IMG. In this way, the pixel value calculation unit 203 calculates a plurality of first representative pixel values ​​212 from the plurality of processed images IMG. The first region of interest according to this embodiment includes an image of the body surface of the living body 102. After calculating the first representative pixel value 212 from the processed image IMG, the pixel value calculation unit 203 acquires the next processed image IMG from the imaging unit 101.

[0017] The time estimation unit 204 estimates the time 213 at which each of the plurality of processed images IMG was acquired by the imaging unit 101, based on periodic fluctuations that appear in a sequence of a plurality of pixel values ​​acquired from each of the plurality of processed images IMG. Specifically, the sequence of a plurality of pixel values ​​is a sequence of a plurality of first representative pixel values ​​212.

[0018] The biological signal calculation unit 205 calculates a biological signal from the plurality of first representative pixel values ​​212.

[0019] 3 and 4 are flowcharts showing an example of the operation of the measurement device 100 according to this embodiment.

[0020] In step S301, the imaging unit 101 starts capturing moving images of the living body 102 at a frame rate Fr. That is, the imaging unit 101 starts capturing images of the living body 102 at the frame rate Fr and sequentially capturing multiple captured images in chronological order. The captured images include images of the body surface of the living body 102.

[0021] In step S302, the pixel value calculation unit 203 acquires the latest captured image from the multiple captured images acquired by the imaging unit 101 as a processed image IMG. Specifically, the pixel value calculation unit 203 requests the latest captured image from the imaging unit 101. In response to the request from the pixel value calculation unit 203, the imaging unit 101 transmits the latest captured image to the pixel value calculation unit 203. The pixel value calculation unit 203 acquires the captured image transmitted from the imaging unit 101 as a processed image IMG. Then, the pixel value calculation unit 203 sequentially stores the acquired processed images IMG in the storage unit 201. Note that if the measurement device 100 includes an internal clock (not shown) that measures time indicated by hours, minutes, and seconds, the pixel value calculation unit 203 may associate the processed image IMG of the first frame with the time measured by the internal clock when the processed image IMG of the first frame was acquired, and store the processed image IMG of the first frame in the storage unit 201.

[0022] In step S303, the pixel value calculation unit 203 determines a first region of interest from the processed image IMG acquired in step S302. The first region of interest is a region including an image of a body surface area and including a plurality of pixels. For example, if the processed image IMG includes an image of the face of the living body 102, the first region of interest includes an image of the cheek, the forehead, or the area between the eyebrows. The number of first regions of interest may be one or more. The shape of the first region of interest may be a polygon surrounded by straight lines or a shape surrounded by curved lines. Alternatively, the first region of interest may be a closed region formed by straight lines and curved lines.

[0023] In step S304, pixel value calculation unit 203 calculates first representative pixel values ​​212 of the pixel values ​​of the pixels in the first region of interest determined in step S303. For example, first representative pixel value 212 is the average value, median, or mode of the pixel values ​​of the pixels in the first region of interest. For example, if imaging unit 101 is configured with an image sensor for a camera including RGB (Red, Green, Blue) filters, pixel value calculation unit 203 may calculate first representative pixel values ​​212 of the pixel values ​​of each of R, G, and B pixels in the first region of interest.

[0024] In step S305, the pixel value calculation unit 203 determines whether or not a predetermined number of frames or more of processed images IMG have been acquired. The predetermined number of frames is the number of frames necessary and sufficient for calculating a biological signal in step S408 illustrated in FIG. 4, which will be described later.

[0025] If the predetermined number of frames or more of processed images IMG have not been acquired in step S305, the control unit 202 returns the process to step S302 and continues the process. That is, if the predetermined number of frames or more of processed images IMG have not been acquired in step S305, the pixel value calculation unit 203 acquires the latest captured image acquired by the imaging unit 101 in step S302 as a new processed image IMG. Note that while the control unit 202 is executing the processes of steps S302 to S305, the imaging unit 101 continues the process of capturing images of the living body 102 at a frame rate Fr and sequentially acquiring multiple captured images. On the other hand, if the predetermined number of frames or more of processed images IMG have been acquired in step S305, the control unit 202 proceeds to step S401 illustrated in FIG. 4.

[0026] The operation of the measurement device 100 according to this embodiment will be further described with reference to FIG.

[0027] When the imaging unit 101 captures an image of a living body 102 in an environment where regular flickering of lighting occurs and acquires the captured image, the pixel values ​​of the pixels included in the captured image change periodically. In an environment where regular flickering of lighting occurs, if a periodic signal is observed at a frequency different from the signal, a signal called aliasing is observed. Therefore, when regular flickering of lighting occurs and the frame rate Fr is different from the frequency of the regular flickering of lighting, aliasing is periodically observed in the signal indicating the time-series pixel values.

[0028] Specifically, in an environment where lighting flickers regularly at a frequency of Fq [Hz], aliasing of Fq-n×Fr [Hz] is observed in the nth frame image acquired by the imaging unit 101 at a frame rate of Fr [fps]. It is expected that the periodic occurrence of aliasing will cause the pixel values ​​of pixels included in the captured image acquired by the imaging unit 101 to periodically fluctuate over time in accordance with the frequency of the aliasing. As a result, if the pixel value calculation unit 203 acquires a processed image IMG when aliasing occurs, it is expected that the first representative pixel value 212 will periodically fluctuate. In other words, if the pixel value calculation unit 203 acquires a processed image IMG when aliasing occurs, the periodic fluctuation will include a frequency component due to the regular flickering of lighting.

[0029] Therefore, when the imaging unit 101 captures an image of the living body 102 at a frame rate Fr at which aliasing can be observed, the time estimation unit 204 can estimate the time 213 at which the image, which is the processed image IMG, was captured by the imaging unit 101 from the period at which the first representative pixel value 212 fluctuates in time series due to the first representative pixel value 212 fluctuating due to aliasing. Note that the frame rate Fr is set so that the frequency components of the aliasing are different from the frequency components contained in the biological signal. The frequency components contained in the biological signal are frequency components that indicate changes in pixel values ​​due to changes in the volume of blood vessels in the living body 102.

[0030] For example, in an area where the AC power supply is 100 Hz, lighting flickers at 100 Hz, causing the lighting to blink. If the frame rate Fr is 50 fps under lighting that generates 100 Hz flicker, aliasing distortion does not occur, and the first representative pixel value 212 does not fluctuate. Therefore, the frame rate Fr of 50 fps is not a suitable value for the time estimation unit 204 to estimate the time 213 at which the processed image IMG was acquired by the imaging unit 101.

[0031] Therefore, in step S401, the time estimation unit 204 calculates an expected fluctuation period, which is an expected period for fluctuations in the first representative pixel value 212. For example, the time estimation unit 204 calculates an expected fluctuation period, which is a period for periodic fluctuations in pixel values ​​of multiple pixels in multiple processed images IMG, based on the frequency of regular blinking of lighting, the frame rate Fr, and the frequency of regular blinking of lighting. Alternatively, for example, the time estimation unit 204 may calculate the expected fluctuation period so as to fit the fluctuations in the time series of the first representative pixel value 212 calculated in step S304 illustrated in FIG.

[0032] In step S402, the time estimation unit 204 calculates an expected fluctuation signal 801 (see FIG. 8), which is a signal having the expected fluctuation period calculated in step S401. For example, the expected fluctuation signal 801 is a sine wave having the calculated expected fluctuation period and having a phase and amplitude that match the fluctuation of the first representative pixel value 212 calculated in step S304 illustrated in FIG.

[0033] In step S403, the time estimation unit 204 compares the estimated fluctuation signal 801 calculated in step S402 with the fluctuation in the time series of the first representative pixel value 212 to determine whether a frame drop has occurred.

[0034] For example, when pixel value calculation section 203 executes the processes of steps S303 to S304, the processing time is likely to vary depending on how living body 102 is imaged. Furthermore, for example, if measurement device 100 is a PC (Personal Computer), smartphone, or the like, control section 202 executes multiple processes in parallel using multiple different application programs, etc. Therefore, the processing time for the processes of steps S303 to S304 may vary for each processed image IMG.

[0035] If there is variation in the processing time for the processing of steps S303 to S304, as illustrated in Figure 5, the pixel value calculation unit 203 may not be able to acquire all of the captured images acquired by the imaging unit 101 as processed images IMG, and frame drops may occur in the processing performed by the pixel value calculation unit 203.

[0036] 6, even if the processing time for steps S303 to S304 is constant for each processed image IMG, the processing time for steps S302 to S305 may be longer than the exposure time in the imaging unit 101. In this case, the pixel value calculation unit 203 may generate a captured image that is not acquired as the processed image IMG, resulting in a dropped frame.

[0037] Therefore, for example, the time estimation unit 204 determines whether or not the minimum value of the difference between the expected fluctuation signal 801 and the first representative pixel value 212 calculated from the processed image IMG in the order in which the processed image IMG was acquired by the pixel value calculation unit 203 exceeds a threshold. If the minimum value of the difference between the expected fluctuation signal 801 and the first representative pixel value 212 does not exceed the threshold, the time estimation unit 204 determines that no frame dropping has occurred for the processed image IMG for which the first representative pixel value 212 was calculated. On the other hand, if the minimum value of the difference between the expected fluctuation signal 801 and the first representative pixel value 212 exceeds the threshold, the time estimation unit 204 determines that a frame dropping has occurred immediately before the processed image IMG for which the first representative pixel value 212 was calculated.

[0038] For example, when a 100 Hz flicker occurs and the lighting blinks regularly, and the frame rate Fr is 55 fps, if the first representative pixel value 212 fluctuates at an expected fluctuation period of 5.5 frames, in step S403 the time estimation unit 204 determines that no frame drops have occurred in the multiple processed images IMG from the multiple captured images acquired by the imaging unit 101.

[0039] On the other hand, when a 100 Hz flicker occurs and the illumination blinks regularly, and the frame rate Fr is 55 fps, if the first representative pixel value 212 does not fluctuate at the expected fluctuation period of 5.5 frames, in step S403 the time estimation unit 204 determines that frame dropping has occurred in the multiple processed images IMG from the multiple captured images acquired by the imaging unit 101. That is, the time estimation unit 204 calculates the expected fluctuation period based on the frame rate Fr and the frequency of the regular blinking of the illumination. Then, if the expected fluctuation signal having the calculated expected fluctuation period does not match the multiple first representative pixel values ​​212, it determines that frame dropping has occurred.

[0040] If it is determined in step S403 that no frame dropping has occurred, the time estimation unit 204 assigns a frame number to each of the multiple processed images IMG in the order in which they were acquired in step S404. Then, the control unit 202 proceeds to step S406.

[0041] On the other hand, if it is determined in step S403 that a frame drop has occurred, in step S405 the time estimation unit 204 assigns the frame number of the frame drop to the frame number of the captured image, which is each of the multiple processed images IMG, including the frame number for which the first representative pixel value 212 has not been calculated. Then, the control unit 202 proceeds to step S406.

[0042] In step S406, the time estimation unit 204 estimates the time 213 at which the multiple processed images IMG were acquired by the imaging unit 101, based on the frame number assigned to each of the multiple processed images IMG and the frame rate Fr.

[0043] Based on the period of the aliasing distortion, the time estimation unit 204 estimates the time 213 at which each of the multiple images captured by the imaging unit 101 was acquired. Therefore, when the imaging unit 101 acquires captured images at a frame rate Fr at which aliasing distortion can be observed, the time estimation unit 204 can estimate the time 213 at which the processed image IMG was acquired by the imaging unit 101 from the periodic fluctuation of the first representative pixel value 212.

[0044] For example, if the time measured by the internal clock is associated with the first frame of processed image IMG1, the time estimation unit 204 estimates, starting from the absolute time associated with the first frame of processed image IMG, the absolute time indicating the time when the nth frame of processed image IMG was acquired by the imaging unit 101, from the product of the exposure time determined from the frame rate Fr and the frame number n, where n is a natural number equal to or greater than 2.

[0045] Alternatively, the time estimation unit 204 may estimate a relative time indicating the elapsed time from the time when the first frame processed image IMG is acquired to the time when the nth frame processed image IMG is acquired by the imaging unit 101, from the product of the exposure time determined from the frame rate and the frame number n.

[0046] In step S407, the time estimation unit 204 associates the first representative pixel value 212 calculated from each processed image IMG with the time 213 estimated in step S406. Therefore, by performing the processes in steps S401 to S407, the measuring device 100 according to this embodiment can appropriately identify the date and time when each processed image was captured without missing any captured images.

[0047] In step S408, the biosignal calculation unit 205 calculates a biosignal from the multiple first representative pixel values ​​212 linked to the times 213. For example, the biosignal calculation unit 205 processes a signal indicating a time change in the first representative pixel values ​​212 by independent component analysis and pigment component separation, and calculates the processed result as a biosignal. The time change in the first representative pixel values ​​212 includes information on a change in the volume of blood vessels. For example, the biosignal indicates a pulse wave signal.

[0048] When the biological signal calculation unit 205 performs digital filtering, trend removal, or the like on the calculated biological signal to remove noise from the calculated biological signal, it is preferable to associate times at equal time intervals with values ​​indicated by a signal indicating a time change in the first representative pixel value 212. Therefore, the biological signal calculation unit 205 may interpolate values ​​of portions of the biological signal where frames have been dropped, based on times 213 associated with each of a plurality of processed images IMG. For example, the biological signal calculation unit 205 interpolates first representative pixel values ​​212 of portions of the biological signal where frames have been dropped, and processes the signal where the first representative pixel values ​​212 of portions of the biological signal have been interpolated, thereby interpolating values ​​of portions of the biological signal where frames have been dropped.

[0049] Alternatively, the biological signal calculation unit 205 may correct the signal indicating the time change of the first representative pixel value 212 to have values ​​at equal time intervals by upsampling or downsampling the value indicated by the signal indicating the time change of the first representative pixel value 212 based on the time 213 linked to each of the multiple processed images IMG. As described above, the biological signal calculation unit 205 can calculate the biological signal more accurately by interpolating the values ​​of the portion of the biological signal where frames have been dropped.

[0050] The biological signal output by the biological signal calculation unit 205 may be indicated by linking the value indicated by the biological signal with the time 213 estimated by the time estimation unit 204. In this case, the biological signal output by the biological signal calculation unit 205 may be in a state in which frame dropping has occurred at non-equidistant time intervals.

[0051] FIG. 5 is a diagram showing an example of a case where frame dropping occurs due to variations in processing time for the processes in steps S303 and S304 illustrated in FIG.

[0052] 3, the imaging unit 101 starts processing to capture moving images at a frame rate Fr. At each of time points t101 to t106, the imaging unit 101 captures images of the living body 102 and sequentially captures captured images I11 to I16. The intervals between each of time points t100 to t106 are exposure times determined by the frame rate Fr.

[0053] For example, the pixel value calculation unit 203 acquires the image I11 as the first frame of the processed image IMG. Then, the pixel value calculation unit 203 calculates the first representative pixel value 212 from the first frame of the processed image IMG, and then acquires the latest captured image I12 as the second frame of the processed image IMG.

[0054] Times T111 to T114 each indicate the processing time for each processed image IMG required for the processes in steps S303 to S304. For example, if time T111 is shorter than the exposure time determined by the frame rate Fr, the pixel value calculation unit 203 calculates the first representative pixel value 212 from the first frame processed image IMG, which is the captured image I11, and after the imaging unit 101 acquires the captured image I12, the pixel value calculation unit 203 acquires the second frame processed image IMG from the latest captured image I12 acquired at time t102.

[0055] Similarly, for example, if the time T112 is shorter than the exposure time determined by the frame rate Fr, the pixel value calculation unit 203 calculates the first representative pixel value 212 from the processed image IMG of the second frame, which is the captured image I12, and after the imaging unit 101 acquires the captured image I13, the pixel value calculation unit 203 acquires the latest captured image I13 acquired at time t103 as the processed image IMG of the third frame.

[0056] On the other hand, for example, if the time T113 is longer than the exposure time determined by the frame rate Fr, the imaging unit 101 acquires the captured image I14 and the captured image I15 before the pixel value calculation unit 203 calculates the first representative pixel value 212 from the processed image IMG of the third frame, which is the captured image I13 acquired at time t103. Therefore, the pixel value calculation unit 203 does not acquire the captured image I14 acquired at time t104 as the processed image IMG, but acquires the latest captured image I15 acquired at time t105 as the processed image IMG of the fourth frame.

[0057] Therefore, if there is variation in the processing time for the processing of steps S303 to S304, the pixel value calculation unit 203 may not be able to acquire all of the captured images acquired by the imaging unit 101 as processed images IMG, and frame dropping may occur in the processing performed by the pixel value calculation unit 203.

[0058] Fig. 6 is a diagram showing an example of a case where frame dropping occurs because the processing time for steps S303 to S304 shown in Fig. 3 is longer than the exposure time determined by the frame rate Fr. Time points t100 to t106 and captured images I11 to I16 are similar to time points t100 to t106 and captured images I11 to I16 shown in Fig. 5, and detailed description thereof will be omitted. Times T121 to T124 each indicate the processing time for each processed image IMG that is longer than the exposure time determined by the frame rate Fr for the processing of steps S303 to S304.

[0059] For example, assume that each of times T121 to T123 is longer than the exposure time determined by the frame rate Fr, and therefore the imaging unit 101 acquires captured images I14 and I15 during time T123. In this case, after calculating the first representative pixel value 212 from the processed image IMG of the third frame, which is captured image I13 acquired at time t103, the pixel value calculation unit 203 is unable to acquire captured image I14 acquired at time t104 as the processed image IMG, and instead acquires the latest captured image I15 acquired at time t105 as the processed image IMG of the fourth frame.

[0060] Therefore, if the processing time for steps S303 to S304 is longer than the exposure time, the pixel value calculation unit 203 may not be able to acquire all of the captured images acquired by the imaging unit 101 as processed images IMG, and frame dropping may occur in the processing performed by the pixel value calculation unit 203.

[0061] Fig. 7 is a diagram showing an example of time-series fluctuations in the first representative pixel value 212. In Fig. 7, the horizontal axis indicates the frame number assigned to the processed image IMG, and the vertical axis indicates the first representative pixel value 212.

[0062] For example, when a 100 Hz flicker occurs and the lighting blinks regularly, if the frame rate Fr is 55 fps, aliasing distortion of 10 Hz occurs. In this case, the first representative pixel value 212 fluctuates at an expected fluctuation period of 5.5 frames, as shown in Fig. 7. When the first representative pixel value 212 fluctuates at the expected fluctuation period, no frame drops occur in the processing executed by the pixel value calculation unit 203.

[0063] Fig. 8 is a diagram showing an example of the time-series fluctuation of the first representative pixel value 212 and an expected fluctuation signal 801. In Fig. 8, the horizontal axis indicates the frame number assigned to the processed image IMG, and the vertical axis indicates the first representative pixel value 212.

[0064] If a frame drop occurs in the processing executed by the pixel value calculation unit 203, the expected fluctuation signal 801 does not match the multiple first representative pixel values ​​212. For example, in an environment where 100 Hz flicker occurs and the lighting blinks regularly, if the frame rate Fr is 55 fps, the expected fluctuation signal 801 is a sine wave having an expected fluctuation period of 5.5 frames.

[0065] When a frame is dropped, the expected fluctuation signal 801 does not match the multiple first representative pixel values ​​212, as illustrated in Fig. 8. Specifically, when the pixel value calculation unit 203 does not acquire the captured image of the eighth frame as the processed image IMG and a frame is dropped for the captured image of the eighth frame, a difference occurs between the first representative pixel value 212 calculated from the processed image IMG of the eighth frame and the expected fluctuation signal 801, as illustrated in Fig. 8.

[0066] FIG. 9 is a diagram showing an example of a case where frame numbers of captured images are assigned to the processed image IMG for which the first representative pixel values ​​212 shown in FIG. 8 have been calculated.

[0067] For example, if frame dropping occurs in the captured image of the 8th frame and the captured image of the 16th frame, as illustrated in Fig. 9, the frame numbers of the 8th and 16th frames are assigned as frame numbers for which the first representative pixel value 212 is not calculated, and frame numbers are assigned to each of the multiple processed images IMG. Then, in step S407 illustrated in Fig. 4, the time estimation unit 204 estimates the time at which each of the multiple processed images IMG was acquired by the imaging unit 101, based on the frame number assigned to each of the multiple processed images IMG and the frame rate Fr. Then, the biological signal calculation unit 205 interpolates the value of the 8th frame and the value of the 16th frame, in which frame dropping occurred, from among the values ​​indicated by the biological signal, based on the time estimated by the time estimation unit 204. This allows the biological signal calculation unit 205 to calculate the biological signal more accurately.

[0068] As described above, the measurement device 100 of this embodiment can properly identify the date and time when an image was captured without missing any captured images, so even if a frame is dropped, the influence of the dropped frame can be suppressed and the biological signal can be calculated.

[0069] Second Embodiment The second embodiment will be described with reference to Figures 10 to 12. In the drawings, the same or similar elements are denoted by the same reference numerals, and redundant explanations will be omitted. Configurations and processes having substantially the same functions as those of other embodiments will be denoted by the same reference numerals, and explanations will be omitted, and differences from other embodiments will be described.

[0070] Fig. 10 is a block diagram showing an example of the configuration of a measurement device 100 according to this embodiment. The measurement device 100 shown in Fig. 10 differs from the measurement device 100 shown in Fig. 2 in that the pixel value calculation unit 203 of the measurement device 100 shown in Fig. 10 further calculates a second representative pixel value 1001 of the pixel values ​​of the pixels in the second region of interest.

[0071] The pixel value calculation unit 203 according to this embodiment calculates, from each of the plurality of processed images IMG, a first representative pixel value 212 of pixel values ​​of two or more pixels in a first region of interest that does not include an image of the body surface of the living body 102. In this way, the pixel value calculation unit 203 calculates a plurality of first representative pixel values ​​212 from each of the plurality of processed images IMG.

[0072] Furthermore, the pixel value calculation unit 203 according to this embodiment calculates, from each of the plurality of processed images IMG, a second representative pixel value 1001 of the pixel values ​​of two or more pixels in a second region of interest including an image of the body surface of the living body 102. In this way, the pixel value calculation unit 203 calculates a plurality of second representative pixel values ​​1001 from each of the plurality of processed images IMG.

[0073] The biological signal calculation section 205 according to this embodiment calculates a biological signal from a plurality of second representative pixel values ​​1001.

[0074] 11 and 12 are flowcharts showing an example of the operation of the measurement device 100 according to this embodiment. The processing in steps S1101 and S1102 shown in Fig. 11 is the same as that in steps S301 and S302 shown in Fig. 3, and therefore detailed description thereof will be omitted.

[0075] In step S1103, the pixel value calculation unit 203 determines a first region of interest that is included in the processed image IMG and does not include an image of the body surface of the living body 102, and a second region of interest that includes an image of the body surface of the living body 102. The first region of interest according to this embodiment indicates an image of a portion that does not move and does not change color within the angle of view captured by the imaging unit 101. For example, the first region of interest indicates an image of a wall, ceiling, etc. The second region of interest is similar to the first region of interest according to the first embodiment, and therefore a detailed description thereof will be omitted.

[0076] In an image of the body surface of the living body 102, pixel values ​​change periodically due to aliasing caused by the regular flickering of lighting, and pixel values ​​also change periodically due to changes in the volume of blood vessels in the living body 102. Therefore, in an image of a portion that does not move and does not change color, the periodic changes in pixel values ​​caused by aliasing caused by the regular flickering of lighting appear more clearly than in an image of the body surface of the living body 102.

[0077] In step S1104, the pixel value calculation unit 203 calculates the first representative pixel value 212 of the pixel values ​​of the pixels in the first region of interest determined in step S1103. The process of step S1104 is similar to the process of step S304 illustrated in FIG. 3, and therefore a detailed description thereof will be omitted.

[0078] In step S1105, pixel value calculation unit 203 calculates second representative pixel values ​​1001 of the pixel values ​​of the pixels in the second region of interest determined in step S1103. For example, second representative pixel value 1001 is the average value, median, or mode of the pixel values ​​of the pixels in the second region of interest. For example, if imaging unit 101 is configured with an image sensor for a camera including RGB (Red, Green, Blue) filters, pixel value calculation unit 203 may calculate second representative pixel values ​​1001 of the pixel values ​​of each of the R, G, and B pixels in the second region of interest.

[0079] In step S1106, the pixel value calculation unit 203 determines whether or not a predetermined number of frames of processed images IMG have been acquired. The process in step S1106 is the same as step S305 illustrated in FIG. 3, and therefore a detailed description thereof will be omitted.

[0080] If the predetermined number of frames or more of processed images IMG have not been acquired in step S1106, the control unit 202 returns the process to step S1102 and continues the process. On the other hand, if the predetermined number of frames or more of processed images IMG have been acquired in step S1106, the control unit 202 proceeds to step S1201 illustrated in FIG.

[0081] The operation of the measurement device 100 according to this embodiment will be further described with reference to Fig. 12. The processing in steps S1201 to S1202 illustrated in Fig. 12 is similar to the processing in steps S401 to S402 illustrated in Fig. 4, and therefore detailed description thereof will be omitted.

[0082] In step S1203, the time estimation unit 204 determines whether a frame has been dropped by comparing the expected fluctuation signal 801 calculated in step S1202 with the fluctuation in the time series of the first representative pixel value 212. That is, the time estimation unit 204 determines whether a frame has been dropped by comparing the expected fluctuation signal 801 with the fluctuation in the first representative pixel value 212 calculated from the first region of interest including an image of a portion that does not move and does not change color. Because the first region of interest does not include an image of the body surface of the living body 102 but includes an image of a portion that does not move and does not change color, pixel values ​​do not change periodically due to changes in the volume of blood vessels in the living body 102. Therefore, periodic changes in pixel values ​​due to aliasing caused by the influence of regular blinking of lighting are clearly apparent. This allows the time estimation unit 204 according to this embodiment to determine whether a frame has been dropped more accurately than the time estimation unit 204 in the first embodiment.

[0083] If no frame drop has occurred in step S1203, the control unit 202 proceeds to step S1204. On the other hand, if a frame drop has occurred in step S1203, the control unit 202 proceeds to step S1205. The processes in steps S1204 to S1206 are the same as the processes in steps S404 to S406 illustrated in FIG. 4, and therefore will not be described in detail.

[0084] The first region of interest in this embodiment includes images of parts that do not move or change color within the angle of view captured by the imaging unit 101, and does not include images of the body surface of the living body 102.As a result, the time estimation unit 204 in this embodiment can estimate the time 213 at which the processed image IMG was acquired by the imaging unit 101 more accurately than the time estimation unit 204 in the first embodiment, from the fluctuations in the time series of the first representative pixel value 212.

[0085] In step S1207, the time estimation unit 204 associates the second representative pixel value 1001 for the second region of interest with the time 213 at which the processed image IMG, from which the second representative pixel value 1001 was calculated, was acquired by the imaging unit 101. Then, the control unit 202 proceeds to step S1208. The processing of step S1208 is similar to step S408 illustrated in FIG. 4, and therefore a detailed description thereof will be omitted.

[0086] As described above, the measurement device 100 of this embodiment can calculate biological signals more accurately than the measurement device 100 of the first embodiment, because a more accurate time is linked to the second representative pixel value 1001 than in the measurement device 100 of the first embodiment.

[0087] (Variation) In a modified example of the measuring device 100 according to this embodiment, the first region of interest may include all pixels in each of the processed images IMG. That is, the pixel value calculation unit 203 according to this embodiment calculates, for each of the multiple processed images IMG, a first representative pixel value 212 of the pixel values ​​of all pixels in the processed image IMG. Therefore, the measuring device 100 according to this modified example calculates the first representative pixel value 212 without performing a process for identifying a region that does not include an image of the body surface of the living body 102. The measuring device 100 according to this modified example can reduce the processing time required for the process for identifying the first region of interest.

[0088] The processes executed in the above embodiments are not limited to the processing modes exemplified in the above embodiments. The above-described functional blocks may be realized using either a logic circuit (hardware) formed in an integrated circuit or the like, or software using a CPU. The processes executed in the above embodiments may be executed by multiple computers. For example, some of the processes executed by the control unit 202 may be executed by another computer, or all of the processes may be shared and executed by multiple computers.

[0089] The present disclosure is not limited to the above-described embodiments, and may be replaced with a configuration that is substantially the same as the configuration shown in the above-described embodiments, a configuration that achieves the same effect, or a configuration that can achieve the same purpose. The present disclosure also includes within its technical scope embodiments obtained by appropriately combining the technical means disclosed in different embodiments. Furthermore, new technical features can be formed by combining the technical means disclosed in each embodiment. [Explanation of symbols]

[0090] 100 Measuring device, 101 Imaging unit, 102 Living body, 201 Memory unit, 202 Control unit, 203 Pixel value calculation unit, 204 Time estimation unit, 205 Living body signal calculation unit, 212 First representative pixel value, 213 Time, 801 Estimated fluctuation signal, 1001 Second representative pixel value

Claims

1. an imaging unit that uses light from an illumination that generates regular blinking to image the living body at a frame rate that is set based on a frequency that is different from a frequency component that indicates a change in pixel value due to a change in the volume of blood vessels in the living body and that is different from the blinking frequency, thereby sequentially acquiring a plurality of captured images; a time estimation unit that estimates a time at which each of the plurality of captured images was acquired based on periodic fluctuations that appear in a time series of a plurality of pixel values ​​acquired from each of the plurality of captured images, which are at least a portion of the plurality of captured images; Equipped with Measuring device.

2. a pixel value calculation unit that calculates a first representative pixel value of pixel values ​​of two or more pixels within a first region of interest from each of the plurality of images, thereby calculating a plurality of first representative pixel values ​​from the plurality of images, respectively; Furthermore, The plurality of pixel values ​​are the plurality of first representative pixel values. The measuring device according to claim 1 .

3. The time estimation unit calculates a period in the periodic fluctuation based on the frame rate and the frequency of the blinking, and determines that a frame drop has occurred when a signal having the period does not match the plurality of first representative pixel values. The measuring device according to claim 2 .

4. The periodic fluctuation includes a frequency component due to the flickering. The measuring device according to claim 1 .

5. the first region of interest includes an image of a body surface of the living body; a biosignal calculation unit that calculates a biosignal from the plurality of first representative pixel values; Further equipped The measuring device according to claim 2 .

6. the first region of interest does not include an image of a body surface of the living body; the pixel value calculation unit calculates a second representative pixel value of pixel values ​​of two or more pixels within a second region of interest from each of the plurality of images, thereby calculating a plurality of second representative pixel values ​​from the plurality of images, a biosignal calculation unit that calculates a biosignal from the second representative pixel values; Further equipped The measuring device according to claim 2 .

7. The biological signal indicates a pulse wave signal.

7. The measuring device according to claim 5 or 6.

8. a step of capturing an image of the living body using light from an illumination that generates regular blinking, at a frame rate that is set based on a frequency that is different from a frequency component that indicates a change in pixel value due to a change in the volume of blood vessels in the living body and that is different from the frequency of the blinking, thereby sequentially acquiring a plurality of captured images; estimating the time at which each of the plurality of images was acquired based on periodic fluctuations appearing in a time series of a plurality of pixel values ​​acquired from each of a plurality of images that are at least a portion of the plurality of captured images; Contains Measurement method.

Images