Measuring device, measuring method and recording medium

Abstract

The measuring device includes: an imaging unit for capturing images of a living organism and acquiring dynamic images; a pixel value calculation unit for calculating representative values ​​of pixel values ​​in a region of interest based on each image constituting the dynamic image, wherein the region of interest includes an image of the living organism; and a pulse wave calculation unit for calculating pulse wave signals based on the time variation of the representative values. The imaging unit captures images of the living organism at a frame rate that can separate the frequency components caused by flicker and frame rate contained in the pulse wave signal from the frequency components caused by the time variation of the living organism contained in the pulse wave signal.

Description

Technical Field

[0001] This disclosure relates to measuring devices, measuring methods, and recording media. Background Technology

[0002] International patent application WO2016 / 163019 discloses the following technique: by analyzing the differential signal of image information from two or three regions close to each other along the blood flow, noise from the outside can be suppressed, and vital information such as pulse rate, pulse waveform, pulse wave propagation velocity, and blood pressure can be measured. Japanese Patent Application Publication No. 2019-042145 discloses the following technique: calculating illumination variation components and hemoglobin components based on dynamic images of two wavebands, and estimating heart rate variations based on the calculated hemoglobin components.

[0003] Japanese Patent Application Publication No. 2005-033616 discloses the following technology: when the vertical synchronization frequency fv (Hz) of the imaging device is set and N is a positive integer, when 120 / fv is an integer, an electronic shutter speed is set to an exposure time of N / 120 (seconds), and when 100 / fv is an integer, an electronic shutter speed is set to an exposure time of N / 100 (seconds). Summary of the Invention

[0004] The technical problem to be solved by the present invention

[0005] In the technologies disclosed in International Publication No. WO2016 / 163019 and Japanese Patent Application Publication No. 2019-042145, there is a concern that when the frequency and pulse rate of noise caused by flicker are of the same degree, it is impossible to separate the noise caused by flicker from the frequency component caused by the temporal variation of a living organism. Therefore, one aspect of the object of this disclosure is to provide a measuring device, a measuring method, and a recording medium capable of separating the noise caused by flicker from the frequency component caused by the temporal variation of a living organism.

[0006] One aspect of this disclosure relates to a measuring apparatus comprising: an imaging unit for capturing images of a living organism and acquiring dynamic images; a pixel value calculation unit for calculating representative values ​​of pixel values ​​of a region of interest based on each image constituting the dynamic image, wherein the region of interest includes an image of the living organism; and a pulse wave calculation unit for calculating a pulse wave signal based on the time variation of the representative values, wherein the imaging unit captures images of the living organism at a frame rate capable of separating frequency components caused by flicker and frame rate contained in the pulse wave signal from frequency components caused by time variation of the living organism contained in the pulse wave signal.

[0007] One aspect of this disclosure relates to a measurement method comprising: a step of photographing a living organism and acquiring a dynamic image; a step of calculating a representative value of pixel values ​​of a region of interest based on each image constituting the dynamic image, wherein the region of interest includes an image of the living organism; and a step of calculating a pulse signal based on the time variation of the representative value, wherein, in the step of acquiring the dynamic image, the living organism is photographed at a frame rate capable of separating frequency components caused by flicker and frame rate contained in the pulse signal from frequency components caused by time variation of the living organism contained in the pulse signal.

[0008] One aspect of this disclosure relates to a computer-readable recording medium containing a program that causes a computer to perform the following functions: capturing a living organism and acquiring a moving image; calculating representative values ​​of pixel values ​​of a region of interest based on each image constituting the moving image, wherein the region of interest contains an image of the living organism; and calculating a pulse signal based on the time variation of the representative values, wherein, in the function of acquiring the moving image, the living organism is captured at a frame rate capable of separating frequency components caused by flicker and frame rate contained in the pulse signal from frequency components caused by time variation of the living organism contained in the pulse signal. Attached Figure Description

[0009] Figure 1 This is a diagram illustrating an example of how a measuring device is used.

[0010] Figure 2 This is a block diagram illustrating an example of the configuration of the measuring device according to the first embodiment.

[0011] Figure 3 This is a diagram illustrating examples of information related to blinking.

[0012] Figure 4 This is a flowchart illustrating an example of the operation of the measuring device according to the first embodiment.

[0013] Figure 5 This is a block diagram illustrating an example of the configuration of the measuring device according to the second embodiment.

[0014] Figure 6 This is a diagram showing an example of a frame rate table.

[0015] Figure 7 This is a flowchart illustrating an example of the operation of the measuring device according to the second embodiment.

[0016] Figure 8 This is a graph illustrating an example of the relationship between the frequency of folding distortion and the frame rate when the flicker frequency is 120Hz.

[0017] Figure 9 It is magnification Figure 8 The example graph shows the frame rate range of 35fps to 55fps.

[0018] Figure 10 This is a block diagram illustrating an example of the configuration of the measuring device according to the third embodiment.

[0019] Figure 11 This is a flowchart illustrating an example of the operation of the measuring device according to the third embodiment.

[0020] Figure 12 It shows the next step. Figure 11 A flowchart illustrating an example of the operation of the measuring device according to the third embodiment.

[0021] Figure 13 This is a block diagram illustrating an example of the configuration of the measuring device involved in a variation of the third embodiment. Detailed Implementation

[0022] (First Implementation)

[0023] Reference Figures 1-4 The first embodiment will be described below. Furthermore, in the accompanying drawings, the same or equivalent elements will be given the same reference numerals, and repeated descriptions will be omitted.

[0024] Figure 1 This diagram illustrates an example of how the measuring device 100 is used. (As shown...) Figure 1 As illustrated, the measuring device 100 includes a camera unit 101.

[0025] The measuring device 100 calculates a pulse signal representing a pulse wave based on the image acquired by the imaging unit 101. For example, the measuring device 100 can be a PC (Personal Computer), a smartphone, a tablet terminal, a dedicated pulse wave estimation terminal, etc. In this specification, a pulse wave refers to a time-series signal representing a change in blood vessel volume, calculated from a time-series signal representing the pixel values ​​of pixels contained in an image, at the same location on the body surface. In this specification, a pixel value is information representing the brightness of pixels contained in an image, such as the pixel value or brightness value of each R (Red), G (Green), and B (Blue) pixel.

[0026] The imaging unit 101 may include, for example, a CCD (Charge Coupled Devices) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor. The imaging unit 101 may also be configured as a camera image sensor that includes an RGB (Red, Green, Blue) filter.

[0027] Figure 2 This is a block diagram illustrating an example of the configuration of the measuring device 100. The measuring device 100 includes an imaging unit 101, a storage unit 201, and a control unit 202.

[0028] The imaging unit 101 captures images of the living organism 102 opposite to the imaging unit 101, obtaining an image containing an image of the body surface of the living organism 102. For example, the image of the body surface is an image of the forehead, cheek, fingertips, wrist, palm, etc. of the living organism 102.

[0029] Specifically, the imaging unit 101 captures images of the living organism 102 at a predetermined time using a frame rate capable of separating the frequency components of flicker and frame rate contained in the pulse signal from the frequency components caused by the temporal changes of the living organism 102 contained in the pulse signal, thereby acquiring an image containing an image of the surface of the living organism 102. The frequency components caused by the temporal changes of the living organism 102 are frequency components representing the changes in pixel values ​​caused by changes in the volume of blood vessels. Flicker is a periodic change in the brightness of a light source. When flicker occurs, the pixel values ​​of the pixels contained in the image acquired by the imaging unit 101 change periodically. When a periodic signal is observed at a frequency different from the signal frequency, a signal called folding distortion is observed. Therefore, when flicker occurs and the frame rate is different from the frequency of flicker, folding distortion is observed in the signal representing the pixel values ​​of a time series.

[0030] Storage unit 201 is a recording medium capable of recording various data, programs, etc., and is composed of, for example, a hard disk, an SSD (Solid State Drive), a semiconductor memory, etc. Storage unit 201 stores blink-related information 211 (see reference) regarding the settings of the imaging unit 101. Figure 3 ).

[0031] The control unit 202 performs various processes according to the programs and data stored in the storage unit 201. The control unit 202 includes a processor such as a CPU (Central Processing Unit).

[0032] The control unit 202 includes a shooting control unit 203, a pixel value calculation unit 204, a pulse calculation unit 205, and a vital signs calculation unit 206.

[0033] The shooting control unit 203 determines the optimal frame rate 212 based on the presence or absence of flickering or the flickering frequency fq. Then, the shooting control unit 203 controls the settings of the shooting unit 101 to image the living being 102 at the optimal frame rate 212. Specifically, the shooting control unit 203 determines shooting control information 213 to capture the living being 102 at the optimal frame rate 212. The shooting control information 213 includes information indicating the exposure time. That is, controlling the settings of the shooting unit 101 includes adjusting the exposure time.

[0034] The pixel value calculation unit 204 calculates a representative value 215 of the pixel value of the region of interest of the image containing the living organism 102 based on each image constituting the dynamic image 214.

[0035] The pulse calculation unit 205 calculates the pulse signal 216 based on the time change of the representative value 215.

[0036] The vital signs calculation unit 206 calculates the vital signs of the subject from the pulse signal 216. For example, the vital signs of the subject include indicators related to blood pressure and indicators related to pulse rate.

[0037] Figure 3 This is a diagram illustrating an example of flicker-related information 211. The flicker-related information 211 represents the presence or absence of flicker and the flicker frequency fq. Figure 3 The example of flicker-related information 211 indicates that flickering exists. Furthermore, Figure 3 The illustrated flicker-related information 211 indicates that the flicker frequency fq is 120Hz. For example, the flicker frequency fq indicated by the flicker-related information 211 corresponds to the lighting value. For example, in western Japan using a 60Hz power supply, flicker at a frequency of 120Hz may occur, while in eastern Japan using a 50Hz power supply, flicker at a frequency of 100Hz may occur. It should be noted that when no flicker is indicated, the flicker-related information 211 may not indicate the flicker frequency fq.

[0038] Figure 4 This is a flowchart illustrating an example of the operation of the measuring device 100 according to this embodiment. In this example, when the measuring device 100 is started, the control unit 202 begins... Figure 4 The illustrated step S401 is processed. This begins in the control unit 202. Figure 4 During the illustrated step S401, the flashing-related information 211 is stored in the storage unit 201.

[0039] In step S401, the shooting control unit 203 determines whether flickering exists. Specifically, when flicker-related information 211 indicates that flickering exists, the shooting control unit 203 determines that flickering exists. On the other hand, when flicker-related information 211 indicates that there is no flickering, the shooting control unit 203 determines that there is no flickering.

[0040] If no flickering is determined in step S401, in step S402, the imaging control unit 203 determines the optimal frame rate 212. Specifically, the imaging control unit 203 determines imaging control information 213, causing the imaging unit 101 to image the living organism 102 at the specified frame rate. For example, the specified frame rate is 60 fps (Frames per second). Then, the control unit 202 transfers the processing to step S404.

[0041] On the other hand, when flickering is determined to exist in step S401, in step S403, the shooting control unit 203 determines the optimal frame rate 212 based on the flickering frequency fq. Specifically, when flickering is determined to exist in step S401, the shooting control unit 203 determines the optimal frame rate 212 based on the flickering frequency fq indicated by the flickering related information 211. More specifically, when flickering exists, the shooting control unit 203 determines the optimal frame rate 212 such that the absolute value of the difference between the flickering frequency fq and an integer multiple of the optimal frame rate 212 is equal to or greater than a threshold TH. For example, the threshold TH is a predetermined value. Alternatively, the threshold TH may also be a value corresponding to the category of the vital signs of the measured object. Then, the control unit 202 transfers the processing to step S404.

[0042] In step S404, the shooting control unit 203 controls the shooting unit 101 to acquire the dynamic image 214, so that the shooting unit 101 captures the living organism 102 within a specified time at the optimal frame rate 212 determined in step S402 or step S403. The shooting unit 101 outputs each image constituting the acquired dynamic image 214 to the pixel value calculation unit 204.

[0043] For example, the camera unit 101 controls itself to capture images of the living organism 102 at an optimal frame rate 212 for a predetermined time, and outputs each image constituting the motion picture 214 to the pixel value calculation unit 204. In this case, when flickering occurs, the number of images acquired within the predetermined time is obtained by dividing the number of flickering events within the predetermined time by an integer. Alternatively, when the camera unit 101 acquires each image constituting the motion picture 214, it may also output each image constituting the motion picture 214 to the pixel value calculation unit 204 at a time when the pixel value calculation unit 204 is able to accept image input.

[0044] For example, in the presence of flicker, folding distortion is observed in the timing signal representing the temporal changes in the pixel values ​​of the pixels contained in the motion picture 214. Folding distortion is a frequency component caused by flicker and frame rate contained in the timing signal, which has a frequency that is the absolute value of the difference between the flicker frequency fq and an integer multiple of the frame rate. That is, the frequency fa of folding distortion is calculated by equation (1). N is a positive integer. The frame rate fr is the actual frame rate at which the imaging unit 101 acquires each image constituting the motion picture 214.

[0045] [Number 1]

[0046]

[0047] For example, when the flicker frequency fq is 120Hz, and the imaging unit 101 captures the living organism 102 with an exposure time of 1 / 60th of a second, no folding distortion is observed in the timing signal representing the time-varying pixel values ​​of the pixels included in the moving image 214. Therefore, when the flicker frequency fq is 120Hz, it is desirable to capture the living organism 102 with an exposure time of 1 / 60th of a second to suppress the effect of flicker. However, in reality, the imaging unit 101 may not be able to stably capture the living organism 102 with an exposure time of 1 / 60th of a second. For example, if the processing time in the pixel value calculation unit 204 is longer than the inter-frame time based on the optimal frame rate 212, the actual frame rate fr is lower than the optimal frame rate 212. Therefore, if the imaging control unit 203 determines the optimal frame rate 212 to be 60fps, the exposure time may be shorter than 1 / 60th of a second in order to acquire each image constituting the moving image 214 at a frame rate of 60fps.

[0048] Furthermore, since the exposure time can be set in decimals in the shooting unit 101, 1 / 60 second is a repeating decimal, so the shooting unit 101 cannot accurately set the exposure time to 1 / 60 second.

[0049] Alternatively, due to the processing time of the shooting unit 101 and the inter-frame processing time constituting the moving image 214, the frame rate fr sometimes becomes unstable and deviates randomly. In this case, even if the shooting control unit 203 determines the optimal frame rate 212 to be 60fps, the shooting unit 101 may sometimes stabilize at a frame rate different from 60fps, which is the optimal frame rate 212.

[0050] For example, suppose the imaging unit 101 acquires each image that makes up the motion picture 214, where the flicker frequency fq is 120 Hz and the imaging unit 101 is at a frame rate fr of 59.9 fps. In this case, a folding distortion with a frequency fa of 0.2 Hz (=|120-2×59.9|) is observed. That is, the pixel values ​​of each image that makes up the motion picture 214 change in a 5-second period.

[0051] Furthermore, for example, suppose the imaging unit 101 acquires each image that makes up the dynamic image 214, where the flicker frequency fq is 120 Hz and the imaging unit 101 is at a frame rate fr of 59.5 fps. In this case, folding distortion with a frequency fa of 1 Hz (=|120-2×59.5|) is observed.

[0052] For example, the frequency component fa based on folding distortion, which is 1 Hz, is the same as the frequency component in the pulse signal 216 calculated in step S407 described later, when the pulse rate is 60 beats per minute. The resting pulse frequency of a healthy adult is 0.7 Hz to 1.5 Hz. Therefore, if the pulse signal 216 contains 1 Hz of folding distortion, the vital signs calculation unit 206 cannot properly calculate the pulse rate from the pulse signal 216 in step S409 described later.

[0053] Therefore, even if the actual frame rate fr of each image constituting the dynamic image 214 is lower than the optimal frame rate 212, the shooting control unit 203 determines the optimal frame rate 212 to be such that the frequency components caused by the flicker and frame rate contained in the pulse signal 216 calculated in step S407 described later, as well as the frequency components caused by the time change of the living body 102 contained in the pulse signal 216, are included.

[0054] In step S405, the pixel value calculation unit 204 determines the region of interest for each image constituting the motion image 214. The region of interest is a region in each image that is part of a body surface area and contains multiple pixels. For example, if the motion image 214 includes the face region of the living being 102, the region of interest may include the cheek, forehead, or area between the eyebrows. The number of regions of interest can be one or more. The shape of the region of interest can be a polygon surrounded by straight lines or a shape surrounded by curves. Alternatively, the region of interest may be a closed region composed of straight lines and curves.

[0055] In step S406, the pixel value calculation unit 204 calculates a representative value 215 for the pixel values ​​of the region of interest determined in step S405 for each image constituting the dynamic image 214. For example, the representative value 215 is the average, median, or mode of the pixel values ​​of each pixel in the region of interest. Alternatively, if the capturing unit 101 is configured with an image sensor for a camera that includes an RGB filter, the pixel value calculation unit 204 can also calculate representative values ​​for R, G, and B pixels respectively.

[0056] In step S407, the pulse wave calculation unit 205 calculates the pulse wave signal 216 based on the time change of the representative value 215. Specifically, the pulse wave calculation unit 205 calculates the pulse wave signal 216 based on the time change of the representative value 215 corresponding to the same location in the body surface region. For example, the pulse wave calculation unit 205 processes the signal representing the time change of the representative value 215 through multivariate analysis such as principal component analysis and independent component analysis, and calculates the pulse wave signal 216 as the processed result. The time change of the representative value includes information on changes in blood vessel volume.

[0057] In step S408, the vital signs calculation unit 206 extracts the frequency components of a predetermined frequency band from the pulse signal 216. For example, the predetermined frequency band is the frequency band corresponding to the category of the vital signs of the measured object. Specifically, the vital signs calculation unit 206 extracts the frequency band signal corresponding to the category of the vital signs of the measured object from the pulse signal 216 by inputting the pulse signal 216 into a bandpass filter of the frequency band corresponding to the category of the vital signs of the measured object.

[0058] Here, the shooting control unit 203 determines the optimal frame rate 212 such that the absolute value of the difference between the flicker frequency fq and an integer multiple of the frame rate is above the threshold TH, thereby enabling the separation of the frequency components of flicker and frame rate contained in the pulse signal 216 from the frequency components caused by the time change of the life form 102 contained in the pulse signal 216.

[0059] The pulse signal 216 is not limited to folding distortion, but also contains noise at higher frequencies than the frequency components of the time variation caused by the living organism 102 contained in the pulse signal 216. Therefore, the vital sign calculation unit 206 can extract the frequency components of the frequency band corresponding to the category of the living organism index of the measured object from the pulse signal 216 by removing frequency components higher than the frequency band corresponding to the category of the living organism index of the measured object.

[0060] For example, when the flickering frequency fq is 120Hz and the imaging unit 101 acquires each image constituting the dynamic image 214 at an actual frame rate fr of 55fps in step S404, a folding distortion with a frequency fa of 10Hz (=|120-2×55|) is observed.

[0061] For example, if the vital sign of the measured object is the pulse rate, the frequency band corresponding to the pulse rate as a vital sign is 0.7 Hz or higher and 1.5 Hz or lower. Then, folding distortion at a frequency fa of 10 Hz is observed. In this case, for example, the vital sign calculation unit 206 inputs a bandpass filter with a frequency band of 0.7 Hz or higher and 1.5 Hz or lower to the pulse signal 216. As a result, the vital sign calculation unit 206 can remove the frequency component with a frequency of 10 Hz from the pulse signal 216.

[0062] Furthermore, for example, when the flicker frequency fq is 100Hz and the imaging unit 101 acquires each image constituting the motion image 214 at an actual frame rate fr of 52fps, a folding distortion with a frequency fa of 4Hz (=|100-2×52|) is observed.

[0063] Furthermore, for example, in step S403, the shooting control unit 203 determines the optimal frame rate 212 to be 60fps. In this case, with a flicker frequency fq of 120Hz, the shooting unit 101 acquires each image constituting the dynamic image 214 at an actual frame rate fr of 62fps. In this case, folding distortion with a frequency fa of 4Hz (=|120-2×62|) is observed.

[0064] Therefore, for example, the vital signs calculation unit 206 inputs a low-pass filter with a cutoff frequency of 4Hz to the pulse signal 216. As a result, the vital signs calculation unit 206 is able to remove frequency components with frequencies higher than 4Hz from the pulse signal 216.

[0065] Therefore, in step S403, the shooting control unit 203 determines the optimal frame rate 212 such that the absolute value of the difference between the flicker frequency and an integer multiple of the frame rate is above the threshold TH, thereby enabling the separation of the frequency components caused by flicker and frame rate from the frequency components caused by the time change of the life form 102 contained in the pulse signal 216.

[0066] In step S409, the vital signs calculation unit 206 calculates the vital signs of the subject based on the signal extracted in step S408. For example, if the category of the vital signs of the subject is blood pressure, the vital signs calculation unit 206 calculates the highest blood pressure based on the angle of ascent of the signal extracted in step S408. Alternatively, if the category of the vital signs of the subject is pulse rate, the vital signs calculation unit 206 calculates the pulse rate based on the number of peak values ​​of the signal extracted in step S408.

[0067] Based on the above, the measuring device 100 of this embodiment determines an optimal frame rate 212 in order to separate the frequency components caused by flicker and frame rate in the pulse signal 216, as well as the frequency components caused by the time variation of the living organism 102 in the pulse signal 216. The measuring device 100 of this embodiment can suppress the influence of noise caused by flicker and calculate vital signs such as blood pressure based on the dynamic image 214 obtained by capturing the living organism 102.

[0068] Furthermore, for example, Japanese Patent Application Publication No. 2005-033616 discloses that when the imaging unit 101 acquires each image constituting the dynamic image 214 at a frame rate of 60fps, by acquiring each image constituting the dynamic image 214 with an exposure time of 1 / 120 second, the effect of flicker with a frequency fq of 120Hz can be suppressed. However, the measuring device 100 relating to this embodiment may also involve the imaging unit 101 capturing the living body 102 with an exposure time longer than 1 / 120 second and acquiring the dynamic image at a frame rate fr of 60fps. In other words, even when the imaging unit 101 captures the living body 102 with an exposure time longer than 1 / 120 second and acquires the dynamic image at a frame rate of 60fps, the measuring device 100 according to this embodiment can still separate the frequency components contained in the pulse signal 216 caused by flicker and frame rate and the frequency components caused by the time variation of the living body 102 contained in the pulse signal 216. Therefore, compared with the case of photographing a living organism 102 using the technology disclosed in Japanese Patent Application Publication No. 2005-033616, the measuring device 100 of this embodiment can properly calculate the vital signs of the measured object even when photographing the living organism 102 in a dark environment.

[0069] (A variation of the first embodiment)

[0070] As a variation of the measuring device 100 according to this embodiment, the imaging control unit 203 can determine the value associated with the category of the vital signs of the measured object as the optimal frame rate 212 even without flickering. Therefore, the imaging unit 101 can capture images of the living organism 102 at a frame rate corresponding to the category of the vital signs of the measured object.

[0071] (Second Implementation)

[0072] Reference Figures 5-9 The second embodiment will now be described. Furthermore, for the accompanying drawings, the same or equivalent elements will be assigned the same reference numerals, and redundant descriptions will be omitted. Descriptions will be omitted for clarity regarding configurations and processes that have substantially the same function as other embodiments, using common reference numerals, and points differing from other embodiments will be explained.

[0073] Figure 5 This is a block diagram illustrating an example of the configuration of the measuring device 100 according to this embodiment. Figure 2 The illustrated measuring device 100 and Figure 5 The difference of the illustrated measuring device 100 is that: Figure 5 The illustrated measuring device 100 includes a flicker detection unit 501 and stores a frame rate table 511 in the storage unit 201.

[0074] The flicker detection unit 501 detects at least one of the following: the presence or absence of flicker and the flicker frequency.

[0075] Figure 6 This is a diagram illustrating an example of frame rate table 511. Registered frame rates are associated with frame rate table 511. Specifically, in frame rate table 511, frame rates whose absolute value of the difference between a registered frequency and an integer multiple of a registered frame rate is above a threshold are registered frame rates.

[0076] For example, with a registration frequency of 120Hz, the registration frame rate is 50fps. Furthermore, for example, with a registration frequency of 100Hz, the registration frame rate is 60fps. Additionally, Figure 6 The registration frequency and registration frame rate illustrated are merely examples and are not intended to limit the registration frequency and registration frame rate to specific values. Figure 6 The values ​​shown are as follows.

[0077] For example, the registered frequency and registered frame rate can be associated with each other in the frame rate table 511 according to the manufacturer or model of the measuring device 100. Alternatively, the user can input the registered frequency and registered frame rate using the operation unit (not shown) of the measuring device 100, and register the registered frequency and registered frame rate associated with each other in the frame rate table 511.

[0078] Figure 7 This is a flowchart illustrating an example of the operation of the measuring device 100 according to this embodiment. In this example, when the measuring device 100 is started, the control unit 202 begins... Figure 7 The illustrated step S701 is processed. It begins in control unit 202. Figure 7 During the illustrated step S701, the frame rate table 511 is stored in the storage unit 201.

[0079] In step S701, the imaging control unit 203 activates the imaging unit 101. When activated, the imaging unit 101 begins processing to capture images within the imaging range. The images acquired in step S701 are images acquired to detect the presence or absence of flickering. When the imaging unit 101 captures images to detect the presence or absence of flickering, the living organism 102 may not be present within the imaging range.

[0080] In step S702, the flicker detection unit 501 determines whether flicker is detected. For example, the flicker detection unit 501 calculates a brightness value based on the pixel values ​​of pixels in a predetermined region contained in an image captured to detect the presence or absence of flicker. The predetermined region is a region different from the body surface. In this case, the flicker detection unit 501 detects the peak time of the time change of the brightness value based on a timing signal representing the time change of the calculated brightness value. When the peak time of the time change of the brightness value is periodic, the flicker detection unit 501 determines that flicker is detected. For example, when the peak time of the time change of the brightness value is a preset period caused by flicker, the flicker detection unit 501 determines that flicker is detected. On the other hand, when the peak time of the time change of the brightness value is not periodic, the flicker detection unit 501 determines that flicker is not detected. In addition, the flicker detection unit 501 can capture the imaging range at multiple frame rates and determine that flicker is detected when the periods of the peak times of the time change of the brightness value are different from each other.

[0081] If, in step S702, it is determined that there is no flickering, in step S703, the shooting control unit 203 determines the specified frame rate as the optimal frame rate 212. The processing in step S703 is similar to... Figure 4 The process of step S402 illustrated is the same, so a detailed description is omitted. Then, the control unit 202 transfers the process to... Figure 4 Example step S404.

[0082] On the other hand, when flickering is detected in step S702, in step S704, the flicker detection unit 501 detects the flickering frequency fq. For example, if the flicker detection unit 501 detects a peak moment in the change of brightness value over time, it detects the flickering frequency fq from the time interval of the detected moment. Alternatively, the flicker detection unit 501 can calculate a spectrum related to a timing signal representing the time change of brightness value. Furthermore, the flicker detection unit 501 can also detect a peak frequency within a predetermined frequency band in the calculated spectrum as the flickering frequency fq.

[0083] In step S705, the shooting control unit 203 selects a registration frequency in the frame rate table 511. The registration frequency is the value of the flicker frequency fq that is closest to that detected in step S704.

[0084] In step S706, the shooting control unit 203 determines the optimal frame rate 212 in the frame rate table 511, which is associated with the selected registration frequency. Then, the control unit 202 transfers the processing to... Figure 4 Example step S404.

[0085] In this manner, the shooting control unit 203 according to this embodiment determines the frame rate associated with the flicker frequency fq as the optimal frame rate. Therefore, the shooting control unit 203 according to this embodiment may be able to determine a frame rate higher than the optimal frame rate 212 determined by the shooting control unit 203 according to the first embodiment as the optimal frame rate 212. For example, assuming that the memory is stored in the storage unit 201... Figure 5 Example frame rate table 511. In this case, for example, when the registration frequency selected in step S704 is 100Hz, the shooting control unit 203 determines the optimal frame rate 212 to be 60Hz.

[0086] Figure 8 This is a graph illustrating an example of the relationship between the frequency of folding distortion and the frame rate at a flicker frequency of 120Hz. Figure 8 In the graph, the horizontal axis represents the frame rate, and the vertical axis represents the frequency of folding distortion. Curves 801 to 804 show the relationship between the frequency of folding distortion (|120-f|, |120-2f|, |120-3f|, and |120-4f|) and the frame rate f, respectively.

[0087] Figure 9 It is magnification Figure 8 The example graph shows the frame rate range of 35fps to 55fps.

[0088] For example, when the flashing frequency fq is 120Hz, the imaging unit 101 captures images of the living organism 102. For example, refer to... Figure 9 At an actual frame rate of 45fps, the frequency of folding distortion above 15Hz was observed (fa). Furthermore, for example, referring to... Figure 9 When the actual frame rate fr is 50fps, a folding distortion frequency fa above 20Hz is observed. Therefore, for example, by setting the optimal frame rate 212 to 50fps, the shooting control unit 203 can extract the frequency component with a frequency below 15Hz from the pulse signal 216 even when the actual frame rate fr is reduced to 45fps.

[0089] As described above, the measuring device 100 of this embodiment detects the presence or absence of flicker and controls the setting of the imaging unit 101 to image the living organism 102 at a pre-registered frame rate according to the flicker frequency fq. Therefore, the measuring device 100 of this embodiment can suppress the influence of noise caused by flicker and can image the living organism 102 at the highest possible frame rate. As a result, the measuring device 100 of this embodiment can suppress the influence of noise caused by flicker and can acquire a pulse signal 216 with a higher temporal resolution than the pulse signal 216 calculated by the measuring device 100 of the first embodiment.

[0090] (Third Implementation) Refer to Figures 10-12 The third embodiment will now be described. Furthermore, for the accompanying drawings, the same or equivalent elements will be assigned the same reference numerals, and redundant descriptions will be omitted. Descriptions of configurations and processes having substantially the same functions as other embodiments will be omitted, using common reference numerals, and points differing from other embodiments will be explained.

[0091] Figure 10 This is a block diagram illustrating an example of the configuration of the measuring device 100 according to this embodiment. Figure 6 The illustrated measuring device 100 and Figure 10 The difference of the illustrated measuring device 100 is that: Figure 10 The illustrated measuring device 100 includes a shooting interval calculation unit 1001 and a shooting control unit 1002 instead of a shooting control unit 203.

[0092] The shooting interval calculation unit 1001 calculates the actual frame rate fr based on the time difference between each image that constitutes the dynamic image 214.

[0093] The shooting control unit 1002 controls the settings of the shooting unit 101 based on the difference between the actual frame rate fr and the optimal frame rate 212. That is, the shooting control unit 1002 determines the shooting control information 1011 based on the difference between the actual frame rate fr and the optimal frame rate 212. Specifically, the shooting control unit 1002 controls the settings of the shooting unit 101 so that the difference between the actual frame rate fr and the optimal frame rate 212 falls within a predetermined range.

[0094] Figure 11 This is a flowchart illustrating an example of the operation of the measuring device 100 in this embodiment. Figure 11 The processing of steps S1101 to S1106 shown in the example is... Figure 7 The processes in steps S701 to S706 shown in the example are the same, so detailed explanations are omitted.

[0095] In step S1107, the shooting control unit 1002 adjusts the exposure time ET1 based on the optimal frame rate 212 determined in step S1103 or step S1106, and determines the settings of the shooting unit 101. That is, the shooting control unit 1002 determines the shooting control information 1011, which includes information about the exposure time ET1. The shooting control information 1011 includes information indicating the exposure time.

[0096] In step S1108, the imaging unit 101 captures an image of the living organism 102 by setting the imaging unit 101 as determined in step S1107 or step S1204 (described later). That is, the imaging control unit 1002 controls the imaging unit 101 to capture an image at the exposure time ET1 shown in the imaging control information 1011.

[0097] In step S1109, the pixel value calculation unit 204 calculates a representative value 215 of the pixel value of the region of interest for the image acquired in step S1108. For example, the processing in step S1109 is similar to... Figure 4 The processing of step S406 is the same as illustrated. Alternatively, the pixel value calculation unit 204 may normalize the representative value 215 based on the exposure time. For example, the pixel value calculation unit 204 may normalize the representative value 215 to the value when the exposure time is 1 second. For example, if the representative value 215 is 200, and the exposure time for obtaining the image with the calculated representative value 215 is 10 ms, then the pixel value calculation unit 204 may normalize the representative value 215 to 20000 [ / second] (=200 ÷ 0.010 seconds).

[0098] In step S1110, the shooting control unit 1002 obtains the time when the image was acquired in step S1108. For example, when the shooting control unit 1002 obtains the time when the image was acquired, the shooting control unit 1002 stores the frame number assigned to the image and the time information associated with the time when the image was acquired in the storage unit 201.

[0099] In step S1111, the shooting control unit 1002 determines whether a predetermined number of frames of images have been acquired, based on the settings of the shooting unit 101 determined in step S1107 or step S1204 described later. For example, the predetermined number of frames is two consecutive frames.

[0100] If the predetermined number of frames are not acquired in step S1111, the control unit 202 returns the process to step S1108. That is, the control unit 202 repeats the processes of steps S1108 to S1111 based on the determined settings of the imaging unit 101 until the predetermined number of frames are acquired. Conversely, if the predetermined number of frames are acquired in step S1111, the control unit 202 transfers the process to... Figure 12 The illustrated step S1201.

[0101] Next, refer to Figure 12 The operation of the measuring device 100 in this embodiment will be further explained.

[0102] In step S1201, the shooting interval calculation unit 1001 calculates the image based on a predetermined number of frames. Figure 11The actual frame rate fr is calculated based on the time difference obtained in step S1110. For example, the shooting interval calculation unit 1001 calculates the time difference of images that have been captured by a predetermined number of frames by obtaining time information from the storage unit 201.

[0103] For example, when the predetermined number of frames is two frames, the shooting interval calculation unit 1001 calculates the absolute value of the difference between the time t1 and the time t0 at which the images of the two consecutive frames are acquired as the time difference. Time t0 is the time when the image of the frame is acquired at the earlier time in the two consecutive frames. Time t1 is the time when the image of the frame is acquired after time t0 in the two consecutive frames. Furthermore, the shooting interval calculation unit 1001 calculates the reciprocal of the calculated time difference as the actual frame rate fr. That is, the shooting interval calculation unit 1001 calculates the actual frame rate fr using the formula fr = 1 / (t1-t0).

[0104] Alternatively, for example, if the specified number of frames is 3 or more, the actual frame rate fr can be calculated based on the time difference between two consecutive frames obtained from the 3 or more frames.

[0105] For example, when the specified number of frames is N frames, the times at which the images of each frame are acquired are time t. i N represents an integer greater than or equal to 3, and i represents an integer greater than or equal to 0 and less than N-1. In this case, the frame rates of two consecutive frames in N frames are 1 / (t). k -t k-1 k is an integer greater than or equal to 1 and less than or equal to N-1. The shooting interval calculation unit 1001 calculates a representative value of the frame rate associated with two consecutive frames in N frames, as the actual frame rate fr. For example, the shooting interval calculation unit 1001 calculates the frame rate fr by using fr=(Σ N-1 k=1 (1 / (t k -t k-1 The actual frame rate fr is calculated using the formula fr=((1 / (t3-t2)+1 / (t2-t1)+1 / (t1-t0)) / 3. For example, when N=4, the shooting interval calculation unit 1001 calculates the actual frame rate fr using the formula fr=((1 / (t3-t2)+1 / (t2-t1)+1 / (t1-t0)) / 3.

[0106] Alternatively, the shooting interval calculation unit 1001 can also use fr=(N-1) / (t) N-1 The actual frame rate fr is calculated using the formula -t0). For example, when the specified number of frames is 30 frames, the shooting interval calculation unit 1001 calculates the actual frame rate fr using fr=29 / (t0). 29 The actual frame rate fr is calculated using the formula -t0).

[0107] If the actual frame rate fr is lower than the optimal frame rate 212 in step S1202, then in step S1203, the shooting control unit 1002 determines that the actual frame rate fr is lower than the optimal frame rate 212. Figure 11 The example step S1108 uses a new exposure time ET2 that is shorter than the exposure time ET1 when the image is acquired. By making the new exposure time ET2 shorter than the exposure time ET1, the shooting control unit 1002 makes the subsequent actual frame rate fr2 higher than the actual frame rate fr1 when the image is acquired in step S1107.

[0108] Furthermore, when the new exposure time ET2 is shorter than the exposure time ET1, the pixel value PIX2 of the image 12 captured at the new exposure time ET2 is smaller than the pixel value PIX1 of the image 11 captured at the exposure time ET1 for the same area of ​​the living organism 102. As a result, the signal-to-noise ratio of the pixel value PIX2 may be lower than that of the pixel value PIX1.

[0109] Therefore, the shooting control unit 1002 can also determine a new exposure time ET2, so that the new exposure time ET2 is at or above the preset lower limit exposure time ETMIN. Furthermore, the shooting control unit 1002 can also set the lower limit exposure time ETMIN based on the brightness of the imaging environment where the measuring device 100 is located. For example, the shooting control unit 1002 sets the lower limit exposure time ETMIN in a relatively bright environment to be shorter than the lower limit exposure time ETMIN in a relatively dark environment. Additionally, the shooting control unit 1002 can also set the lower limit exposure time ETMIN such that the representative value 215 is at or above the preset lower limit pixel value PIXMIN.

[0110] In step S1204, the shooting control unit 1002 determines the settings of the shooting unit 101, which include the new exposure time ET2 determined in step S1203. Specifically, the shooting control unit 1002 determines shooting control information 1011 including the new exposure time ET2. Then, the control unit 202 transfers the processing to... Figure 11 The example step S1108.

[0111] On the other hand, if the actual frame rate fr is higher than or equal to the optimal frame rate 212 in step S1202, in step S1205, the shooting control unit 1002 determines whether the difference between the actual frame rate fr and the optimal frame rate 212 is within a specified range. In step S1205, if the difference between the actual frame rate fr and the optimal frame rate 212 is not within the specified range, the control unit 202 returns the processing to normal. Figure 11The example step S1108 is shown. On the other hand, if the difference between the actual frame rate fr and the optimal frame rate 212 is within a specified range in step S1205, in step S1206, the imaging unit 101, based on the settings of the imaging unit 101 determined in step S1107 or step S1204, captures the living organism 102 within a specified time to obtain a dynamic image 214. Then, the control unit 202 transfers the processing to... Figure 4 Example step S405.

[0112] Furthermore, depending on the processing time of the imaging unit 101 and the processing time of the pixel value calculation unit 204, the difference between the actual frame rate fr and the optimal frame rate 212 may sometimes be outside the specified range. Therefore, when the imaging unit 101 is photographing the living organism 102 after adjusting the exposure time more than a specified number of times, the imaging control unit 1002 may change the optimal frame rate 212 if the difference between the actual frame rate fr and the optimal frame rate 212 is outside the specified range. For example, when the flicker frequency fq is 120Hz and the imaging unit of the imaging control unit 1002 determines the optimal frame rate 212 to be 60fps, the imaging control unit 1002 may set the optimal frame rate 212 to 40fps if the difference between the actual frame rate fr and the optimal frame rate 212 is outside the predetermined range.

[0113] Similarly, the shooting control unit 1002 can also set the exposure time as the lower limit exposure time ETMIN. When the shooting unit 101 is shooting the living organism 102, if the difference between the actual frame rate fr and the optimal frame rate 212 is not within the specified range, the optimal frame rate 212 is changed.

[0114] Based on the above, the measuring device 100 of this embodiment adjusts the exposure time according to the actual frame rate fr in a manner close to the optimal frame rate 212. Therefore, since the measuring device 100 of this embodiment requires processing time in the imaging unit 101 and processing time in the pixel value calculation unit 204, it can separate the noise caused by flickering contained in the pulse signal 216 from the frequency components caused by the time variation of the living organism contained in the pulse signal 216, even when the actual frame rate fr is lower than the optimal frame rate 212.

[0115] (Modification 1 of the third embodiment)

[0116] As a variation 1 of the measuring device 100 according to this embodiment, the shooting control unit 1002 can determine the optimal frame rate 212 based on the flashing frequency fq using a predetermined calculation formula. For example, the shooting control unit 1002 can determine the optimal frame rate 212 as the value obtained by dividing the flashing frequency fq by an integer N. For example, when the flashing frequency fq is 120Hz, the shooting control unit 1002 can determine 60fps (=120 / 2) as the optimal frame rate 212. Furthermore, for example, when the flashing frequency fq is 100Hz, the shooting control unit 1002 can determine 50fps (=100 / 2) as the optimal frame rate 212.

[0117] (Modification 2 of the third embodiment)

[0118] As a variation 2 of the measuring device 100 according to this embodiment, the shooting control unit 1002, when capable of setting an upper limit value for the frame rate, can also determine the upper limit value of the frame rate as the optimal frame rate 212 determined in step S1103 or step S1106. Therefore, the shooting control unit 1002 can prevent the actual frame rate fr from exceeding the optimal frame rate 212. Consequently, the shooting control unit 1002 can easily set the difference between the actual frame rate fr and the optimal frame rate 212 within a specified range.

[0119] In addition, if the measuring device 100 cannot set the upper limit of the frame rate and the actual frame rate fr exceeds the optimal frame rate 212, the shooting control unit 1002 can also adjust the actual frame rate fr to be lower by adjusting the new exposure time ET2 to be longer than the exposure time ET1.

[0120] Furthermore, a longer exposure time results in a larger pixel count and a higher signal-to-noise ratio. Therefore, by making the exposure time relatively longer, the signal-to-noise ratio is relatively higher for the frequency components related to the temporal changes of the organism 102 contained in the pulse signal 216. Here, even with a relatively long exposure time, the actual frame rate fr remains unchanged even if the difference between the actual frame rate fr and the optimal frame rate 212 is within a predetermined range. Therefore, the shooting control unit 1002 can adjust the exposure time to make the exposure time relatively longer when the difference between the actual frame rate fr and the optimal frame rate 212 is within a specified range.

[0121] However, there is a concern that if the exposure time is relatively long, the pixel values ​​may become too large and saturated. Therefore, the shooting control unit 1002 can also preset an upper limit value, i.e., an upper limit representative value, for the representative value 215. In this case, the shooting control unit 1002 can also adjust the exposure time to be relatively long if the difference between the actual frame rate fr and the optimal frame rate 212 is within a specified range and the representative value 215 is less than the upper limit representative value.

[0122] (Modification 3 of the third embodiment)

[0123] As a variation 3 of the measuring device 100 according to this embodiment, the measuring device 100 can perform in parallel the processing of the pulse calculation unit 205 calculating the pulse signal 216 and the processing of the imaging unit 101 capturing the living organism 102 to obtain a dynamic image 214. That is, the control unit 202 can also perform in parallel. Figure 4 The illustrated steps S405 to S407 processing and Figure 11 Example step S1108~ Figure 12 The processing in S1206 is illustrated. Therefore, the measuring device 100 according to this modification can readjust the exposure time after adjusting the exposure time, even if the frequency components caused by flicker and frame rate in the pulse signal 216 cannot be removed due to changes in the lighting environment. As a result, the measuring device 100 according to this modification can appropriately extract the frequency components caused by the time variation of the living organism 102 from the pulse signal 216 even when the lighting environment changes after adjusting the exposure time.

[0124] Additionally, the shooting control unit 1002 can adjust the exposure time at intervals exceeding a specified time interval. For example, when... Figure 11 In the illustrated steps S1107 or S1204, if the exposure time is adjusted but the preset minimum switching time has elapsed, the shooting control unit 1002 can maintain the exposure time even if the difference between the actual frame rate fr and the optimal frame rate 212 is not within the specified range. Then, if, after adjusting the exposure time in step S1107 or S1204, the exposure time has elapsed and the difference between the actual frame rate fr and the optimal frame rate 212 is not within the specified range, the shooting control unit 1002 can return to the previous state. Figure 11 The example shown is step S1108. Therefore, the shooting control unit 1002 can prevent frequent changes in exposure time when the difference between the actual frame rate fr and the optimal frame rate 212 is not within the specified range.

[0125] (Modification 4 of the third embodiment)

[0126] As a variation of the measuring device 100 in this embodiment, the flashing-related information 211 may also be stored in the storage unit 201. Figure 13 This is a block diagram illustrating an example of the configuration of the measuring device 100 involved in this modified example. Figure 10 The illustrated measuring device 100 and Figure 13 The difference of the illustrated measuring device 100 is that: Figure 13The illustrated measuring device 100 does not have a flicker detection unit 501, but it stores flicker-related information 211 in the storage unit 201.

[0127] According to the shooting control unit 1002 of this modified example, the optimal frame rate 212 is determined based on the presence or absence of flashing indicated by the flashing-related information 211 or the flashing frequency fq indicated by the flashing-related information 211. Therefore, the measuring device 100 of this modified example can control the exposure time so that the actual frame rate fr is close to the optimal frame rate based on the flashing frequency fq set according to the lighting conditions.

[0128] This disclosure is not limited to the above-described embodiments and variations. Various modifications can be made within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of this invention. Furthermore, new technical features can be formed by combining the technical methods disclosed in each embodiment.

Claims

1. A measuring device, characterized in that including: a photographing section that photographs a living body and acquires a dynamic image; a pixel value calculation section that calculates a representative value of pixel values of a region of interest, which contains an image of the living body, from each image constituting the dynamic image; and a pulse calculation section that calculates a pulse signal from a temporal change in the representative value, the photographing section photographs the living body at a plurality of frame rates, respectively, the measurement device further includes a flicker detection section that determines that flicker is detected when a period of a peak timing of a temporal change in a luminance value calculated from pixel values included in images photographed by the photographing section at frame rates different from each other among the plurality of frame rates is different, the photographing section photographs the living body at a frame rate of the dynamic image at which a frequency component caused by the flicker and the frame rate of the dynamic image can be separated from a frequency component caused by a temporal change in the living body included in the pulse signal.

2. The measuring device of claim 1, wherein, An absolute value of a difference between an integer multiple of a frequency of the flicker and the frame rate of the dynamic image is equal to or greater than a threshold value.

3. The measurement device according to claim 1, wherein the flicker detection section detects at least one selected from a presence or absence of the flicker and a frequency of the flicker, the measurement device further includes a photographing control section that determines an optimal frame rate based on the presence or absence of the flicker or the frequency of the flicker, the photographing section photographs the living body at the optimal frame rate.

4. The measuring device of claim 3, wherein, The photographing control section controls a setting of the photographing section according to a difference between an actual frame rate and the optimal frame rate, the actual frame rate being calculated from a time difference at which the each image is acquired.

5. The measuring device of claim 4, wherein, The case of controlling the setting includes a case of adjusting an exposure time.

6. The measuring device of claim 5, wherein, The photographing control section performs the adjustment of the exposure time at intervals of a prescribed time interval or more.

7. The measuring device according to any one of claims 4 to 6, characterized in that, The photographing control section controls the setting so that the difference between the actual frame rate and the optimal frame rate is within a prescribed range.

8. The measuring device according to any one of claims 3 to 6, characterized in that In the absence of the flicker, the photographing control section determines a prescribed frame rate as the optimal frame rate.

9. The measuring device according to any one of claims 3 to 6, characterized in that, In the absence of the flicker, the photographing control section determines a value associated with a category of a vital sign of a measurement target as the optimal frame rate.

10. The measuring device according to any one of claims 1 to 6, characterized in that The number of images acquired in a prescribed time is a number obtained by dividing a number of occurrences of flicker in the prescribed time by an integer.

11. A method of measurement, characterized by, including: a photographing section that photographs a living body and acquires a dynamic image; a pixel value calculation section that calculates a representative value of pixel values of a region of interest, which contains an image of the living body, from each image constituting the dynamic image; and a pulse calculation section that calculates a pulse signal from a temporal change in the representative value, in the process of acquiring the dynamic image, the photographing section photographs a photographing range of the photographing section at a plurality of frame rates, respectively, the measurement method further includes a process of determining that flicker is detected when a period of a peak timing of a temporal change in a luminance value calculated from pixel values included in images photographed at frame rates different from each other among the plurality of frame rates is different, In the function of acquiring the dynamic image, the living body is imaged at a frame rate of the dynamic image that enables separation of a frequency component caused by the flicker and the frame rate of the dynamic image from a frequency component caused by a temporal change of the living body included in the pulse signal.

12. A computer-readable recording medium recording a program, characterized by comprising: The program causes the computer to function as: a function of imaging a living body and acquiring a dynamic image by an imaging unit; a function of calculating a representative value of pixel values of a region of interest from each image constituting the dynamic image, the region of interest including an image of the living body; and a function of calculating a pulse signal from a temporal change of the representative value, in the function of acquiring the dynamic image, the imaging range of the imaging unit is imaged at a plurality of frame rates, the program further causes the computer to function as: when a period of a peak timing of a temporal change of a luminance value calculated from pixel values included in images imaged at frame rates different from each other among the plurality of frame rates is different, determining that flicker is detected, in the function of acquiring the dynamic image, the living body is imaged at a frame rate of the dynamic image that enables separation of a frequency component caused by the flicker and the frame rate of the dynamic image from a frequency component caused by a temporal change of the living body included in the pulse signal.

Images