Measuring device, measuring method, and program

Abstract

To provide a measuring apparatus capable of separating a noise caused by a flicker and a frequency component caused by the time change of an organism from each other.SOLUTION: The measuring apparatus comprises: an imaging unit which images an organism to acquire a moving image; a pixel value calculation unit which calculates a representative value of pixel values in a region of interest including an image of the organism, from each image constituting the moving image; and a pulse wave calculation unit which calculates a pulse wave signal from the time change of the representative value. The imaging unit images the organism with a frame rate capable of separating a frequency component caused by a flicker and a frame rate included in the pulse wave signal and a frequency component caused by the time change of the organism included in the pulse wave signal from each other.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] This disclosure relates to measuring devices, measuring methods, and programs. [Background technology]

[0002] Patent Document 1 discloses a technique for measuring biological information such as pulse rate, pulse waveform, pulse wave propagation velocity, and blood pressure by analyzing the difference signal of image information from two or three adjacent regions along the blood flow, thereby suppressing external noise. Patent Document 2 discloses a technique for calculating illumination fluctuation components and hemoglobin components based on moving images of two different wavelength ranges, and for estimating heart rate variability based on the calculated hemoglobin component. Patent Document 3 discloses a technique in which, when the vertical synchronization frequency of the imaging device is fv (Hz) and N is a positive integer, the electronic shutter speed is set to an exposure time of N / 120 (seconds) if 120 / fv is an integer, and to an exposure time of N / 100 (seconds) if 100 / fv is an integer. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] International release WO2016 / 163019 [Patent Document 2] Japanese Patent Publication No. 2019-042145 [Patent Document 3] Japanese Patent Publication No. 2005-033616 [Overview of the project] [Problems that the invention aims to solve]

[0004] In the technologies disclosed in Patent Documents 1 and 2, if the frequency of the flicker noise and the pulse rate are similar, there is a risk that the flicker noise and the frequency components caused by the time changes of the living body cannot be separated. Therefore, one aspect of this disclosure aims to provide a measuring device, a measuring method, and a program that can separate the flicker noise from the frequency components caused by the time changes of the living body. [Means for solving the problem]

[0005] A measuring device according to one embodiment of the present disclosure includes an imaging unit that images a living body and acquires a moving image, a pixel value calculation unit that calculates a representative value of the pixel value of a region of interest including the image of the living body from each image constituting the moving image, and a pulse wave calculation unit that calculates a pulse wave signal from the time change of the representative value, wherein the imaging unit images the living body at a frame rate that allows for the separation of frequency components caused by flicker and frame rate included in the pulse wave signal and frequency components caused by the time change of the living body included in the pulse wave signal.

[0006] A measurement method according to one embodiment of the present disclosure includes the steps of: imaging a living organism to acquire a moving image; calculating a representative value of the pixel value of a region of interest including an image of the living organism from each image constituting the moving image; and calculating a pulse wave signal from the time change of the representative value, wherein in the step of acquiring the moving image, the living organism is imaged at a frame rate that allows for the separation of frequency components caused by flicker and frame rate included in the pulse wave signal from frequency components caused by the time change of the living organism included in the pulse wave signal.

[0007] A program according to one embodiment of the present disclosure causes a computer to perform the following functions: a function to image a living organism and acquire a moving image; a function to calculate a representative value of the pixel values ​​of a region of interest, including an image of the living organism, from each image constituting the moving image; and a function to calculate a pulse wave signal from the time change of the representative value. In the function to acquire the moving image, the living organism is imaged at a frame rate that allows for the separation of frequency components caused by flicker and frame rate included in the pulse wave signal from frequency components caused by the time change of the living organism included in the pulse wave signal. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows an example of how the measuring device is used. [Figure 2] This is a block diagram showing an example of the configuration of a measuring device according to the first embodiment. [Figure 3] This figure shows an example of information related to flicker. [Figure 4] This is a flowchart illustrating an example of the operation of the measuring device according to the first embodiment. [Figure 5] This is a block diagram showing an example of the configuration of a measuring device according to the second embodiment. [Figure 6] This figure shows an example of a frame rate table. [Figure 7] This flowchart shows an example of the operation of the measuring device according to the second embodiment. [Figure 8] This graph shows an example of the relationship between aliasing distortion frequency and frame rate when the flicker frequency is 120 Hz. [Figure 9] This is an enlarged view of the graph shown in Figure 8, specifically the range where the frame rate is between 35fps and 55fps. [Figure 10] This is a block diagram showing an example of the configuration of a measuring device according to the third embodiment. [Figure 11] This flowchart shows an example of the operation of the measuring device according to the third embodiment. [Figure 12] Figure 11 shows a flowchart illustrating an example of the operation of the measuring device according to the third embodiment. [Figure 13] This is a block diagram showing an example of the configuration of a measuring device according to a modified example of the third embodiment. [Modes for carrying out the invention]

[0009] (First Embodiment) Referring to FIGS. 1 to 4, a first embodiment will be described. In the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant descriptions are omitted.

[0010] FIG. 1 is a diagram showing an example of the usage mode of the measuring device 100. As illustrated in FIG. 1, the measuring device 100 includes an imaging unit 101.

[0011] The measuring device 100 calculates a pulse wave signal indicating a pulse wave from an image acquired by the imaging unit 101. For example, the measuring device 100 is a PC (Personal Computer), a smartphone, a tablet terminal, a dedicated pulse wave estimation terminal, or the like. In this specification, the pulse wave is a time-series signal indicating a change in the volume of a blood vessel, which is calculated from a time-series signal indicating the pixel value of a pixel included in an image with respect to the same position on the body surface. In this specification, the pixel value is information indicating the brightness of a pixel included in an image, and for example, is the pixel value of a pixel for each of R (Red), G (Green), and B (Blue) or the luminance value of a pixel.

[0012] For example, the imaging unit 101 is composed of a CCD (Charged Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The imaging unit 101 may be composed of an image sensor for a camera including an RGB (Red Green Blue) filter.

[0013] FIG. 2 is a block diagram showing an example of the configuration of the measuring device 100. The measuring device 100 includes an imaging unit 101, a storage unit 201, and a control unit 202.

[0014] The imaging unit 101 images the living body 102 facing the imaging unit 101 and acquires an image including an image of the body surface of the living body 102. For example, the image of the body surface is an image of the forehead of the living body 102, an image of the cheek, an image of the fingertip, an image of the wrist, an image of the palm, or the like.

[0015] Specifically, the imaging unit 101 images the living organism 102 for a predetermined time at a frame rate that allows for the separation of frequency components caused by flicker and frame rate in the pulse wave signal from frequency components caused by the time change of the living organism 102 in the pulse wave signal, thereby acquiring an image including the surface of the living organism 102. The frequency components caused by the time change of the living organism 102 are frequency components that indicate changes in pixel values ​​due to changes in the volume of blood vessels. Flicker is a periodic change in the brightness of a light source. When flicker occurs, the pixel values ​​of pixels included in the image acquired by the imaging unit 101 change periodically. When a periodic signal is observed at a frequency different from the signal, a signal called aliasing distortion is observed. Therefore, when flicker occurs and the frame rate is different from the frequency of the flicker, aliasing distortion is observed in the signal indicating the time-series pixel values.

[0016] The storage unit 201 is a recording medium capable of storing various data, programs, etc., and is composed of, for example, a hard disk, an SSD (Solid State Drive), or a semiconductor memory. The storage unit 201 stores flicker-related information 211 (see Figure 3) concerning the settings of the imaging unit 101.

[0017] The control unit 202 executes various processes according to the programs and data stored in the memory unit 201. The control unit 202 is implemented by a processor such as a CPU (Central Processing Unit).

[0018] The control unit 202 includes an imaging control unit 203, a pixel value calculation unit 204, a pulse wave calculation unit 205, and a vital indicator calculation unit 206.

[0019] The imaging control unit 203 determines the optimal frame rate 212 based on the presence or absence of flicker or the flicker frequency fq. Then, the imaging control unit 203 controls the settings of the imaging unit 101 so that the living organism 102 is imaged at the optimal frame rate 212. Specifically, the imaging control unit 203 determines imaging control information 213 so that the living organism 102 is imaged at the optimal frame rate 212. The imaging control information 213 includes information indicating the exposure time. In other words, controlling the settings of the imaging unit 101 includes adjusting the exposure time.

[0020] The pixel value calculation unit 204 calculates a representative value 215 of the pixel value of the region of interest, which includes the image of the living organism 102, from each image that makes up the moving image 214.

[0021] The pulse wave calculation unit 205 calculates a pulse wave signal 216 from the time change of the representative value 215.

[0022] The vital sign indicator calculation unit 206 calculates the vital signs to be measured from the pulse wave signal 216. For example, the vital signs to be measured include indicators related to blood pressure and indicators related to pulse rate.

[0023] Figure 3 shows an example of flicker-related information 211. Flicker-related information 211 indicates the presence or absence of flicker and the flicker frequency fq. The flicker-related information 211 exemplified in Figure 3 indicates the presence of flicker. Furthermore, the flicker-related information 211 exemplified in Figure 3 indicates that the flicker frequency fq is 120 Hz. For example, the flicker frequency fq indicated by the flicker-related information 211 is a value corresponding to the lighting. For example, in Japan, flicker with a frequency of 120 Hz may occur in western Japan, which uses a 60 Hz power supply, and flicker with a frequency of 100 Hz may occur in eastern Japan, which uses a 50 Hz power supply. Note that if the flicker-related information 211 indicates the absence of flicker, it is not necessary to indicate the flicker frequency fq.

[0024] Figure 4 is a flowchart illustrating an example of the operation of the measuring device 100 according to this embodiment. In this example, when the measuring device 100 is started, the control unit 202 starts processing step S401 as illustrated in Figure 4. At the time when the control unit 202 starts processing step S401 as illustrated in Figure 4, it is assumed that the storage unit 201 has flicker-related information 211 stored in it.

[0025] In step S401, the imaging control unit 203 determines whether or not there is flicker. Specifically, if the flicker-related information 211 indicates that there is flicker, the imaging control unit 203 determines that there is flicker. On the other hand, if the flicker-related information 211 indicates that there is no flicker, the imaging control unit 203 determines that there is no flicker.

[0026] If it is determined that there is no flicker in step S401, the imaging control unit 203 determines a predetermined frame rate as the optimal frame rate 212 in step S402. Specifically, the imaging control unit 203 determines imaging control information 213 so that the imaging unit 101 images the living organism 102 at the predetermined frame rate. For example, the predetermined frame rate is 60 fps (frames per second). Then, the control unit 202 moves the process to step S404.

[0027] On the other hand, if it is determined in step S401 that flicker is present, the imaging control unit proceeds to step S403. 203 The optimal frame rate 212 is determined based on the flicker frequency fq. Specifically, the imaging control unit 203If flicker is detected in step S401, the optimal frame rate 212 is determined based on the flicker frequency fq indicated by the flicker-related information 211. More specifically, if flicker is present, the imaging control unit 203 determines the optimal frame rate 212 such that the absolute value of the difference between the flicker frequency fq and an integer multiple of the optimal frame rate 212 is greater than or equal to the threshold TH. For example, the threshold TH is a predetermined value. Alternatively, the threshold TH may be a value corresponding to the type of vital indicator being measured. Then, the control unit 202 proceeds to step S404.

[0028] In step S404, the imaging control unit 203 controls the imaging unit 101 to acquire a moving image 214 by imaging the living body 102 for a predetermined time at the optimal frame rate 212 determined in step S402 or step S403. The imaging unit 101 outputs each image constituting the acquired moving image 214 to the pixel value calculation unit 204.

[0029] For example, the imaging unit 101 controls itself to image the living organism 102 at an optimal frame rate 212 for a predetermined time, and outputs each image constituting the moving image 214 to the pixel value calculation unit 204. In this case, if there is flicker, the number of images acquired in the predetermined time is the number obtained by dividing the number of flicker occurrences in the predetermined time by an integer. Alternatively, when the imaging unit 101 has acquired each image constituting the moving image 214, the imaging unit 101 may output each image constituting the moving image 214 to the pixel value calculation unit 204 at a timing when the pixel value calculation unit 204 can accept image input.

[0030] For example, if flicker is present, aliasing distortion is observed in the time-series signal showing the time change in the pixel values ​​of the pixels included in the video 214. Aliasing distortion is a frequency component caused by the flicker and frame rate included in the time-series signal, and in the time-series signal, it has a frequency that is the absolute value of the difference between the flicker frequency fq and an integer multiple of the frame rate. In other words, the frequency fa of aliasing distortion is calculated by equation (1), where N is a positive integer. The frame rate fr is the actual frame rate at which the imaging unit 101 acquired each image constituting the video 214.

number

[0031] For example, if the flicker frequency fq is 120 Hz, and the imaging unit 101 images the living organism 102 with an exposure time of 1 / 60 second, no aliasing distortion is observed in the time-series signal showing the time change of pixel values ​​of pixels included in the moving image 214. Therefore, when the flicker frequency fq is 120 Hz, it is desirable to image the living organism 102 with an exposure time of 1 / 60 second in order to suppress the effect of flicker. However, in reality, the imaging unit 101 cannot always stably image the living organism 102 with an exposure time of 1 / 60 second. For example, if the processing time in the pixel value calculation unit 204 is longer than the time between frames based on the optimal frame rate 212, the actual frame rate fr will be lower than the optimal frame rate 212. Therefore, if the imaging control unit 203 determines the optimal frame rate 212 to be 60fps, the exposure time may be shorter than 1 / 60th of a second in order to acquire each image that makes up the moving image 214 at a frame rate of 60fps.

[0032] Furthermore, if the exposure time can be set as a decimal in the imaging unit 101, 1 / 60th of a second is a repeating decimal, so the imaging unit 101 cannot accurately set an exposure time of 1 / 60th of a second.

[0033] Alternatively, the frame rate fr may not be stable and may fluctuate randomly due to the processing time of the imaging unit 101 and the processing time between frames that make up the moving image 214. In that case, even if the imaging control unit 203 determines the optimal frame rate 212 to be 60fps, the imaging unit 101 may stabilize at a frame rate different from the optimal frame rate 212 of 60fps.

[0034] For example, suppose the flicker frequency fq is 120 Hz, and the imaging unit 101 acquires each image constituting the moving image 214 at a frame rate fr of 59.9 fps. In that case, aliasing distortion with a frequency fa of 0.2 Hz (= |120 ​​- 2 × 59.9|) is observed. In other words, the pixel values ​​of each image constituting the moving image 214 will fluctuate with a period of 5 seconds.

[0035] Furthermore, for example, suppose the flicker frequency fq is 120 Hz, and the imaging unit 101 acquires each image constituting the moving image 214 at a frame rate fr of 59.5 fps. In that case, aliasing distortion with a frequency fa of 1 Hz (= |120 ​​- 2 × 59.5|) is observed.

[0036] For example, a frequency component with a frequency fa of 1 Hz due to aliasing distortion is the same as the frequency component in the pulse wave signal 216 calculated in step S407 (described later) when the pulse rate is 60 beats / minute. Since the frequency of a resting pulse in a healthy adult is 0.7 Hz to 1.5 Hz, if the pulse wave signal 216 contains aliasing distortion of 1 Hz, the vital indicator calculation unit 206 in step S409 (described later) cannot properly calculate the pulse rate from the pulse wave signal 216.

[0037] Therefore, the imaging control unit 203 determines the optimal frame rate 212 so that even if the actual frame rate fr at which each image constituting the moving image 214 is acquired is lower than the optimal frame rate 212, it can separate the frequency components caused by the flicker and frame rate in the pulse wave signal 216 calculated in step S407 (described later) from the frequency components caused by the time change of the living organism 102 in the pulse wave signal 216.

[0038] In step S405, the pixel value calculation unit 204 determines a region of interest for each image constituting the moving image 214. The region of interest is a part of the body surface region in each image and is a region containing multiple pixels. For example, if the moving image 214 includes the face region of the living body 102, the region of interest includes the cheek area, the forehead area, or the area between the eyebrows. There may be one or more regions of interest. The shape of the region of interest may be a polygon enclosed by straight lines, or a shape enclosed by curves. Alternatively, the region of interest may be a closed region composed of straight lines and curves.

[0039] In step S406, the pixel value calculation unit 204 calculates a representative value 215 of the pixel values ​​of the region of interest determined in step S405 for each image constituting the moving image 214. For example, the representative value 215 is the mean, median, or mode of the pixel values ​​of each pixel in the region of interest. If the imaging unit 101 is composed of an image sensor for a camera that includes RGB filters, the pixel value calculation unit 204 may also calculate representative values ​​of the pixel values ​​for R, G, and B respectively.

[0040] In step S407, the pulse wave calculation unit 205 calculates the pulse wave signal 216 from the time change of the representative value 215. Specifically, the pulse wave calculation unit 205 calculates the pulse wave signal 216 from the time change of the representative value 215 corresponding to the same position in a region of the body surface. For example, the pulse wave calculation unit 205 processes the signal showing the time change of the representative value 215 by multivariate analysis such as principal component analysis and independent component analysis, and calculates the processed result as the pulse wave signal 216. The time change of the representative value includes information on changes in the volume of blood vessels.

[0041] In step S408, the vital indicator calculation unit 206 extracts a signal of a predetermined frequency band from the pulse wave signal 216. For example, the predetermined frequency band is a frequency band corresponding to the type of vital indicator being measured. Specifically, the vital indicator calculation unit 206 inputs the pulse wave signal 216 into a bandpass filter of a frequency band corresponding to the type of vital indicator being measured, thereby extracting a signal of a frequency band corresponding to the type of vital indicator being measured from the pulse wave signal 216.

[0042] Here, the imaging control unit 203 determines the optimal frame rate 212 such that the absolute value of the difference between the flicker frequency fq and an integer multiple of the frame rate is greater than or equal to the threshold TH. This allows the vital indicator calculation unit 206 to separate the frequency components in the pulse wave signal 216 that are caused by the flicker and frame rate from the frequency components in the pulse wave signal 216 that are caused by the time change of the living organism 102.

[0043] The pulse wave signal 216 contains noise with frequencies higher than those caused by the time changes of the biological tissue 102, not just aliasing distortion. Therefore, the vital indicator calculation unit 206 can extract frequency components in the frequency band corresponding to the type of vital indicator being measured from the pulse wave signal 216 by removing high-frequency components from the pulse wave signal 216 that are higher than the frequency band corresponding to the type of vital indicator being measured.

[0044] For example, if the flicker frequency fq is 120 Hz, and in step S404 the imaging unit 101 acquires each image constituting the moving image 214 at an actual frame rate fr of 55 fps, then aliasing distortion with a frequency fa of 10 Hz (= |120 ​​- 2 × 55|) is observed.

[0045] For example, suppose the vital indicator to be measured is pulse rate, and the frequency band corresponding to the type of vital indicator, which is pulse wave number, is between 0.7 Hz and 1.5 Hz. And suppose aliasing distortion with frequency fa of 10 Hz is observed. In that case, for example, the vital indicator calculation unit 206 inputs a bandpass filter with a frequency band of 0.7 Hz and 1.5 Hz to the pulse wave signal 216. As a result, the vital indicator calculation unit 206 can exclude the frequency component with a frequency of 10 Hz from the pulse wave signal 216.

[0046] Furthermore, suppose, for example, that the flicker frequency fq is 100 Hz and the imaging unit 101 acquires each image constituting the video 214 at an actual frame rate fr of 52 fps. In that case, aliasing distortion with a frequency fa of 4 Hz (= |100 - 2 × 52|) is observed.

[0047] Furthermore, for example, suppose in step S403 the imaging control unit 203 determines the optimal frame rate 212 to be 60 fps. In that case, suppose the flicker frequency fq is 120 Hz and the imaging unit 101 acquires each image constituting the moving image 214 at an actual frame rate fr of 62 fps. In that case, aliasing distortion with a frequency fa of 4 Hz (= |120 ​​- 2 × 62|) is observed.

[0048] For example, the vital indicator calculation unit 206 inputs a low-pass filter with a cutoff frequency of 4 Hz to the pulse wave signal 216. This allows the vital indicator calculation unit 206 to remove frequency components with frequencies higher than 4 Hz from the pulse wave signal 216.

[0049] Therefore, in step S403, the imaging control unit 203 determines the optimal frame rate 212 such that the absolute value of the difference between the flicker frequency and an integer multiple of the frame rate is greater than or equal to the threshold TH. This allows the vital indicator calculation unit 206 to separate the frequency components caused by the flicker and frame rate from the frequency components caused by the time change of the biological tissue 102 contained in the pulse wave signal 216.

[0050] In step S409, the vital indicator calculation unit 206 calculates the vital indicator to be measured from the signal extracted in step S408. For example, if the type of vital indicator to be measured is blood pressure, the vital indicator calculation unit 206 calculates the systolic blood pressure based on the rising angle of the signal extracted in step S408. Also, for example, if the type of vital indicator to be measured is pulse rate, the vital indicator calculation unit 206 calculates the pulse rate based on the number of peaks in the signal extracted in step S408.

[0051] Based on the above, the measuring device 100 according to this embodiment determines an optimal frame rate 212 so that it can separate the frequency components caused by flicker and frame rate in the pulse wave signal 216 from the frequency components caused by the time change of the living body 102 in the pulse wave signal 216. The measuring device 100 according to this embodiment can suppress the effects of noise caused by flicker and calculate vital indicators such as blood pressure from the moving image 214 of the living body 102.

[0052] Furthermore, for example, Patent Document 3 discloses that when the imaging unit 101 acquires each image constituting the moving image 214 at a frame rate of 60 fps, the effect of flicker with a frequency fq of 120 Hz can be suppressed by acquiring each image constituting the moving image 214 with an exposure time of 1 / 120 second. However, in the measurement device 100 according to this embodiment, the imaging unit 101 may image the living body 102 with an exposure time longer than 1 / 120 second and acquire the moving image at a frame rate fr of 60 fps. In other words, even when the imaging unit 101 images the living body 102 with an exposure time longer than 1 / 120 second and acquires the moving image at a frame rate of 60 fps, the measurement device 100 according to this embodiment can separate the frequency components caused by the flicker and frame rate included in the pulse wave signal 216 from the frequency components caused by the time change of the living body 102 included in the pulse wave signal 216. Therefore, the measuring device 100 according to this embodiment can appropriately calculate vital indicators of the target of measurement even when imaging the living body 102 in a dark environment, compared to when imaging the living body 102 using the technology disclosed in Patent Document 3.

[0053] (Modification of the first embodiment) As a modification of the measuring device 100 according to this embodiment, the imaging control unit 203 may determine the optimal frame rate 212 as a value associated with the type of vital indicator to be measured when there is no flicker. This allows the imaging unit 101 to image the living body 102 at a frame rate corresponding to the type of vital indicator to be measured.

[0054] (Second embodiment) The second embodiment will be described with reference to Figures 5 to 9. In the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant explanations are omitted. Configurations and processes that have substantially the same function as the other embodiments are referred to by the same reference numerals and their explanations are omitted, while the differences from the other embodiments will be explained.

[0055] Figure 5 is a block diagram showing an example of the configuration of the measuring device 100 according to this embodiment. The difference between the measuring device 100 illustrated in Figure 2 and the measuring device 100 illustrated in Figure 5 is that the measuring device 100 illustrated in Figure 5 includes a flicker detection unit 501 and stores a frame rate table 511 in the storage unit 201.

[0056] The flicker detection unit 501 detects at least one selected from the group consisting of the presence or absence of flicker and the frequency of the flicker.

[0057] Figure 6 shows an example of a frame rate table 511. In the frame rate table 511, registered frame rates are associated with registered frequencies. Specifically, the frame rate table 511 registers frame rates such that the absolute value of the difference between the registered frequency and an integer multiple of the registered frame rate is greater than or equal to a threshold.

[0058] For example, if the registration frequency is 120Hz, the registration frame rate is 50fps. Also, for example, if the registration frequency is 100Hz, the registration frame rate is 60fps. Note that the registration frequency and registration frame rate exemplified in Figure 6 are examples only, and it is not intended to limit the registration frequency and registration frame rate to the values ​​exemplified in Figure 6.

[0059] For example, the registered frequency and registered frame rate may be associated with each manufacturer and model of the measuring device 100 and registered in the frame rate table 511. Alternatively, the user may input the registered frequency and registered frame rate using an operating unit (not shown) provided on the measuring device 100, thereby associating the registered frequency and registered frame rate and registering them in the frame rate table 511.

[0060] Figure 7 is a flowchart illustrating an example of the operation of the measuring device 100 according to this embodiment. In this example, when the measuring device 100 is started, the control unit 202 starts the process of step S701 as illustrated in Figure 7. At the time when the control unit 202 starts the process of step S701 as illustrated in Figure 7, the storage unit 201 is assumed to have the frame rate table 511 stored in it.

[0061] In step S701, the imaging control unit 203 activates the imaging unit 101. When activated, the imaging unit 101 starts the process of imaging the imaging range and acquiring an image. The image acquired in step S701 is an image acquired to detect the presence or absence of flicker. When the imaging unit 101 acquires an image to detect the presence or absence of flicker, there does not need to be a living organism 102 in the imaging range.

[0062] In step S702, the flicker detection unit 501 determines whether or not flicker has been detected. For example, the flicker detection unit 501 calculates a brightness value from the pixel values ​​of pixels within a predetermined area included in the image captured in order to detect the presence or absence of flicker. The predetermined area is an area different from the body surface. In this case, the flicker detection unit 501 detects the timing of the peak of the time change in brightness value from the time-series signal showing the time change of the calculated brightness value. If the timing of the peak of the time change in brightness value is periodic, the flicker detection unit 501 determines that flicker has been detected. For example, the flicker detection unit 501 determines that flicker has been detected if the timing of the peak of the time change in brightness value is a period caused by a preset flicker. On the other hand, if the timing of the peak of the time change in brightness value is not periodic, the flicker detection unit 501 determines that flicker has not been detected. Furthermore, the flicker detection unit 501 may capture the imaging range at multiple frame rates and determine that flicker has been detected if the timing periods of the peaks of the time change in brightness values ​​differ at different frame rates.

[0063] If no flicker is detected in step S702, in step S703 the imaging control unit 203 determines a predetermined frame rate as the optimal frame rate 212. The process in step S703 is the same as the process in step S402 illustrated in Figure 4, so a detailed explanation is omitted. Then, the control unit 202 proceeds to step S404 illustrated in Figure 4.

[0064] On the other hand, if flicker is detected in step S702, the flicker detection unit 501 detects the flicker frequency fq in step S704. For example, if the flicker detection unit 501 detects the timing of the peak of the time change in the brightness value, it detects the flicker frequency fq from the time interval of the detected timing. Alternatively, the flicker detection unit 501 may calculate a frequency spectrum relating to a time-series signal showing the time change in the brightness value. Then, the flicker detection unit 501 may detect the peak frequency within a predetermined frequency band in the calculated frequency spectrum as the flicker frequency fq.

[0065] In step S705, the imaging control unit 203 selects a registered frequency from the frame rate table 511 that is the closest value to the flicker frequency fq detected in step S704.

[0066] In step S706, the imaging control unit 203 determines the registered frame rate associated with the selected registered frequency in the frame rate table 511 as the optimal frame rate 212. Then, the control unit 202 proceeds to step S404, which is illustrated in Figure 4.

[0067] Thus, the imaging control unit 203 according to this embodiment determines the frame rate associated with the flicker frequency fq as the optimal frame rate. As a result, the imaging control unit 203 according to this embodiment may be able to determine a frame rate higher than the optimal frame rate 212 determined by the imaging control unit 203 according to the first embodiment as the optimal frame rate 212. For example, suppose that the storage unit 201 stores a frame rate table 511 as illustrated in Figure 5. In that case, for example, if the registered frequency selected in step S704 is 100 Hz, the imaging control unit 203 determines the optimal frame rate 212 to be 60 Hz.

[0068] Figure 8 is a graph showing an example of the relationship between aliasing distortion frequency and frame rate f when the flicker frequency fq is 120 Hz. In Figure 8, the horizontal axis represents the frame rate, and the vertical axis represents the aliasing distortion frequency. Graphs 801 to 804 show the relationship between aliasing distortion frequencies of |120-f|, |120-2f|, |120-3f|, and |120-4f|, respectively, and the frame rate f.

[0069] Figure 9 is an enlarged view of the graph shown in Figure 8, specifically the range where the frame rate f is between 35fps and 55fps.

[0070] For example, suppose the imaging unit 101 images a living organism 102 when the flicker frequency fq is 120 Hz. Referring to Figure 9, for example, if the actual frame rate fr is 45 fps, an aliasing distortion frequency fa of 15 Hz or higher is observed. Also referring to Figure 9, for example, if the actual frame rate fr is 50 fps, an aliasing distortion frequency fa of 20 Hz or higher is observed. Therefore, for example, by the imaging control unit 203 determining the optimal frame rate 212 to 50 fps, even if the actual frame rate fr drops to 45 fps, the vital indicator calculation unit 206 can extract signals with frequency components lower than 15 Hz from the pulse wave signal 216.

[0071] Based on the above, the measuring device 100 according to this embodiment detects the presence or absence of flicker and controls the settings of the imaging unit 101 to image the living body 102 at a pre-registered frame rate according to the flicker frequency fq. As a result, the measuring device 100 according to this embodiment can suppress the effects of noise due to flicker and image the living body 102 at the highest possible frame rate. Consequently, the measuring device 100 according to this embodiment can suppress the effects of noise due to flicker and obtain a pulse wave with higher temporal resolution than the pulse wave signal 216 calculated by the measuring device 100 according to the first embodiment. signal You can obtain 216.

[0072] (Third embodiment) The third embodiment will be described with reference to Figures 10 to 12. In the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant explanations are omitted. Configurations and processes that have substantially the same function as the other embodiments are referred to by the same reference numerals and their explanations are omitted, while the differences from the other embodiments will be explained.

[0073] Figure 10 is a block diagram showing an example of the configuration of the measuring device 100 according to this embodiment. The difference between the measuring device 100 illustrated in Figure 6 and the measuring device 100 illustrated in Figure 10 is that the measuring device 100 illustrated in Figure 10 includes an imaging interval calculation unit 1001 and an imaging control unit 1002 instead of an imaging control unit 203.

[0074] The imaging interval calculation unit 1001 calculates the actual frame rate fr from the time difference between the acquisition of each image constituting the moving image 214.

[0075] The imaging control unit 1002 controls the settings of the imaging unit 101 according to the difference between the actual frame rate fr and the optimal frame rate 212. In other words, the imaging control unit 1002 determines the imaging control information 1011 according to the difference between the actual frame rate fr and the optimal frame rate 212. Specifically, the imaging control unit 1002 controls the settings of the imaging unit 101 so that the difference between the actual frame rate fr and the optimal frame rate 212 is within a predetermined range.

[0076] Figure 11 is a flowchart showing an example of the operation of the measuring device 100 according to this embodiment. The processes of steps S1101 to S1106 illustrated in Figure 11 are the same as the processes of steps S701 to S706 illustrated in Figure 7, so a detailed explanation is omitted.

[0077] In step S1107, the imaging control unit 1002 adjusts the exposure time ET1 based on the optimal frame rate 212 determined in step S1003 or step S1106, and determines the settings for the imaging unit 101. In other words, the imaging control unit 1002 determines imaging control information 1011 which includes information indicating the exposure time ET1. The imaging control information 1011 includes information indicating the exposure time.

[0078] In step S1108, the imaging unit 101 images the living organism 102 and acquires an image using the settings for the imaging unit 101 determined in step S1107 or step S1204, which will be described later. In other words, the imaging control unit 1002 controls the imaging unit 101 to take an image with an exposure time ET1 indicated by the imaging control information 1011.

[0079] In step S1109, the pixel value calculation unit 204 calculates a representative value 215 of the pixel values ​​of the region of interest for the image acquired in step S1108. For example, the process in step S1109 is the same as the process in step S406 illustrated in Figure 4. Alternatively, the pixel value calculation unit 204 may normalize the representative value 215 based on the exposure time. For example, the pixel value calculation unit 204 may normalize the representative value 215 to the value for an exposure time of 1 second. For example, suppose the representative value 215 is 200, and the exposure time when the image for which the representative value 215 was calculated was acquired was 10 ms. In that case, the pixel value calculation unit 204 may normalize the representative value 215 to 20000 [ / second] (= 200 ÷ 0.010 seconds).

[0080] In step S1110, the imaging control unit 1002 obtains the time when the image was acquired in step S1108. For example, if the imaging control unit 1002 obtains the time when the image was acquired, the imaging control unit 1002 stores time information in the storage unit 201 that associates the time when the image was acquired with the frame number assigned to the image.

[0081] In step S1111, the imaging control unit 1002 determines whether a predetermined number of frames of images have been acquired based on the settings of the imaging unit 101 determined in step S1107 or step S1204, which will be described later. For example, a predetermined number of frames is two consecutive frames.

[0082] If a predetermined number of frames have not been acquired in step S1111, the control unit 202 returns to step S1108. In other words, the control unit 202 repeats the process from step S1108 to step S1111 with the determined settings of the imaging unit 101 until a predetermined number of frames have been acquired. On the other hand, if a predetermined number of frames have been acquired in step S1111, the control unit 202 proceeds to step S1201, which is illustrated in Figure 12.

[0083] Next, with reference to Figure 12, the operation of the measuring device 100 according to this embodiment will be described further.

[0084] In step S1201, the imaging interval calculation unit 1001 calculates the actual frame rate fr from the time difference between the images acquired in step S1110, as illustrated in Figure 11, for a predetermined number of frames. For example, the imaging interval calculation unit 1001 calculates the time difference between the acquisition of a predetermined number of frames by acquiring time information from the storage unit 201.

[0085] For example, if the predetermined number of frames consists of two frames, the imaging interval calculation unit 1001 calculates the absolute value of the difference between time t1 and time t0, when the images of two consecutive frames were acquired, as the time difference. Time t0 is the time when the image of the earlier of the two consecutive frames was acquired. Time t1 is the time when the image of the frame acquired after time t0 of the two consecutive frames. The imaging interval calculation unit 1001 then calculates the reciprocal of the calculated time difference as the actual frame rate fr. In other words, the imaging interval calculation unit 1001 calculates the actual frame rate fr using the formula fr = 1 / (t1-t0).

[0086] Alternatively, for example, if the predetermined number of frames is three or more frames, the actual frame rate fr may be calculated from the time difference between the acquisition of images of two consecutive frames among the three or more frames.

[0087] For example, if a predetermined number of frames is N frames, the time at which the image of each frame was acquired is, respectively, time t i Assume that N is an integer greater than or equal to 3, and i is an integer greater than or equal to 0 and less than or equal to N-1. In that case, the frame rates of two consecutive frames out of N frames are 1 / (t k ―t k-1 ) where k is an integer between 1 and N-1. The imaging interval calculation unit 1001 calculates a representative value of the frame rate for two consecutive frames out of the N frames as the actual frame rate fr. For example, the imaging interval calculation unit 1001 calculates fr = (Σ N-1 k=1(1 / (t k - t k-1 )) / (N - 1), the actual frame rate fr is calculated. For example, when N = 4, the imaging interval calculation unit 1001 calculates the actual frame rate fr according to the calculation formula fr = ((1 / (t3 - t2) + 1 / (t2 - t1) + 1 / (t1 - t0)) / 3).

[0088] Alternatively, the imaging interval calculation unit 1001 may calculate the actual frame rate fr according to the calculation formula fr = (N - 1) / (t N-1 - t0). For example, when the number of predetermined frames is 30 frames, the imaging interval calculation unit 1001 calculates the actual frame rate fr according to the calculation formula fr = 29 / (t 29 - t0).

[0089] If the actual frame rate fr in step S1202 is lower than the optimal frame rate 212, in step S1203, the imaging control unit 1002 determines a new exposure time ET2 that is shorter than the exposure time ET1 when the image was acquired in step S1108 illustrated in FIG. 11. By making the new exposure time ET2 shorter than the exposure time ET1, the subsequent actual frame rate fr2 will be higher than the actual frame rate fr1 when the image was acquired in step S1107.

[0090] Note that when the imaging control unit 1002 makes the new exposure time ET2 shorter than the exposure time ET1, for the images of the same region of the living body 102, the pixel value PIX2 of the image I2 captured with the new exposure time ET2 will be smaller than the pixel value PIX1 of the image I1 captured with the exposure time ET1. As a result, the signal-to-noise ratio of the pixel value PIX2 may decrease compared to the signal-to-noise ratio of the pixel value PIX1.

[0091] Therefore, the imaging control unit 1002 may determine the new exposure time ET2 such that the new exposure time ET2 is greater than or equal to a preset lower limit exposure time ETMIN. Alternatively, the imaging control unit 1002 may set the lower limit exposure time ETMIN according to the brightness of the imaging environment in which the measuring device 100 is located. For example, the imaging control unit 1002 may set the lower limit exposure time ETMIN in a relatively bright environment to be shorter than the lower limit exposure time ETMIN in a relatively dark environment. Alternatively, the imaging control unit 1002 may set the lower limit exposure time ETMIN such that the representative value 215 is greater than or equal to a preset lower limit pixel value PIXMIN.

[0092] In step S1204, the imaging control unit 1002 determines the settings for the imaging unit 101, including the new exposure time ET2 determined in step S1203. Specifically, the imaging control unit 1002 determines imaging control information 1011, including the new exposure time ET2. Then, the control unit 202 proceeds to step S1108, which is illustrated in Figure 11.

[0093] On the other hand, if the actual frame rate fr in step S1202 is greater than or equal to the optimal frame rate 212, in step S1205, the imaging control unit 1002 determines whether the difference between the actual frame rate fr and the optimal frame rate 212 is within a predetermined range. If the difference between the actual frame rate fr and the optimal frame rate 212 is not within the predetermined range in step S1205, the control unit 202 returns to step S1108, as illustrated in Figure 11. On the other hand, if the difference between the actual frame rate fr and the optimal frame rate 212 is within the predetermined range in step S1205, the imaging unit 101 images the living body 102 for a predetermined time and acquires a moving image 214 using the settings of the imaging unit 101 determined in step S1107 or step S1204. Then, the control unit 202 proceeds to step S405, as illustrated in Figure 4.

[0094] Furthermore, depending on the processing time of the imaging unit 101 and the processing time of the pixel value calculation unit 204, the difference between the actual frame rate fr and the optimal frame rate 212 may not fall within a predetermined range. Therefore, the imaging control unit 1002 may adjust the exposure time beyond a predetermined number of times, and if the difference between the actual frame rate fr and the optimal frame rate 212 does not fall within a predetermined range when the imaging unit 101 images the living organism 102, it may change the optimal frame rate 212. For example, if the flicker frequency fq is 120 Hz, and the imaging control unit 1002 However, if the difference between the actual frame rate fr and the optimal frame rate 212 is not within a predetermined range when the optimal frame rate 212 is determined to be 60fps, the imaging control unit 1002 The optimal frame rate of 212 may be determined to be 40fps.

[0095] Similarly, the imaging control unit 1002 may change the optimal frame rate 212 if, after determining the exposure time to the lower limit exposure time ETMIN and the imaging unit 101 images the living organism 102, the difference between the actual frame rate fr and the optimal frame rate 212 is not within a predetermined range.

[0096] As described above, the measuring device 100 according to this embodiment adjusts the exposure time to approach the optimal frame rate 212 according to the actual frame rate fr. As a result, even if the actual frame rate fr is lower than the optimal frame rate 212 due to the processing time of the imaging unit 101 and the processing time of the pixel value calculation unit 204, the measuring device 100 according to this embodiment can separate the noise due to flicker included in the pulse wave signal 216 from the frequency components caused by the time changes of the living body included in the pulse wave signal 216.

[0097] (Modification 1 of the third embodiment) As a modification 1 of the measuring device 100 according to this embodiment, the imaging control unit 1002 may determine the optimal frame rate 212 according to a predetermined calculation formula based on the flicker frequency fq. For example, the imaging control unit 1002 may determine the optimal frame rate 212 as the value obtained by dividing the flicker frequency fq by an integer N. For example, if the flicker frequency fq is 120 Hz, the imaging control unit 1002 may determine 60 fps (= 120 / 2) as the optimal frame rate 212. Alternatively, for example, if the flicker frequency fq is 100 Hz, the imaging control unit 1002 may determine 50 fps (= 100 / 2) as the optimal frame rate 212.

[0098] (Modification 2 of the third embodiment) As a second modification of the measuring device 100 according to this embodiment, if the imaging control unit 1002 can set an upper limit for the frame rate, it may set the upper limit for the frame rate to the optimal frame rate 212 determined in step S1103 or step S1106. This allows the imaging control unit 1002 to prevent the actual frame rate fr from exceeding the optimal frame rate 212. As a result, the imaging control unit 1002 can easily keep the difference between the actual frame rate fr and the optimal frame rate 212 within a predetermined range.

[0099] Furthermore, if the measuring device 100 is not capable of setting an upper limit for the frame rate, and the actual frame rate fr exceeds the optimal frame rate 212, the imaging control unit will... 1002 The system may also be adjusted so that the actual frame rate fr decreases by making the new exposure time ET2 longer than the exposure time ET1.

[0100] Furthermore, the longer the exposure time, the larger the pixel value and the relatively higher the signal-to-noise ratio. Therefore, by relatively increasing the exposure time, the signal-to-noise ratio becomes relatively higher with respect to the frequency components caused by the time change of the biological tissue 102 contained in the pulse wave signal 216. Here, if the difference between the actual frame rate fr and the optimal frame rate 212 is within a predetermined range, the imaging control unit...1002 However, even if the exposure time is relatively increased, the actual frame rate fr may not change. Therefore, the imaging control unit 1002 may adjust the exposure time to be relatively longer if the difference between the actual frame rate fr and the optimal frame rate 212 is within a predetermined range.

[0101] However, if the exposure time is made relatively long, the pixel values ​​may become too large and saturate. Therefore, the imaging control unit 1002 may set an upper limit representative value, which is the upper limit of the representative value 215, in advance. In that case, the imaging control unit 1002 may adjust the exposure time to be relatively longer if the difference between the actual frame rate fr and the optimal frame rate 212 is within a predetermined range and the representative value 215 is smaller than the upper limit representative value.

[0102] (Modification 3 of the third embodiment) As a third modification of the measuring device 100 according to this embodiment, the measuring device 100 may perform in parallel the process in which the pulse wave calculation unit 205 calculates the pulse wave signal 216 and the process in which the imaging unit 101 images the living body 102 and acquires a moving image 214. In other words, the control unit 202 may perform in parallel the processes from steps S405 to S407 illustrated in Figure 4 and the processes from steps S1108 illustrated in Figure 11 to S1206 illustrated in Figure 12. As a result, the measuring device 100 according to this modification can readjust the exposure time even if, after adjusting the exposure time, the lighting environment changes and it becomes impossible to remove the frequency components caused by flicker and frame rate contained in the pulse wave signal 216.

[0103] Furthermore, the imaging control unit 1002 may adjust the exposure time at intervals longer than a predetermined time interval. For example, if the imaging control unit 1002 has adjusted the exposure time in step S1107 or step S1204 as illustrated in Figure 11, and the preset minimum switching time has not elapsed, it may maintain the exposure time even if the difference between the actual frame rate fr and the optimal frame rate 212 is not within a predetermined range. Then, if the difference between the actual frame rate fr and the optimal frame rate 212 is not within a predetermined range after the minimum switching time has elapsed since the imaging control unit 1002 adjusted the exposure time in step S1107 or step S1204, the control unit 202 may return to step S1108 as illustrated in Figure 11. This prevents the imaging control unit 1002 from frequently changing the exposure time when the difference between the actual frame rate fr and the optimal frame rate 212 is not within a predetermined range.

[0104] (Modification 4 of the third embodiment) As a fourth modification of the measuring device 100 according to this embodiment, flicker-related information 211 may be stored in the storage unit 201. Figure 13 is a block diagram showing an example of the configuration of the measuring device 100 according to this modification. The difference between the measuring device 100 illustrated in Figure 10 and the measuring device 100 illustrated in Figure 13 is that the measuring device 100 illustrated in Figure 13 does not have a flicker detection unit 501, and flicker-related information 211 is stored in the storage unit 201.

[0105] The imaging control unit 1002 in this modified example determines the optimal frame rate 212 based on the presence or absence of flicker indicated by the flicker-related information 211, or the flicker frequency fq indicated by the flicker-related information 211. As a result, the measuring device 100 in this modified example can control the exposure time so that the actual frame rate fr approaches the optimal frame rate based on the flicker frequency fq set according to the illumination.

[0106] This disclosure is not limited to the embodiments described above, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. Furthermore, new technical features can be formed by combining the technical means disclosed in each embodiment. [Explanation of symbols]

[0107] 100 Measuring device, 101 Imaging unit, 102 Biological body, 201 Storage unit, 202 Control unit, 203 Imaging control unit, 204 Pixel value calculation unit, 205 Pulse wave calculation unit, 206 Vital indicator calculation unit, 211 Flicker-related information, 212 Optimal frame rate, 213 Imaging control information, 214 Moving image, 215 Representative value, 216 Pulse wave signal, 402 Imaging control unit, 501 Flicker detection unit, 511 Frame rate table, 1001 Imaging interval calculation unit, 1002 Imaging control unit, 1011 Imaging control information

Claims

1. An imaging unit that captures images of living organisms and acquires moving images, A pixel value calculation unit calculates representative pixel values ​​of a region of interest, including the image of the living organism, from each image constituting the aforementioned moving image. A pulse wave calculation unit that calculates a pulse wave signal from the time change of the aforementioned representative value, Equipped with, The imaging unit captures the imaging range of the imaging unit at each of the multiple frame rates. A flicker detection unit determines that flicker has been detected if, among the plurality of frame rates, the timing periods of the peaks of the time changes in the brightness values ​​calculated from the pixel values ​​included in the image captured by the imaging unit are different at different frame rates. When the flicker is detected by the flicker detection unit, the imaging control unit detects the frequency of the flicker and determines the optimal frame rate such that the absolute value of the difference between the detected flicker frequency and an integer multiple of the optimal frame rate is greater than or equal to a threshold. Furthermore, The imaging unit is a measuring device that images the living organism at the optimal frame rate.

2. The imaging control unit controls the settings of the imaging unit according to the difference between the actual frame rate calculated from the time difference in which each image was acquired and the optimal frame rate. The measuring device according to claim 1.

3. Controlling the aforementioned settings includes adjusting the exposure time. The measuring device according to claim 2.

4. The imaging control unit adjusts the exposure time at intervals of a predetermined time or longer. The measuring device according to claim 3.

5. The imaging control unit controls the settings so that the difference between the actual frame rate and the optimal frame rate falls within a predetermined range. The measuring device according to any one of claims 2 to 4.

6. The imaging control unit determines a predetermined frame rate as the optimal frame rate if there is no flicker. The measuring device according to any one of claims 1 to 5.

7. The imaging control unit determines the optimal frame rate as a value associated with the type of vital indicator being measured, in the absence of flicker. The measuring device according to any one of claims 1 to 5.

8. The process of imaging a living organism and acquiring a moving image, A step of calculating representative pixel values ​​of the region of interest, which includes the image of the living organism, from each image constituting the aforementioned moving image, A step of calculating a pulse wave signal from the time change of the aforementioned representative value, Includes, In the process of acquiring the aforementioned moving image, the imaging range is captured at each of the multiple frame rates. If, among the plurality of frame rates, the timing period of the peak of the time change in the brightness value calculated from the pixel value included in the image captured in the process of acquiring the moving image is different at different frame rates, then it is determined that flicker has been detected. When the flicker is detected, the steps include: detecting the frequency of the flicker, and determining the optimal frame rate such that the absolute value of the difference between the detected flicker frequency and an integer multiple of the optimal frame rate is greater than or equal to a threshold; It further includes, A measurement method wherein, in the step of acquiring the aforementioned moving image, the living organism is imaged at the optimal frame rate.

9. On the computer, A function that captures images of living organisms and acquires moving images, A function to calculate representative pixel values ​​of the region of interest, including the image of the living organism, from each image constituting the aforementioned moving image, A function to calculate a pulse wave signal from the time change of the aforementioned representative value, Make it run, In the function for acquiring the aforementioned moving image, the imaging range is captured at each of the multiple frame rates. A function that determines that flicker has been detected when, among the plurality of frame rates, the timing period of the peak of the time change in the brightness value calculated from the pixel value included in the image captured by the function that acquires the moving image is different at different frame rates, A function that, when the flicker is detected, detects the frequency of the flicker and determines the optimal frame rate such that the absolute value of the difference between the detected flicker frequency and an integer multiple of the optimal frame rate is greater than or equal to a threshold, Let's execute it further, A program for acquiring the aforementioned moving image, wherein the living organism is imaged at the optimal frame rate.

Images