Measurement device, respiratory rate measurement method, and program

Abstract

To provide a measuring device capable of outputting a respiratory rate by guaranteeing reliability, while suppressing a measuring time.SOLUTION: A measuring device includes: an imaging part for acquiring a dynamic image by imaging an organism; a respiratory rate calculation part for calculating a respiratory rate by detecting a vibration cycle of a respiratory signal included in a time change of multiple pixel values included in the dynamic image; a reliability calculation part for calculating reliability of the respiratory rate from a comparison result between the respiratory signal and a noise signal included in the time change; and an output part for outputting the respiratory rate, when the reliability is equal to or more than a threshold.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a measurement device, a respiratory rate measurement method, and a program. [Background technology]

[0002] Patent Document 1 discloses a technique for measuring the respiratory rate of a person being measured by detecting periodic vibrations that can be considered to represent breathing from a moving image of the person being measured. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-171574 Summary of the Invention [Problem to be solved by the invention]

[0004] When conducting screening tests to check health status by measuring respiratory rate at stores, event venues, etc., the shorter the measurement time required to measure each person's respiratory rate, the more efficiently it will be possible to measure the respiratory rates of a large number of people.

[0005] The technology disclosed in Patent Document 1 requires changing the time window for calculating the power spectrum according to the minimum respiratory rate in the measurement range in order to detect periodic vibrations that can be considered respiration from the captured video of the subject. Furthermore, the technology disclosed in Patent Document 1 requires a larger time window for calculating the power spectrum as the minimum respiratory rate in the measurement range decreases. As a result, the technology disclosed in Patent Document 1 may require a longer measurement time to measure the respiratory rate as the minimum respiratory rate in the measurement range decreases. Furthermore, the technology disclosed in Patent Document 1 may calculate an incorrect respiratory rate due to the influence of noise contained in the video. Therefore, one aspect of the present disclosure aims to provide a measurement device, a respiratory rate measurement method, and a program that can output a respiratory rate with guaranteed reliability while reducing measurement time. [Means for solving the problem]

[0006] A measurement device according to one aspect of the present disclosure includes an imaging unit that captures an image of a living body to acquire a moving image, a respiratory rate calculation unit that calculates the respiratory rate by detecting the oscillation period of a respiratory signal contained in the time changes of multiple pixel values ​​contained in the moving image, a reliability calculation unit that calculates the reliability of the respiratory rate from the result of comparing the respiratory signal with a noise signal contained in the time changes, and an output unit that outputs the respiratory rate if the reliability is equal to or greater than a threshold.

[0007] A respiratory rate measurement method according to one aspect of the present disclosure includes the steps of capturing an image of a living body to obtain a moving image, calculating the respiratory rate by detecting the oscillation period of a respiratory signal contained in the time changes of a plurality of pixel values ​​contained in the moving image, calculating the reliability of the respiratory rate from the result of comparing the respiratory signal with a noise signal contained in the time changes, and outputting the respiratory rate if the reliability is equal to or greater than a threshold value.

[0008] A program according to one aspect of the present disclosure causes a computer to perform the following functions: capturing an image of a living body to obtain a moving image; calculating a respiratory rate by detecting the vibration period of a respiratory signal contained in the time changes of multiple pixel values ​​contained in the moving image; calculating the reliability of the respiratory rate from the results of comparing the respiratory signal with a noise signal contained in the time changes; and outputting the respiratory rate if the reliability is equal to or greater than a threshold value. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing an example of how the measurement device is used. [Figure 2] 1 is a block diagram showing an example of the configuration of a measurement device according to a first embodiment. [Figure 3] 4 is a flowchart showing an example of the operation of the measurement device according to the first embodiment. [Figure 4] 1 shows an example of images constituting a moving image. [Figure 5A]10 is a graph showing an example of a time-series signal indicating a change in moving speed over time. [Figure 5B] 5B is a graph showing an example of a change over time in the autocorrelation value of the time-series signal illustrated in FIG. 5A. [Figure 6A] 10 is a graph showing an example of a change over time in an autocorrelation value for a time-series signal indicating a change over time in a moving speed. [Figure 6B] 6B is a graph showing an example of the power spectrum of the time-series signal illustrated in FIG. 6A. [Figure 7] 4 is a flowchart illustrating an example of the operation of the measurement device according to the first embodiment, following FIG. 3. [Figure 8] 10A and 10B are diagrams illustrating an example of a time-series signal generated from moving images acquired by capturing images of a living body at different measurement times, and a power spectrum. [Figure 9] FIG. 10 is a block diagram showing an example of the configuration of a measurement device according to a second embodiment. [Figure 10] 10 is a flowchart showing an example of the operation of the measurement device according to the second embodiment. [Figure 11] FIG. 10 is a block diagram showing an example of the configuration of a measurement device according to a third embodiment. [Figure 12] 10 is a flowchart showing an example of the operation of the measurement device according to the third embodiment. [Figure 13] FIG. 1 is a diagram showing an example of an image including images of multiple faces. DETAILED DESCRIPTION OF THE INVENTION

[0010] (First embodiment) The first embodiment will be described with reference to Figures 1 to 8. In the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant explanations will be omitted.

[0011] 1 is a diagram showing an example of a usage mode of a measuring device 100 according to this embodiment. The measuring device 100 includes an imaging unit 101. The measuring device 100 calculates the respiratory rate from an image acquired by the imaging unit 101. For example, the measuring device 100 is a PC (Personal Computer), a smartphone, a tablet terminal, a terminal dedicated to measuring the respiratory rate, etc.

[0012] The imaging unit 101 captures an image of the living body 102 to obtain a moving image 211 (see FIG. 2). The moving image 211 includes an image of the face and chest of the living body 102. For example, the imaging unit 101 is configured with a CCD (Charged Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) image sensor. The imaging unit 101 may also be configured with an image sensor for a camera including RGB (Red Green Blue) filters.

[0013] 2 is a block diagram showing an example of the configuration of the measurement device 100. The measurement device 100 includes an imaging unit 101, a storage unit 201, and a control unit 202.

[0014] The image capturing unit 101 captures an image of the living body 102 during the measurement time to obtain a moving image 211 .

[0015] The storage unit 201 is a recording medium capable of recording various data, programs, etc., and is configured by, for example, a hard disk, an SSD (Solid State Drive), a semiconductor memory, etc.

[0016] The control unit 202 executes various processes in accordance with the programs and data stored in the storage unit 201. The control unit 202 is realized by a processor such as a CPU (Central Processing Unit), for example.

[0017] The control unit 202 includes a region specifying unit 203 , a signal generating unit 204 , a respiratory rate calculating unit 205 , a reliability calculating unit 206 , a determining unit 207 , an output unit 208 , and a measurement time setting unit 209 .

[0018] The region specifying unit 203 specifies a region 212 including an image of the chest of the living body 102 from each image constituting the moving image 211 .

[0019] The signal generating unit 204 generates a time-series signal 213 that indicates changes over time in multiple pixel values ​​included in a moving image 211. The multiple pixel values ​​are pixel values ​​within an area 212 that includes an image of the chest of the living body 102. The time-series signal 213 includes a respiratory signal 213a and a noise signal 213b. The respiratory signal 213a indicates changes over time in pixel values ​​due to respiratory movement of the living body 102. The noise signal 213b indicates changes over time in pixel values ​​due to noise caused by lighting or the like.

[0020] The respiratory rate calculation unit 205 calculates the respiratory rate 214 by detecting the oscillation period of a respiratory signal 213 a included in the time-varying changes in the values ​​of a plurality of pixels included in the moving image 211 .

[0021] The reliability calculation unit 206 calculates the reliability 215 of the respiratory rate 214 from the result of comparing the respiratory signal 213a with the noise signal 213b.

[0022] The determination unit 207 determines whether the reliability 215 is equal to or greater than a threshold value.

[0023] The output unit 208 outputs the respiratory rate 213 when the reliability 215 is equal to or greater than a threshold value.

[0024] The measurement time setting unit 209 sets the measurement time. Furthermore, if the reliability 215 is lower than the threshold, the measurement time setting unit 209 extends the measurement time.

[0025] 3 is a flowchart showing an example of the operation of the measurement device 100 according to this embodiment. In this example, when the image capturing unit 101 is activated, the control unit 202 starts the process of step S301 illustrated in FIG. 3. The image capturing range of the image capturing unit 101 is assumed to include the face and chest of the living body 102. Furthermore, at the time of starting the process of step S301, the measurement time setting unit 209 is assumed to have set the measurement time as an initial value. For example, the initial value of the measurement time is 3 seconds.

[0026] In step S301, the imaging unit 301 captures an image of the living body 102 for the measurement time set by the measurement time setting unit 209, acquires a moving image 211, and stores the acquired moving image 211 in the storage unit 201. For example, the imaging unit 101 captures an image of the living body 102 for the measurement time at 30 to 60 fps (frames per second) and acquires the moving image 211. The moving image 211 includes an image of the face of the living body 102 and an image of the chest.

[0027] In step S302, the region identification unit 203 determines whether or not a facial image of the living body 102 can be detected from each image constituting the video 211. Each image constituting the video 211 is an image of each frame constituting the video 211. For example, the region identification unit 203 detects feature points such as the eyes, nose, and mouth from each image constituting the video 211, and detects a facial image based on the detected feature points. For example, the region identification unit 203 detects a facial image using a method such as a cascade detector, a support vector machine, or a convolutional neural network (CNN).

[0028] If a face image is not detected in step S302, the control unit 202 returns the process to step S301. That is, the imaging unit 301 repeats capturing images of the living body 102 for the measurement time until a face image of the living body 102 is detected from each image constituting the moving image 211.

[0029] If a face image is detected in step S302, in step S303, the region specifying unit 203 specifies a region 212 including an image of the chest of the living body 102 based on the position of the face image detected in step S302. For example, the region 212 is a region of a predetermined size located below the position of the region including the face image in the +Y direction (see FIG. 4).

[0030] FIG. 4 shows an example of an image 401 constituting a moving image 211. The region 212 shown in FIG. 4 is a region of a predetermined size located below the position of the face image 402 included in the image 401. For example, the region of the predetermined size is a rectangular region whose length in the vertical Y direction is the same as that of the region including the face image 402, and whose length in the horizontal X direction is twice that of the region including the face image 402. For example, as shown in FIG. 4, the region 212 includes an image of the chest of the living body 102. Note that the region 212 only needs to include a portion of the chest of the living body 102, and does not have to include the entire chest of the living body 102.

[0031] When the living organism 102 inhales, the rib cage expands. When the living organism 102 exhales, the rib cage contracts. In other words, when the living organism 102 breathes, the living organism 102 performs respiratory movement, which moves the chest and shoulders up and down. Therefore, when the imaging unit 101 captures an image of the living organism 102 and acquires a moving image 211, the pixel values ​​of an area 212 included in the moving image 211 change over time due to respiratory movement.

[0032] Therefore, in the next step S304, the signal generation unit 204 generates a time-series signal 213 that indicates a change over time in the pixel values ​​of a plurality of pixels in the region 212. The change over time is indicated by a change in the values ​​of a plurality of pixels included in the video 211, or a change in a velocity vector determined from the values ​​of a plurality of pixels included in the video 211.

[0033] For example, the signal generating unit 204 calculates a representative value of the pixel values ​​of any of R (red), G (green), and B (blue) pixels in the region 212 for each image constituting the moving image 211. The representative value may be an average value, a median value, a mode value, or the like.

[0034] Alternatively, the signal generating unit 204 may calculate, for each image constituting the moving image 211, a representative value by multiplying the average value of the pixel values ​​of each R, G, and B pixel in the region 212 by a weighting coefficient and adding the resulting values.

[0035] The signal generating unit 204 then generates a time-series signal 213 indicating a change over time in the calculated representative value. Alternatively, the signal generating unit 204 may perform noise suppression processing on a signal indicating a change over time in the calculated representative value for each predetermined time, and generate the noise-suppressed signal as the time-series signal 213. For example, the signal generating unit 204 may input the signal indicating a change over time in the calculated representative value for each predetermined time to an autocorrelation function to calculate an autocorrelation value for each predetermined time. The signal generating unit 204 may then generate a signal indicating a change over time in the autocorrelation value as the time-series signal 213. Alternatively, the signal generating unit 204 may input the signal indicating a change over time in the calculated representative value for each predetermined time to a low-pass filter that extracts frequency components equal to or lower than a predetermined frequency, thereby generating the time-series signal 213. In this way, the signal generating unit 204 can generate a time-series signal 213 from which frequency components exceeding the predetermined frequency have been removed.

[0036] Alternatively, the signal generating unit 204 may calculate a vector V(i) indicating the movement speed in the Y direction for each image constituting the moving image 211. The vector V(i) indicates the movement speed in the Y direction from the time when the image of frame i-1 constituting the moving image 211 is captured to the time when the image of frame i is captured.

[0037] For example, the signal generation unit 204 calculates a vector V(i) indicating the movement speed in the Y direction for a plurality of pixels in the region 212 included in the image of frame i constituting the moving image 211, using a method such as a block matching method or a gradient method. x,y Vector V(i,Pix x,y ) is calculated as a vector V(i) relating to the image of frame i constituting the video 211. x,yindicates a pixel whose coordinate values ​​are x and y and is included in the region 212. x,y ) is the pixel Pix in the area 212 included in the image of the frame i. x,y For example, the vector V(i,Pix x,y ) is the representative value of all pixels Pix x,y Vector V(i,Pix x,y ) is the average value.

[0038] Then, the signal generating unit 204 generates a time-series signal 213 indicating a time change of the vector V(i). Alternatively, the signal generating unit 204 may perform noise suppression processing on the signal indicating a time change of the vector V(i) and generate the noise-suppressed signal as the time-series signal 213.

[0039] For example, the signal generating unit 204 may input a signal indicating a time change in the moving speed in the Y direction for each predetermined time into an autocorrelation function to calculate an autocorrelation value for each predetermined time. Then, the signal generating unit 204 may generate a signal indicating a time change in the autocorrelation value as the time-series signal 213. Alternatively, the signal generating unit 204 may generate the time-series signal 213 by inputting the signal indicating a time change in the moving speed in the Y direction into a low-pass filter that extracts frequency components equal to or lower than a predetermined frequency.

[0040] 5A is a graph showing an example of a time-series signal indicating a change in moving speed over time. In FIG. 5A, the horizontal axis represents time, and the vertical axis represents a signal value indicating moving speed. Time-series signal 501 is time-series signal 213 indicating the moving speed in the Y direction between frames included in moving image 211.

[0041] Fig. 5B is a graph showing an example of the change over time in the autocorrelation value for the time-series signal 501 shown in Fig. 5A. In Fig. 5B, the horizontal axis represents time, and the vertical axis represents the autocorrelation value. Time-series signal 502 is time-series signal 213 that indicates the change over time in the autocorrelation value for the time-series signal 501 shown in Fig. 5A. As shown in Fig. 5B, noise contained in time-series signal 501 shown in Fig. 5A is suppressed and smoothed in time-series signal 502.

[0042] In the following step S305, the signal generating unit 204 performs frequency analysis on the time-series signal 213 and calculates a power spectrum of the time-series signal 213.

[0043] In step S306, the respiratory rate calculation unit 205 calculates the respiratory rate 214 from the oscillation period of the respiratory signal 213a included in the time-series signal 213. Specifically, the respiratory rate calculation unit 205 calculates the respiratory rate 214 from the maximum peak frequency in the power spectrum calculated in step S305. More specifically, the respiratory rate calculation unit 205 calculates the respiratory rate by considering the maximum peak frequency within a target frequency range in the power spectrum calculated in step S305 as the respiratory frequency. For example, the target frequency range is a frequency range of 2 to 200 breaths per minute.

[0044] In step S307, the reliability calculation unit 206 calculates the magnitude of the respiratory signal 213a from the time-series signal 213. For example, the magnitude of the respiratory signal 213a is the power spectrum value of the maximum peak frequency in the time-series signal 213. In other words, the magnitude of the respiratory signal 213a is the power spectrum value of the maximum peak frequency within the target frequency range in the power spectrum calculated in step S306.

[0045] Alternatively, the magnitude of the respiratory signal 213a may be a statistical value of power spectrum values ​​within a predetermined range including the maximum peak frequency in the time-series signal 213. For example, the predetermined range is a range from a frequency 0.1 Hz lower than the maximum peak frequency to a frequency 0.1 Hz higher than the maximum peak frequency. For example, the statistical value of power spectrum values ​​within the predetermined range is the sum of the power spectrum values ​​within the predetermined range.

[0046] In step S308, the reliability calculation unit 206 calculates the magnitude of the noise signal 213b from the time-series signal 213. For example, the magnitude of the noise signal 213b is the power spectrum value of the second or lower peak frequency in the time-series signal 213. In other words, the magnitude of the noise signal 213b is the power spectrum value of the second or lower peak frequency in the power spectrum values ​​calculated in step S306.

[0047] For example, the magnitude of the noise signal 213b is the power spectrum value of the second largest peak frequency in the time-series signal 213. Alternatively, the magnitude of the noise signal 213b may be the power spectrum value showing the third largest peak in the time-series signal 213.

[0048] Alternatively, if the magnitude of the respiratory signal 213a is a statistical value of power spectrum values ​​within a predetermined range including the maximum peak frequency in the time-series signal 213, the magnitude of the noise signal 213b may be a statistical value of power spectrum values ​​outside the predetermined range. For example, the statistical value of power spectrum values ​​outside the predetermined range is the sum of power spectrum values ​​outside the predetermined range. Note that the reliability calculation unit 206 may calculate the magnitude of the noise signal 213b by combining a plurality of calculation methods.

[0049] In step S309, the reliability calculation unit 206 calculates the reliability 215 from the ratio between the magnitude of the respiratory signal 213a calculated in step S307 and the magnitude of the noise signal 213b calculated in step S308. For example, the reliability calculation unit 206 calculates the ratio between the magnitude of the respiratory signal 213a and the magnitude of the noise signal 213b as the reliability 215. The reliability 215 indicates a signal-to-noise ratio (SNR). In other words, a larger value of the reliability 215 indicates that, in the temporal changes in the pixel values ​​included in the moving image 211 indicated by the time-series signal 213, the influence of the temporal changes in the pixel values ​​due to the respiratory movement of the living body 102 is greater than the influence of the temporal changes in the pixel values ​​due to noise. Then, the control unit 202 proceeds to step S701 illustrated in FIG. 7.

[0050] 6A is a graph showing an example of the time change in the autocorrelation value of a time-series signal indicating the time change in moving speed. In FIG. 6A, the horizontal axis represents time, and the vertical axis represents the autocorrelation value. Time-series signal 601 is time-series signal 213 indicating the autocorrelation value of the moving speed in the Y direction for each predetermined time period for the same position within region 212.

[0051] Fig. 6B is a graph showing an example of the power spectrum of the time-series signal 601 shown in Fig. 6A. In Fig. 6B, the horizontal axis represents frequency, and the vertical axis represents power spectrum values. A power spectrum 602 shows the power spectrum of the time-series signal 601. A power spectrum value P611 is the magnitude of the power spectrum of the maximum peak frequency in the time-series signal 601. In other words, the power spectrum value P611 indicates the magnitude of the respiratory signal 213a.

[0052] The power spectrum value P612 is the magnitude of the second largest power spectrum in the time-series signal 213 illustrated in Fig. 6A. In other words, the power spectrum value P612 indicates the magnitude of the noise signal 213b.

[0053] The reliability calculation unit 206 calculates the reliability 215 from the ratio between the power spectrum value P611 and the power spectrum value P612. For example, the reliability calculation unit 206 calculates the ratio between the power spectrum value P611 and the power spectrum value P612 as the reliability 215.

[0054] Next, the operation of the measurement device 100 will be further described with reference to FIG.

[0055] In step S701, the determination unit 207 determines whether the reliability 215 is equal to or greater than a threshold. The reliability 215 being equal to or greater than the threshold indicates that, in the temporal changes in the pixel values ​​included in the moving image 211 indicated by the time-series signal 213, the influence of the temporal changes in the pixel values ​​caused by the respiratory movement of the living organism 102 is sufficiently greater than the influence of the temporal changes in the pixel values ​​caused by noise. On the other hand, if the reliability 215 is lower than the threshold, there is a risk that, in the temporal changes in the pixel values ​​included in the moving image 211 indicated by the time-series signal 213, the influence of the temporal changes in the pixel values ​​caused by noise is relatively greater.

[0056] If the reliability 215 is lower than the threshold in step S701, the measurement time setting unit 209 extends the measurement time by a predetermined time in step S702. For example, the predetermined time is the time it takes for the imaging unit 101 to acquire one frame of image. Then, the control unit 202 returns the process to step S301 illustrated in Fig. 3. That is, the measurement time setting unit 209 extends the measurement time, continues imaging the living body 102, and repeatedly calculates the respiratory rate 214 until the reliability 215 becomes equal to or greater than the threshold.

[0057] On the other hand, if the reliability 215 is equal to or greater than the threshold in step S701, the output unit 208 outputs the respiratory rate 214 calculated in step S306 illustrated in Fig. 3 in step S703. For example, the output unit 208 displays text indicating the respiratory rate 214 on a liquid crystal display or the like. Alternatively, for example, the output unit 208 may output a sound indicating the respiratory rate 214 from a speaker. Then, the control unit 202 ends the process of measuring the respiratory rate 214.

[0058] That is, when the influence of the time-varying pixel value change due to the respiratory movement of the living body 102 in the time-varying pixel values ​​included in the moving image 211, which is indicated by the time-series signal 213, is sufficiently greater than the influence of the time-varying pixel value change due to noise, the output unit 208 outputs the respiratory rate 214. This allows the measuring device 100 according to this embodiment to guarantee the reliability of the output respiratory rate 214.

[0059] Fig. 8 shows an example of time-series signals and power spectra generated from moving images 211 acquired by capturing images of a living body 102 at different measurement times. In Fig. 8, for time-series signals 801, 803, and 805, the horizontal axis represents time and the vertical axis represents autocorrelation values. Also, in Fig. 8, for power spectrum 802, 804, and 806, the horizontal axis represents frequency and the vertical axis represents power spectrum values.

[0060] A time-series signal 801 represents a time-series signal 213 generated from a video image 211 acquired at a measurement time T1. A power spectrum 802 represents the power spectrum of the time-series signal 801.

[0061] For example, suppose that in step S307 the reliability calculation unit 206 calculates the power spectrum value P811 of the maximum peak frequency in the time-series signal 801 as the magnitude of the respiratory signal 213a. Furthermore, suppose that in step S308 the reliability calculation unit 206 calculates the power spectrum value P812 of the second peak frequency in the time-series signal 801 as the magnitude of the noise signal 213b. In this case, in step S309 the reliability calculation unit 206 calculates the reliability 215 from the ratio of the power spectrum value P811 to the power spectrum value P812. If the reliability 215 is lower than the threshold, in step S702 the measurement time setting unit 209 extends the measurement time to time T2, which is longer than time T1.

[0062] A time-series signal 803 represents the time-series signal 213 generated from the video image 211 acquired at the measurement time T2. A power spectrum 804 represents the power spectrum of the time-series signal 803.

[0063] For example, in step S307, the reliability calculation unit 206 calculates the power spectrum value P813 of the maximum peak frequency in the time-series signal 803 as the magnitude of the respiratory signal 213a. Furthermore, in step S308, the reliability calculation unit 206 calculates the power spectrum value P814 of the second peak frequency in the time-series signal 803 as the magnitude of the noise signal 213b. In this case, in step S309, the reliability calculation unit 206 calculates the reliability 215 from the ratio of the power spectrum value P813 to the power spectrum value P814. If the reliability 215 is lower than the threshold, in step S702, the measurement time setting unit 209 extends the measurement time to time T3, which is longer than time T2.

[0064] A time-series signal 805 represents the time-series signal 213 generated from the video image 211 acquired at the measurement time T3. A power spectrum 806 represents the power spectrum of the time-series signal 805.

[0065] For example, in step S307, the reliability calculation unit 206 calculates the power spectrum value P815 of the maximum peak frequency in the time-series signal 805 as the magnitude of the respiratory signal 213a. Furthermore, in step S308, the reliability calculation unit 206 calculates the power spectrum value P816 of the second peak frequency in the time-series signal 805 as the magnitude of the noise signal 213b. In that case, in step S309, the reliability calculation unit 206 calculates the reliability 215 from the ratio between the power spectrum value P815 and the power spectrum value P816.

[0066] If the reliability 215 is equal to or greater than the threshold, in step S703 the output unit 208 outputs the respiratory rate 214 calculated from the maximum peak frequency PEFQ in the time-series signal 805. Then, the control unit 202 ends the process of measuring the respiratory rate 214. That is, the measurement device 100 changes the measurement time according to the reliability 215, and outputs the respiratory rate 214 when the reliability 215 is equal to or greater than the threshold. In this way, the measurement device 100 can output the respiratory rate 214 with guaranteed reliability while reducing the measurement time.

[0067] (Modification 1 of the first embodiment) As a first modification of the measuring device 100 according to the present embodiment, when the measurement time setting unit 209 extends the measurement time, the control unit 202 may execute the processes of steps S305 to S309 and the process of step S701 each time a predetermined number of frames of images (two or more frames) are acquired. That is, when the measurement time setting unit 209 extends the measurement time, each time a predetermined number of frames of images are acquired, the respiratory rate calculation unit 205 may calculate the respiratory rate 214, the reliability calculation unit 206 may calculate the reliability 215, and the determination unit 207 may determine whether the reliability 215 is equal to or greater than a threshold. In this way, when the measurement time setting unit 209 extends the measurement time, the measuring device 100 according to this first modification can reduce the processing time compared to when the respiratory rate 214 and the reliability 215 are calculated each time one frame of images is acquired.

[0068] (Modification 2 of the first embodiment) As a second modification of the measuring device 100 according to this embodiment, the measurement time setting unit 209 may set at least one selected from the group consisting of a lower limit and an upper limit of the measurement time. For example, the lower limit of the measurement time is 3 seconds. Also, for example, the upper limit of the measurement time is 20 seconds.

[0069] Assume that the measurement time setting unit 209 sets an upper limit for the measurement time. In this case, if the imaging unit 101 continues to capture images of the living body 102 for the measurement time that is the upper limit, and the reliability 215 does not become equal to or greater than the threshold, the control unit 202 terminates the process of measuring the respiratory rate 214. Furthermore, the output unit 208 outputs a message indicating an error. For example, the output unit 208 displays a message such as "The respiratory rate could not be measured. Please measure again" on a liquid crystal display or the like. Alternatively, the output unit 208 may output a sound indicating the message from a speaker.

[0070] Second Embodiment The second embodiment will be described with reference to Figures 9 and 10. In the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant explanations will be omitted. Configurations and processes having substantially the same functions as those of other embodiments will be denoted by the same reference numerals, and explanations will be omitted, and differences from other embodiments will be described.

[0071] Fig. 9 is a block diagram showing an example of the configuration of the measurement device 100 according to this embodiment. The difference between the measurement device 100 shown in Fig. 9 and the measurement device 100 shown in Fig. 2 is that the measurement device 100 shown in Fig. 9 includes a measurement time setting unit 901 instead of the measurement time setting unit 209.

[0072] The measurement time setting unit 901 sets the lower limit and upper limit of the measurement time to the same value, that is, the measurement time setting unit 901 does not extend the measurement time.

[0073] If the reliability 215 is lower than the threshold value, the respiratory rate calculation section 205 according to this embodiment ends the process of calculating the respiratory rate 214.

[0074] If the reliability 215 is lower than the threshold, the output unit 208 according to this embodiment outputs a message indicating an error.

[0075] FIG. 10 is a flowchart showing an example of the operation of the measurement device 100 according to this embodiment.

[0076] In step S309 illustrated in FIG. 3, when the reliability calculation unit 206 calculates the reliability 215 from the ratio between the magnitude of the respiratory signal 213a and the magnitude of the noise signal 213b, in step S1001 the determination unit 207 determines whether the reliability 215 is equal to or greater than a threshold value.

[0077] If the reliability 215 is equal to or greater than the threshold in step S1001, the output unit 208 outputs the respiratory rate 214 calculated in step S306 in step S1002. The processes of steps S1001 and S1002 illustrated in Fig. 10 are similar to the processes of steps S701 and S703 illustrated in Fig. 7, and therefore detailed description thereof will be omitted.

[0078] On the other hand, if the reliability 215 is lower than the threshold in step S1001, the output unit 208 outputs a message indicating an error in step S1003. For example, the output unit 208 displays a message such as "The respiratory rate could not be measured. Please measure again" on a liquid crystal display or the like. Alternatively, the output unit 208 may output a sound indicating the message from a speaker.

[0079] In order for the measuring device 100 to measure the respiratory rate 214 of the living body 102, the living body 102 needs to remain within the imaging range of the imaging unit 101. Therefore, the longer the measurement time, the longer the living body 102 needs to remain within the imaging range of the imaging unit 101. However, the measuring device 100 according to this embodiment does not extend the measurement time, and outputs a message indicating an error if the reliability 215 is lower than the threshold. In this way, the measuring device 100 according to this embodiment can limit the time required to measure the respiratory rate by not extending the measurement time. Furthermore, the measuring device 100 according to this embodiment can reduce the burden on the living body 102 by not extending the measurement time.

[0080] (Third embodiment) The third embodiment will be described with reference to Figures 11 to 13. In the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant explanations will be omitted. Configurations and processes having substantially the same functions as those of other embodiments will be denoted by the same reference numerals, and explanations will be omitted, and differences from other embodiments will be described.

[0081] Fig. 11 is a block diagram showing an example of the configuration of the measurement device 100 according to this embodiment. The difference between the measurement device 100 shown in Fig. 11 and the measurement device 100 shown in Fig. 2 is that the measurement device 100 shown in Fig. 11 includes a distance calculation unit 1101.

[0082] When the moving image 211 includes images of multiple faces, the distance calculation unit 1101 calculates the distance 1111 between the images of the multiple faces.

[0083] The output unit 208 according to this embodiment outputs a message indicating an error when the minimum value of the distances 1111 between the images of a plurality of faces is equal to or less than a predetermined distance.

[0084] The respiratory rate calculation unit 205 according to this embodiment ends the process of calculating the respiratory rate 214 when the minimum value of the distance 1111 is equal to or less than the predetermined distance.

[0085] Fig. 12 is a flowchart showing an example of the operation of the measurement device 100 according to this embodiment. In this example, when the measurement device 100 is started up, the control unit 202 starts the process of step S301 shown in Fig. 12. Furthermore, at the time when the control unit 202 starts the process of step S301, the measurement time setting unit 209 is set to an initial value for the measurement time. The processes of steps S1201 to S1202 are the same as the processes of steps S301 to S302 shown in Fig. 3, and therefore detailed description thereof will be omitted.

[0086] If a face image is detected in step S1202, then in step S1203, the distance calculation unit 1101 determines whether or not multiple face images are included in the video 211. For example, if multiple face images are included in at least one image constituting the video 211, the distance calculation unit 1101 determines that the video 211 includes multiple face images.

[0087] If it is determined in step S1203 that the video 211 does not contain multiple facial images, the control unit 202 proceeds to step S303 shown in FIG.

[0088] On the other hand, if it is determined in step S1203 that multiple facial images are included in the video 211, the distance calculation unit 1101 calculates the distance 1111 between the multiple facial images in step S1204.

[0089] If the minimum value of the distance 1111 between the images of the plurality of faces is equal to or less than a predetermined distance, the region specifying unit 203 may erroneously specify a region including part of the regions 212 of the plurality of adjacent living bodies 102 as the region 212 of one living body 102. In this case, the signal generating unit 204 may erroneously generate a signal indicating the time change in pixel values ​​caused by the respiratory movement of the plurality of adjacent living bodies 102 as the time-series signal 213 for one living body 102.

[0090] Therefore, in step S1205, the distance calculation unit 1001 determines whether the minimum value of the distances 1111 between the images of a plurality of faces is equal to or less than a predetermined distance.

[0091] If the minimum value of the distances 1111 between the multiple face images is equal to or less than the predetermined distance in step S1205, the output unit 208 outputs a message indicating an error in step S1206. For example, the output unit 208 displays a message such as "Please keep an appropriate distance from the person next to you" on a liquid crystal display or the like. Alternatively, the output unit 208 may output a sound indicating the message from a speaker. Then, the control unit 202 ends the process of measuring the respiratory rate 214. In other words, if the minimum value of the distances 1111 between the multiple face images is equal to or less than the predetermined distance, the respiratory rate calculation unit 205 ends the process of calculating the respiratory rate 214.

[0092] On the other hand, if the minimum value of the distances 1111 between the multiple face images is longer than the predetermined distance in step S1205, the control unit 202 executes the processes of steps S303 to S309 illustrated in Fig. 3 for each image of the living body 102 included in the moving image 211. Then, in step S701 illustrated in Fig. 7, the control unit 202 determines whether the reliability 215 for each image of the living body 102 included in the moving image 211 is equal to or greater than a threshold.

[0093] For an image of a living organism 102 whose reliability 215 is equal to or greater than the threshold, the output unit 208 outputs the respiratory rate 214 in step S703 illustrated in Fig. 7. For an image of a living organism 102 whose reliability 215 is lower than the threshold, the measurement time setting unit 209 extends the measurement time in step S702 illustrated in Fig. 7. Then, the control unit 202 returns the process to step S301 illustrated in Fig. 3. That is, the imaging unit 101 continues to capture images of living organisms 102 whose reliability 215 is lower than the threshold, and the control unit 202 continues to measure the respiratory rate 214 for the living organism 102.

[0094] 13 is a diagram showing an example of an image 1310 including face images 1301 to 1306. Distance 1111a is the distance between face image 1301 and face image 1302. Distance 1111b is the distance between face image 1302 and face image 1303. Distance 1111c is the distance between face image 1303 and face image 1304. Distance 1111d is the distance between face image 1304 and face image 1305. Distance 1111e is the distance between face image 1305 and face image 1306.

[0095] Area 1311 is an area of ​​a predetermined size below face image 1302 and includes an image of the chest of one living organism 102. Area 1312 is an area of ​​a predetermined size below face image 1303 and includes an image of the chest of another living organism 102. Because areas 1311 and 1312 partially overlap each other, there is a risk that the signal generating unit 204 may erroneously generate a signal indicating the time change in pixel values ​​caused by the respiratory movements of multiple adjacent living organisms 102 as the time-series signal 213 for one living organism 102.

[0096] That is, if the distance between the face image 1302 and the face image 1303 is equal to or less than a predetermined distance, the signal generating unit 204 may erroneously generate a signal indicating the temporal change in pixel values ​​caused by the respiratory movements of a plurality of adjacent living organisms 102 as the time-series signal 213 for one living organism 102. Therefore, when the moving image 211 includes a plurality of face images, the face images need to be sufficiently separated in order for the signal generating unit 204 to generate the time-series signal 213 for each living organism 102.

[0097] Therefore, for example, if distance 1111b is the smallest value among distances 1111a to 1111e and is equal to or less than a predetermined distance, respiration rate calculation unit 205 ends the process of calculating respiration rate 214. Furthermore, if distance 1111b is equal to or less than the predetermined distance, output unit 208 outputs a message indicating an error.

[0098] As described above, when the moving image 211 contains images of multiple faces, the measuring device 100 according to this embodiment can measure the respiration rate 214 for each living organism 102 among the multiple living organisms, with the facial images being sufficiently spaced apart. Therefore, even when the moving image 211 contains images of multiple faces, the measuring device 100 according to this embodiment can output the respiration rate for each living organism 102 with guaranteed reliability while reducing the measurement time.

[0099] (Modification of the third embodiment) As a modification of the measuring device 100 according to this embodiment, the measuring device 100 may include a measurement time setting unit 901 instead of the measurement time setting unit 209. In this case, the respiratory rate calculation unit 205 according to this modification ends the process of calculating the respiratory rate when the reliability 215 is lower than a threshold. Furthermore, the output unit 208 according to this modification outputs a message indicating an error when the reliability 215 is lower than a threshold. As a result, when the moving image 211 includes images of multiple faces, the measuring device 100 according to this modification does not extend the measurement time, thereby limiting the time required to measure the respiratory rate and reducing the burden on multiple living organisms 102.

[0100] The processes performed in the above embodiments are not limited to the processing modes exemplified in the above embodiments. The above-described functional blocks may be realized using either a logic circuit (hardware) formed in an integrated circuit or the like, or software using a CPU. The processes performed in the above embodiments may be executed by multiple computers. For example, some of the processes executed by the functional blocks of the control unit 202 of the measurement device 100 may be executed by another computer, or all of the processes may be shared and executed by multiple computers.

[0101] The present disclosure is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. Furthermore, new technical features can be formed by combining the technical means disclosed in each embodiment. [Explanation of symbols]

[0102] 100 measuring device, 101 imaging unit, 102 living body, 201 memory unit, 202 control unit, 203 region identification unit, 204 signal generation unit, 205 respiratory rate calculation unit, 206 reliability calculation unit, 207 judgment unit, 208 output unit, 209 measurement time setting unit, 211 moving image, 212 region, 213 time series signal, 215 reliability, 301 imaging unit, 401 image, 402 face image, 501 time series signal, 502 time series signal, 601 time series signal, 602 power spectrum, 801 time series signal, 802 power spectrum, 803 time series signal, 804 power spectrum, 805 time series signal, 806 power spectrum, 901 measurement time setting unit, 1001 distance calculation unit, 1101 Distance calculation unit, 1111 distance, 1111a to 1111e distances, 1301 to 1306 face images, 1310 image, 1311 region, 1312 region, P611 power spectrum value, P612 power spectrum value, P811 power spectrum value, P812 power spectrum value, P813 power spectrum value, P814 power spectrum value, P815 power spectrum value, P816 power spectrum value, PEFQ maximum peak frequency

Claims

1. an imaging unit that captures an image of a living body and acquires a moving image; a respiratory rate calculation unit that calculates a respiratory rate by detecting an oscillation period of a respiratory signal included in a time change of a plurality of pixel values ​​included in the moving image; a reliability calculation unit that calculates the reliability of the respiratory rate based on a comparison result between the respiratory signal and a noise signal included in the time change; an output unit that outputs the respiratory rate when the reliability is equal to or greater than a threshold; a measurement time setting unit that sets a measurement time for imaging the living body; Equipped with the imaging unit images the living body during the measurement time to obtain the moving image; The measurement time setting unit extends the measurement time when the reliability is lower than the threshold. Measuring equipment.

2. The reliability calculation unit calculates the reliability from a ratio between the magnitude of the respiratory signal and the magnitude of the noise signal. The measuring device according to claim 1 .

3. the plurality of pixel values ​​are pixel values ​​within a region including an image of a chest of the living body, the time change is represented by a time series signal indicating a fluctuation in the plurality of pixel values ​​or a fluctuation in a velocity vector determined from the plurality of pixel values; The respiratory rate calculation unit calculates the respiratory rate from a maximum peak frequency in the time-series signal.

3. The measuring device according to claim 1 or 2.

4. the magnitude of the respiratory signal is the power spectrum value of the maximum peak frequency; The magnitude of the noise signal is the power spectrum value of the second or lower peak frequency in the time series signal. The measuring device according to claim 3 .

5. the magnitude of the respiratory signal is a statistical value of power spectrum values ​​within a predetermined range including the maximum peak frequency; The magnitude of the noise signal is a statistical value of the power spectrum values ​​outside the predetermined range. The measuring device according to claim 3 .

6. The measurement time setting unit sets at least one selected from the group consisting of a lower limit value and an upper limit value of the measurement time. The measuring device according to any one of claims 1 to 5.

7. the output unit outputs a message indicating an error when the reliability is lower than the threshold; The respiratory rate calculation unit terminates the process of calculating the respiratory rate when the reliability is lower than the threshold value. The measuring device according to any one of claims 1 to 5.

8. If the moving image includes a plurality of facial images, a distance calculation unit is further provided to calculate a distance between the plurality of facial images; the output unit outputs a message indicating an error when the minimum distance is equal to or less than a predetermined distance; The respiratory rate calculation unit terminates the process of calculating the respiratory rate when the minimum value is equal to or less than the predetermined distance. The measuring device according to any one of claims 1 to 7.

9. The measuring device a step of capturing an image of a living body to obtain a moving image; calculating a respiratory rate by detecting an oscillation period of a respiratory signal included in a time change of a plurality of pixel values ​​included in the moving image; calculating a reliability of the respiratory rate from a comparison result between the respiratory signal and a noise signal included in the time change; outputting the respiratory rate when the reliability is equal to or greater than a threshold; setting a measurement time for imaging the living body; Run In the step of acquiring the moving image, the measurement device captures an image of the living body for the measurement time to acquire the moving image, In the step of setting the measurement time, the measurement device extends the measurement time if the reliability is lower than the threshold. How to measure respiratory rate.

10. On the computer, A function to capture images of a living body and acquire moving images; a function of calculating a respiratory rate by detecting an oscillation period of a respiratory signal included in a time change of a plurality of pixel values ​​included in the moving image; a function of calculating the reliability of the respiratory rate from a comparison result between the respiratory signal and a noise signal included in the time change; a function of outputting the respiratory rate when the reliability is equal to or greater than a threshold; a function of setting a measurement time for imaging the living body; Execute In the function of acquiring the moving image, the living body is imaged for the measurement time to acquire the moving image; In the function of setting the measurement time, if the reliability is lower than the threshold, the measurement time is extended. program.

Images