Arrhythmia detection support device and program

Abstract

This arrhythmic state detection assisting device comprises: an acquisition unit that acquires image information obtained by capturing an image of a predetermined part of a subject through an image capture unit that captures a color moving image; a derivation unit that derives a variance value for each period of a periodic change, within a predetermined time, in a green luminance component in the image information acquired by the acquisition unit; and an estimation unit that estimates that the greater the variance value derived by the derivation unit, the more likely it is that the subject is having an arrhythmia.

Description

[Technical Field]

[0001] The present disclosure relates to an arrhythmia detection support device and a program. [Background technology]

[0002] Conventionally, the following techniques have been available as techniques that can be applied to support health management of a user by utilizing the user's pulse rate.

[0003] Japanese Patent Application Laid-Open Publication No. 2019-58258 discloses a mental and physical state estimation device that aims to provide a technology for accurately estimating a subject's mental and physical state, such as stress, without imposing a burden on the subject. This mental and physical state estimation device includes a calculation means for calculating a first similarity indicating the degree of similarity between a first pulse rate for a first period estimated by a first estimation method using multiple luminance values ​​of a first region including a facial part in multiple time-series face images, and a second pulse rate for the first period estimated by a second estimation method different from the first estimation method. The mental and physical state estimation device also includes an estimation means for estimating the subject's mental and physical state using at least one of the first pulse rate, the second pulse rate, and the multiple face images for the first period in accordance with the calculated first similarity.

[0004] Japanese Patent Application Laid-Open Publication No. 2019-136352 discloses a bioinformation display device aimed at improving usability when performing heart rate variability biofeedback. The bioinformation display device includes a housing and an imaging unit that captures an image of a subject's face and acquires video data of a detection area including the subject's facial skin from the captured image. The bioinformation display device also includes a processing unit that acquires bioinformation, including information about the person's pulse wave, from the video data of the detection area and generates an information display screen for performing heart rate variability biofeedback using the bioinformation. The bioinformation display device also includes a display unit that is disposed on the same surface as the imaging unit in the housing and displays the information display screen. The processing unit also generates an information display screen for the heart rate variability biofeedback, including a breathing assistant that assists the subject's breathing. In the bioinformation display device, at least a portion of the breathing assistant is disposed on a straight line passing through the imaging unit, and the straight line passing through the imaging unit perpendicularly intersects with the outer peripheral edge of the display unit closest to the imaging unit.

[0005] Japanese Patent Publication No. 2017-85894 discloses a pulse wave analyzer that aims to improve the accuracy of analyzing a subject's pulse waveform. This pulse wave analyzer extracts images of each of multiple body parts of a subject from multiple captured images of the subject, and generates a pulse waveform for each of the body parts by analyzing the images of each of the body parts extracted from the multiple captured images. The pulse wave analyzer then calculates a first degree of agreement that indicates the degree of agreement between the pulse waveforms of the generated pulse waveforms for each of the body parts.

[0006] U.S. Patent No. 10,004,410 discloses a technique for detecting pulse wave information, including arrhythmia, of a subject using an image of the subject's face. This technique uses red and green brightness components in the image of the subject's face to estimate whether the subject has arrhythmia. Summary of the Invention [Problem to be solved by the invention]

[0007] Incidentally, as a result of research by the inventors of the present invention, it was found that the occurrence of arrhythmia in a subject is highly correlated with the variance value for each cycle within a specified period of the periodic change in the green luminance component in the image information obtained by photographing a predetermined part of the subject.

[0008] However, the techniques disclosed in the above-mentioned patent documents do not take into consideration the variance value, and even if these techniques are applied, there is a problem in that it is not necessarily possible to estimate with high accuracy whether or not a subject is likely to have an arrhythmia.

[0009] The present disclosure has been made in consideration of the above-mentioned problems, and aims to provide an arrhythmia state detection support device and program that can estimate with higher accuracy whether or not there is a high possibility that an arrhythmia is occurring. [Means for solving the problem]

[0010] A first aspect of the present disclosure includes an acquisition unit that acquires image information obtained by capturing images of a predetermined part of a subject using an imaging device that captures color moving images, a derivation unit that derives a variance value for each period within a predetermined period of the periodic change in the green luminance component in the image information acquired by the acquisition unit, and an estimation unit that estimates that the larger the variance value derived by the derivation unit, the more likely the subject is to have arrhythmia.

[0011] A second aspect of the present disclosure is the first aspect, wherein the estimation unit estimates that there is a high possibility that arrhythmia is occurring when the variance value is equal to or greater than a predetermined threshold.

[0012] A third aspect of the present disclosure is the first or second aspect, wherein the part is the face of the subject.

[0013] A fourth aspect of the present disclosure is the third aspect, wherein the parts are parts at multiple locations on the face.

[0014] A fifth aspect of the present disclosure is any one of the first to fourth aspects, wherein the derivation unit derives the variance value in accordance with the movement of the part when the part moves in the moving image indicated by the image information acquired by the acquisition unit.

[0015] A sixth aspect of the present disclosure is any one of the first to fifth aspects, further comprising an identification unit that uses a time series pattern of the green luminance component to identify the type of arrhythmia if the subject has arrhythmia.

[0016] A seventh aspect of the present disclosure causes a computer to execute a process of acquiring image information obtained by photographing a predetermined part of a subject using an imaging device that captures color moving images, deriving a variance value for each period within a predetermined period of periodic changes in the green luminance component in the acquired image information, and estimating that the larger the derived variance value, the more likely the subject is experiencing arrhythmia. [Effects of the Invention]

[0017] According to the present disclosure, it is possible to estimate with higher accuracy whether or not there is a high possibility that arrhythmia is occurring. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a block diagram showing an example of a hardware configuration of an arrhythmia state detection assistance device according to an embodiment. [Figure 2] 1 is a block diagram showing an example of a functional configuration of an arrhythmia state detection assistance device according to an embodiment. [Figure 3] FIG. 10 is a schematic diagram illustrating an example of a configuration of a green luminance component information database according to an embodiment. [Figure 4] FIG. 2 is a schematic diagram showing an example of the configuration of an arrhythmia pattern information database according to an embodiment. [Figure 5]FIG. 1 is a diagram illustrating the principle of a method for estimating the occurrence status of arrhythmia according to an embodiment, and is a schematic diagram showing the configuration of an experiment conducted by the inventors of the present invention. [Figure 6] FIG. 1 is a diagram illustrating the principle of a method for estimating the occurrence status of arrhythmia according to an embodiment, and is a graph showing the state of variance values ​​obtained in an experiment by the inventors of the present invention. [Figure 7] FIG. 1 is a diagram illustrating the principle of a method for estimating the occurrence status of arrhythmia according to one embodiment, and is a graph illustrating the instantaneous pulse rate applied in an experiment conducted by the inventors of the present invention. [Figure 8] 10 is a flowchart illustrating an example of arrhythmia state detection support processing according to an embodiment. [Figure 9] 10 is a front view showing an example of the configuration of an arrhythmia presentation screen displayed when the arrhythmia state detection support process according to one embodiment is executed. FIG. [Figure 10] FIG. 10 is a front view showing an example of the configuration of a sinus rhythm presentation screen displayed when the arrhythmia state detection support process according to one embodiment is executed. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of the present disclosure will be described in detail below with reference to the drawings. Note that the same or equivalent components and parts in each drawing are denoted by the same reference numerals. Also, the dimensional proportions in the drawings are exaggerated for the sake of explanation and may differ from the actual proportions.

[0020] In this embodiment, a description will be given of a case where an arrhythmia state detection assistance device according to the technology of the present disclosure is applied to a general-purpose smartphone. However, the application of the technology of the present disclosure is not limited to smartphones, and it can be applied to other portable information processing devices such as portable game devices, tablet terminals, and notebook personal computers, as well as stationary information processing devices.

[0021] First, the configuration of an arrhythmia state detection support device 10 according to this embodiment will be described with reference to Fig. 1 and Fig. 2. Fig. 1 is a block diagram showing an example of the hardware configuration of the arrhythmia state detection support device 10 according to one embodiment. Fig. 2 is a block diagram showing an example of the functional configuration of the arrhythmia state detection support device 10 according to one embodiment.

[0022] As shown in FIG. 1 , the arrhythmia detection support device 10 according to this embodiment includes a CPU (Central Processing Unit) 11, a memory 12 serving as a temporary storage area, a nonvolatile storage unit 13, and an input unit 14 including a touch panel and various switches. The arrhythmia detection support device 10 according to this embodiment also includes a display unit 15 such as a liquid crystal display, and a medium read / write device (R / W) 16. The arrhythmia detection support device 10 according to this embodiment also includes a wireless communication unit 18 for mobile communication using a predetermined communication standard, an audio output unit 19, and an imaging unit 20 that functions as an imaging device for capturing color moving images. The CPU 11, memory 12, storage unit 13, input unit 14, display unit 15, medium read / write device 16, wireless communication unit 18, audio output unit 19, and imaging unit 20 are interconnected via a bus B. The medium read / write device 16 reads information from and writes information to a recording medium 17.

[0023] On the other hand, the storage unit 13 is realized by an HDD (Hard Disk Drive), an SSD (Solid State Drive), a flash memory, etc. An arrhythmia state detection support program 13A is stored in the storage unit 13 as a storage medium. The arrhythmia state detection support program 13A is stored in the storage unit 13 by setting a recording medium 17, on which the program 13A is written, in the medium reading and writing device 16 and reading the program 13A from the recording medium 17 by the medium reading and writing device 16. The CPU 11 reads the arrhythmia state detection support program 13A from the storage unit 13, expands it in the memory 12, and sequentially executes processes included in the arrhythmia state detection support program 13A.

[0024] As described above, in the arrhythmia state detection support device 10 according to this embodiment, the arrhythmia state detection support program 13A is installed in the arrhythmia state detection support device 10 via the recording medium 17, but this is not limiting. For example, the arrhythmia state detection support program 13A may be installed in the arrhythmia state detection support device 10 by downloading it via the wireless communication unit 18.

[0025] Furthermore, a green luminance component information database 13B and an arrhythmia pattern information database 13C are stored in the storage unit 13. The green luminance component information database 13B and the arrhythmia pattern information database 13C will be described in detail later.

[0026] Next, the functional configuration of the arrhythmia state detection support device 10 according to this embodiment will be described with reference to Fig. 2. As shown in Fig. 2, the arrhythmia state detection support device 10 according to this embodiment includes an acquisition unit 11A, a derivation unit 11B, an estimation unit 11C, and an identification unit 11D. The CPU 11 of the arrhythmia state detection support device 10 executes an arrhythmia state detection support program 13A, thereby functioning as the acquisition unit 11A, the derivation unit 11B, the estimation unit 11C, and the identification unit 11D.

[0027] The acquiring unit 11A according to this embodiment acquires image information obtained by capturing an image of a predetermined part of a subject using an imaging device (in this embodiment, the imaging unit 20) that captures color moving images.

[0028] In this embodiment, the subject's face is used as the predetermined region, but this is not limited thereto. For example, any part of the subject's exposed skin, such as the subject's hands, arms, or neck, can be used as the predetermined region. Furthermore, in this embodiment, multiple regions on the subject's face are used as the predetermined region. In this embodiment, two regions, the forehead and the left cheek, are used as the multiple regions on the face, but this is not limited thereto. For example, a combination of multiple regions, including these regions as well as other regions on the subject's face, such as the right cheek and chin, may be used as the multiple regions on the face.

[0029] Furthermore, the derivation unit 11B according to this embodiment derives a variance value (hereinafter simply referred to as "variance value") for each period of the periodic change in the green luminance component in the image information acquired by the acquisition unit 11A within a predetermined period. The estimation unit 11C according to this embodiment estimates that the larger the variance value derived by the derivation unit 11B, the more likely the subject is to have arrhythmia. In particular, the estimation unit 11C according to this embodiment estimates that the more likely the subject is to have arrhythmia when the variance value is equal to or greater than a predetermined threshold value TH (see also FIG. 8), which will be described later.

[0030] In this embodiment, the threshold value TH is a value obtained in advance by experiment, computer simulation, etc., as a value at which it can be considered that the subject is highly likely to have arrhythmia when the variance value is equal to or greater than this value. However, the present invention is not limited to this, and for example, the threshold value TH may be set in advance for the subject himself / herself depending on the arrhythmia detection accuracy required of the arrhythmia state detection support device 10, the subject's exercise status, the subject's gender, age group, etc.

[0031] Furthermore, in this embodiment, as described above, when the variance value derived by the derivation unit 11B is equal to or greater than a predetermined threshold value TH, it is estimated that there is a high possibility that arrhythmia has occurred. However, this is not limiting. For example, when the difference between the variance value derived by the derivation unit 11B and the variance value of the subject himself / herself over a predetermined period in the past is equal to or greater than a predetermined threshold, it may be estimated that there is a high possibility that arrhythmia has occurred in the subject. In this embodiment, the variance value over the predetermined period in the past can be, for example, the average value of the variance values ​​over the most recent predetermined period (for example, 10 minutes), or the average value of the variance values ​​over a predetermined period including the same time on the previous day (for example, 5 minutes).

[0032] Furthermore, when the part moves in the moving image indicated by the image information acquired by the acquisition part 11A, the derivation part 11B according to this embodiment derives the variance value in accordance with the movement of the part.

[0033] Furthermore, the identifying unit 11D according to this embodiment identifies the type of arrhythmia when the subject has arrhythmia, using the time-series pattern of the green luminance component.

[0034] Next, the green luminance component information database 13B according to this embodiment will be described with reference to Fig. 3. Fig. 3 is a schematic diagram showing an example of the configuration of the green luminance component information database 13B according to one embodiment.

[0035] The green luminance component information database 13B in this embodiment is intended to store information obtained by capturing moving images using the imaging unit 20 of the arrhythmia state detection support device 10 in order to detect the high possibility that the subject is experiencing arrhythmia.

[0036] As shown in FIG. 3, the green luminance component information database 13B according to this embodiment stores information relating to the green luminance component (hereinafter also referred to as the "green luminance component") for each predetermined period of time (0.01 seconds in this embodiment) from the start of imaging by the imaging unit 20.

[0037] That is, the image information obtained by the image capturing unit 20 includes luminance components for each of the primary colors of red, green, and blue, and the green luminance component information database 13B according to this embodiment stores only information about the green luminance component among the luminance components for each primary color in chronological order. However, this is not limiting, and information about the luminance components of all colors of red, green, and blue may be stored.

[0038] Next, the arrhythmia pattern information database 13C according to this embodiment will be described with reference to Fig. 4. Fig. 4 is a schematic diagram showing an example of the configuration of the arrhythmia pattern information database 13C according to one embodiment.

[0039] The arrhythmia pattern information database 13C according to this embodiment is for storing information for identifying the type of arrhythmia when there is a high possibility that the subject has an arrhythmia.

[0040] 4, the arrhythmia pattern information database 13C according to this embodiment stores information on arrhythmia names and patterns for each type of arrhythmia assumed by the arrhythmia state detection support device 10 according to this embodiment. The patterns are information indicating the time-series waveform of a typical pulse of the corresponding arrhythmia.

[0041] Here, the principle of the method for estimating the occurrence status of arrhythmia in the arrhythmia state detection support processing according to this embodiment, which will be described later, will be described with reference to Figs. 5 to 7. Fig. 5 is a diagram for explaining the principle of the method for estimating the occurrence status of arrhythmia according to one embodiment, and is a schematic diagram showing the configuration of an experiment conducted by the inventors of the present invention. Fig. 6 is a diagram for explaining the principle of the method for estimating the occurrence status of arrhythmia according to one embodiment, and is a graph showing the state of variance values ​​obtained in an experiment conducted by the inventors of the present invention. Fig. 7 is a diagram for explaining the principle of the method for estimating the occurrence status of arrhythmia according to one embodiment, and is a graph for explaining the instantaneous pulse rate applied in an experiment conducted by the inventors of the present invention.

[0042] It is known that by analyzing a body image captured with a conventional imaging device, a pulse wave signal (hereinafter also referred to as a "video pulse wave") similar to the pulse wave signal obtained from an existing photoplethysmograph can be obtained from the green luminance component of the body image. However, there are few studies that have observed the video pulse waves of patients with arrhythmia, which affects the video pulse wave. Therefore, the inventors of the present invention observed the effect of arrhythmia on the video pulse wave in patients with atrial fibrillation, among arrhythmia patients.

[0043] The subjects of this observation were 10 healthy volunteers, two males and eight females, and 34 arrhythmia patients admitted to Iwate Medical University Hospital for ablation treatment (31 patients with persistent atrial fibrillation, two patients who fell into sinus rhythm during hospitalization, and one patient receiving pacing therapy with an internal pacemaker). Actual measurements were carried out on 50 patients with atrial fibrillation and 12 healthy adults, but cases in which the reliability of the values ​​was likely to be significantly low due to the influence of environmental conditions at the time of measurement were excluded, and the above-mentioned healthy volunteers and arrhythmia patients were the subjects of this observation.

[0044] As shown in Figure 5, the subject sits on a chair placed in front of a video camera for video recording, with their head fixed on a support stand. The subject wears sunglasses to protect their privacy, and their face is photographed. A photoplethysmograph is also attached to the subject's finger.

[0045] In this state, a moving image of a part of the subject's face was taken, and the photoplethysmogram and electrocardiogram were measured for one minute. The obtained moving image was analyzed to obtain a video pulse wave, and the video pulse wave was compared with the photoplethysmogram and electrocardiogram in terms of pulse rate and pulse wave. Note that in the example shown in Figure 5, a "hand stand and cover" is also provided so that the subject's palm can also be photographed, but this is not included in the measurement this time.

[0046] As a result, using the average pulse rate x of the photoplethysmogram as the reference, the average pulse rate y of the videoplethysmogram was y = 1.0009x - 0.0389 (correlation coefficient R = 0.99) for healthy adults in sinus rhythm, and similarly for patients with atrial fibrillation, the average pulse rate was y = 0.9823x (correlation coefficient R = 0.8), demonstrating a correlation between the photoplethysmogram and the videoplethysmogram.

[0047] Furthermore, the inventors of the present invention adopted, as the variance value for each period of the periodic change of the video pulse wave within a predetermined period, a variance value V obtained by the following equation (1) using the instantaneous pulse rate IP (beats / minute) obtained by substituting the time difference (seconds) between peaks P1 and P2 of adjacent pulse waves shown in FIG. 7 into equation (2).

[0048] V = standard deviation of IP / average pulse rate per minute (1) where: IP=60 / P1-P2(2)

[0049] The variance value V obtained from the above observations was 0.06±0.03 for healthy subjects and 0.20±0.03 for patients with atrial fibrillation, as shown in Figure 6. This indicates that the variance value V obtained from the video pulse wave can almost completely separate healthy subjects from patients with atrial fibrillation.

[0050] From the above observation results, the arrhythmia state detection support device 10 according to this embodiment uses the variance value V to estimate whether or not there is a high possibility that the subject is experiencing arrhythmia.

[0051] Next, the operation of the arrhythmia state detection support device 10 according to this embodiment will be described with reference to Figs. 8 to 10. Fig. 8 is a flowchart showing an example of arrhythmia state detection support processing according to one embodiment. Fig. 9 is a front view showing an example of the configuration of an arrhythmia presentation screen displayed when the arrhythmia state detection support processing according to one embodiment is executed. Fig. 10 is a front view showing an example of the configuration of a sinus rhythm presentation screen displayed when the arrhythmia state detection support processing according to one embodiment is executed.

[0052] The CPU 11 of the arrhythmia state detection support device 10 executes the arrhythmia state detection support program 13A, thereby executing the arrhythmia state detection support process shown in Fig. 8. The arrhythmia state detection support process shown in Fig. 8 is executed when the user inputs an instruction to start execution of the arrhythmia state detection support program 13A via the input unit 14. At this time, the user positions the arrhythmia state detection support device 10 at a position where the imaging unit 20 can capture an image of the user's face.

[0053] In step 100 of FIG. 8, CPU 11 controls the imaging unit 20 to start capturing color moving images, and in step 102, CPU 11 acquires image information indicating one color image in the moving images obtained by capturing from the imaging unit 20.

[0054] In step 104, the CPU 11 extracts the value of the green luminance component from the acquired image information. In this embodiment, the extraction of the value of the green luminance component is performed as follows.

[0055] First, the CPU 11 detects two areas of the user's face, the forehead and the left cheek, from the acquired image information. In this embodiment, the forehead and the left cheek are detected by detecting the user's face area using a conventionally known face recognition technology, and specifying a part of the skin-colored area of ​​the face area, including the upper end, as the forehead area, and a part of the skin-colored area of ​​the face area, including the right end, as the left cheek area. However, this is not limited to this embodiment, and it goes without saying that other conventionally known image recognition technology may be used to detect the two areas, the forehead and the left cheek.

[0056] Next, the CPU 11 extracts the green luminance component value for each region of the forehead and left cheek from the pixel information of each detected region of the forehead and left cheek in the acquired image information, calculates the average value of the green luminance component for each region, and then adopts the larger average value of the calculated average values ​​for each region as the value of the green luminance component, since it considers this value to be less susceptible to adverse effects of various noises.

[0057] In this manner, in this embodiment, the larger of the average values ​​of the green luminance component values ​​in a plurality of regions (two regions, the forehead and the left cheek, in this embodiment) is applied as the green luminance component, but this is not limitative. For example, a further average value of the average values ​​of the green luminance component values ​​in the plurality of regions may be applied as the green luminance component.

[0058] In step 106, CPU 11 stores the extracted value of the green luminance component in green luminance component information database 13B together with the time elapsed since the start of imaging by imaging unit 20. In step 108, CPU 11 determines whether a predetermined first period (three minutes in this embodiment) has elapsed, and if the determination is affirmative, proceeds to step 112, whereas if the determination is negative, proceeds to step 110. In step 110, CPU 11 waits until a predetermined second period (0.05 seconds in this embodiment) has elapsed, and then returns to step 102.

[0059] By repeating the above steps 102 to 110, the green luminance component information database 13B shown in FIG. 3 is successively constructed as an example.

[0060] In step 112, CPU 11 reads the values ​​of the green luminance component for the most recent predetermined period (in this embodiment, 3 minutes) from green luminance component information database 13B, and creates a pulse waveform (corresponding to the above-mentioned "video pulse wave") using the read green luminance component values.

[0061] In step 114, CPU 11 uses the created pulse waveform to derive the above-mentioned variance value V. In step 116, CPU 11 determines whether the derived variance value V is equal to or greater than a threshold value TH (0.13 in this embodiment), and if the determination is affirmative, CPU 11 determines that there is a high possibility that arrhythmia has occurred and proceeds to step 118.

[0062] In step 118, CPU 11 uses the created pulse wave waveform pattern (hereinafter referred to as the "detected pulse wave pattern") to identify the type of arrhythmia that is likely to have occurred (hereinafter referred to as the "arrhythmia type") as follows:

[0063] First, the CPU 11 reads all information (hereinafter referred to as "arrhythmia pattern information") from the arrhythmia pattern information database 13C, and then identifies the pattern most similar to the detected pulse wave pattern from the patterns indicated by the read arrhythmia pattern information. Note that in this embodiment, the pattern is identified using a conventionally known pattern matching technique, but the present invention is not limited to this.

[0064] Then, the CPU 11 identifies the arrhythmia type indicated by the arrhythmia name corresponding to the identified most similar pattern in the arrhythmia pattern information as the arrhythmia type.

[0065] In step 120, the CPU 11 controls the display unit 15 to display an arrhythmia presentation screen having a predetermined configuration, and then proceeds to step .

[0066] An example of an arrhythmia presentation screen according to this embodiment is shown in Fig. 9. As shown in Fig. 9, the arrhythmia presentation screen according to this embodiment displays information indicating a high probability that an arrhythmia has occurred, together with information indicating the type of the identified arrhythmia ("atrial fibrillation" in the example shown in Fig. 9). Therefore, by referring to the arrhythmia presentation screen, the user can understand that a high probability that the user is experiencing atrial fibrillation.

[0067] On the other hand, if the determination in step 116 is negative, it is assumed that the possibility of arrhythmia occurring is not high, and the process proceeds to step 122 .

[0068] In step 122, the CPU 11 controls the display unit 15 to display a sinus rhythm presentation screen having a predetermined configuration, and then proceeds to step .

[0069] An example of a sinus rhythm presentation screen according to this embodiment is shown in Fig. 10. As shown in Fig. 10, the sinus rhythm presentation screen according to this embodiment displays a graph showing the user's heart rate at that time and the most recent trend in the variance value V. Therefore, by referring to the sinus rhythm presentation screen, the user can understand that the possibility of the user having arrhythmia is low and can see the most recent trend in the variance value V.

[0070] In step 124, the CPU 11 determines whether a predetermined end timing has arrived, and if the determination is negative, the process returns to step 102, whereas if the determination is positive, the process proceeds to step 126. In this embodiment, the end timing is the timing when the user inputs an instruction to end the execution of the arrhythmia state detection support program 13A via the input unit 14, but this is not limitative. For example, the end timing may be another timing, such as the timing when the user's face is out of the imaging area captured by the imaging unit 20 for a predetermined period of time (e.g., 10 seconds) or a preset time.

[0071] In step 126, the CPU 11 stops the imaging by the imaging unit 20 that was started in the process of step 100, and then ends the arrhythmia state detection support process.

[0072] As described above, according to one embodiment, the apparatus includes an acquisition unit 11A that acquires image information obtained by capturing a predetermined part of a subject using an imaging device that captures color moving images, a derivation unit 11B that derives a variance value for each period within a predetermined period of the periodic change in the green luminance component in the image information acquired by the acquisition unit 11A, and an estimation unit 11C that estimates that the larger the variance value derived by the derivation unit 11B, the more likely the subject is to have arrhythmia. Therefore, it is possible to estimate with higher accuracy whether or not the subject is likely to have arrhythmia.

[0073] According to one embodiment, if the variance value is equal to or greater than a predetermined threshold, it is estimated that there is a high possibility that arrhythmia has occurred, which makes it possible to more easily estimate whether there is a high possibility that arrhythmia has occurred compared to when the threshold is not used.

[0074] According to one embodiment, the region is the face of the subject, which makes it possible to more easily estimate the arrhythmia state compared to when the region is not the face.

[0075] According to one embodiment, the region is a plurality of regions on the face, which allows the arrhythmia state to be estimated with higher accuracy than when the region is not a plurality of regions.

[0076] Furthermore, according to one embodiment, when the region moves in the moving image represented by the acquired image information, the variance value is derived in accordance with the movement of the region, thereby enabling the arrhythmia state to be estimated with higher accuracy than when the variance value is derived without tracking the movement of the region.

[0077] Furthermore, according to one embodiment, if the subject has arrhythmia, the type of arrhythmia is identified using the time series pattern of the green luminance component, thereby providing greater convenience to the user than when the type of arrhythmia is not identified.

[0078] In the above embodiment, the arrhythmia occurrence status is presented by displaying it on the display unit, but the present invention is not limited to this. For example, the arrhythmia occurrence status may be presented by voice output from the voice output unit 19, or may be presented by printing it on an image forming device such as a printer.

[0079] Furthermore, in the above embodiment, the arrhythmia state detection assistance device of the present invention has been described as being configured as a standalone device (in the above embodiment, a smartphone), but is not limited to this. For example, the arrhythmia state detection assistance device of the present invention may be configured as a system using multiple devices, such as a server device such as a cloud server and a terminal device. In this case, for example, a color video of the user is captured by the terminal device and transferred to the server device, and the server device estimates the occurrence of arrhythmia using the received video, transmits the estimation result to the terminal device, and presents the estimation result on the terminal device.

[0080] Furthermore, in the above embodiment, a case has been described in which it is estimated that the subject is likely to have arrhythmia when the variance value V is equal to or greater than the threshold value TH, but the present invention is not limited to this. For example, it may be estimated that the greater the variance value V, the greater the likelihood that the subject is experiencing arrhythmia. An example of this case is a case in which, in the processing of step 120 of the arrhythmia state detection support processing described above (see FIG. 8 ), information indicating that the greater the variance value V, the greater the likelihood that the subject is experiencing arrhythmia is displayed on the arrhythmia presentation screen.

[0081] Furthermore, in the above embodiment, for example, the following various processors can be used as the hardware structure of the processing units that execute the processes of the acquisition unit 11A, the derivation unit 11B, the estimation unit 11C, and the identification unit 11D. As described above, the various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as a processing unit, as well as dedicated electrical circuits that are processors having a circuit configuration specifically designed to execute specific processes, such as a programmable logic device (PLD) that is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field-Programmable Gate Array), and an ASIC (Application Specific Integrated Circuit).

[0082] The processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA).The processing unit may also be configured with a single processor.

[0083] Examples of configuring a processing unit with a single processor include, first, a form in which one processor is configured with a combination of one or more CPUs and software, and this processor functions as the processing unit, as typified by computers such as client and server. Second, a form in which a processor is used to realize the functions of the entire system, including the processing unit, on a single IC (Integrated Circuit) chip, as typified by systems on chips (SoCs). In this way, the processing unit is configured using one or more of the above-mentioned various processors as a hardware structure.

[0084] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0085] The disclosure of Japanese Patent Application No. 2020-216927, filed on December 25, 2020, is incorporated herein by reference in its entirety. All documents, patent applications, and technical standards described herein are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard was specifically and individually indicated to be incorporated by reference.

[0086] The above-described embodiment may be implemented as a non-transitory storage medium storing the program, such as a CD-ROM (Compact Disc Read Only Memory), a magneto-optical disk, a HDD, a DVD-ROM (Digital Versatile Disc Read Only Memory), a flash memory, or a memory card.

[0087] The following notes are provided regarding the above embodiments. (Additional note 1) A non-transitory storage medium storing a program executable by a computer to execute arrhythmia state detection assistance processing, The arrhythmia state detection support process includes: acquiring image information obtained by photographing a predetermined part of the subject using an imaging device that photographs color moving images; Deriving a variance value for each period within a predetermined period of the periodic change of the green luminance component in the acquired image information; The larger the derived variance value, the more likely the subject is to have arrhythmia. Non-transitory storage medium.

Claims

1. an acquisition unit that acquires image information obtained by capturing an image of a predetermined part of the subject, that is, an exposed part of the subject's skin, using an imaging device that captures color moving images; a derivation unit that derives a variance value for each period within a predetermined period of the periodic change in the green luminance component in the image information acquired by the acquisition unit, the variance value being calculated by dividing the standard deviation of the instantaneous pulse wave number obtained from the time difference between the peak times of adjacent pulse waves by the average pulse rate per unit time; an estimation unit that estimates that the greater the variance value derived by the derivation unit, the greater the possibility that the subject is experiencing arrhythmia; An arrhythmia detection support device comprising:

2. the estimation unit estimates that there is a high possibility that arrhythmia is occurring when the variance value is equal to or greater than a predetermined threshold. The arrhythmia detection support device according to claim 1 .

3. The part is the face of the subject.

3. The arrhythmia detection support device according to claim 1.

4. The region is a plurality of regions on the face. The arrhythmia detection support device according to claim 3 .

5. the derivation unit derives the variance value in accordance with the movement of the part when the part moves in the moving image indicated by the image information acquired by the acquisition unit; The arrhythmia detection support device according to any one of claims 1 to 4.

6. an identification unit that identifies a type of arrhythmia when the subject has arrhythmia, using a time series pattern of the green luminance component; The arrhythmia state detection assistance device according to any one of claims 1 to 5, further comprising:

7. acquiring image information obtained by capturing an image of a predetermined part of the subject, that is, an exposed part of the subject's skin, using an imaging device that captures color moving images; deriving a variance value for each period of the periodic change in the green luminance component in the acquired image information within a predetermined period by dividing the standard deviation of the instantaneous pulse wave number obtained from the time difference between the peak times of adjacent pulse waves by the average pulse rate per unit time; It is estimated that the larger the derived variance value, the more likely the subject is to have arrhythmia. A program that causes a computer to execute a process.

Images