Recovery Degree Estimation Device, Recovery Degree Estimation Method, and Program

Abstract

In this recovery degree estimation device, an image acquiring means acquires an image capturing an eyeball of a patient. An eyeball motion extracting means extracts an eyeball motion characteristic, which is a characteristic of eyeball motion, on the basis of the image. A recovery degree estimation means, using a recovery degree estimation model that is machine-learned in advance, estimates a recovery degree of the patient from the eyeball motion characteristic.

Description

Technical Field

[0001] The present invention relates to a technique for estimating the degree of recovery of a patient.

Background Art

[0002] With medical costs globally squeezing national finances, the number of cerebrovascular disease patients in the country has reached 1.115 million, and the annual medical costs exceed 1.8 trillion yen. With the increase in the number of cerebral infarction patients expected due to the declining birthrate and aging population, there is a strong need to improve the efficiency of operations not only in acute care hospitals but also in convalescent rehabilitation hospitals.

[0003] Since serious sequelae will remain if emergency transportation and measures are not taken promptly after the onset of cerebral infarction, it is important to detect it early among mild symptoms and receive treatment. Approximately half of cerebral infarction patients will develop cerebral infarction again within 10 years, and there is a high possibility of recurrence of the same type of cerebral infarction as the first time. Therefore, there is also a strong need for early detection of signs of recurrence.

[0004] However, in order to measure the degree of recovery of patients in convalescent rehabilitation hospitals, medical staff need to accompany and conduct various tests, which is time-consuming and laborious. As a result, if the frequency of measuring the degree of recovery decreases, there will be no feedback to patients and medical staff, the motivation for rehabilitation of patients will decrease, the amount of rehabilitation will decrease, or the review of inappropriate rehabilitation plans will be delayed and the recovery effect will decrease. In addition, the signs of recurrence are difficult for the patient himself / herself to notice and often cannot be detected in regular examinations and consultations.

[0005] Patent Document 1 describes making a more objective quantification of the recovery status regarding walking from the movements during walking and the movement of the line of sight of a patient. Patent Document 2 describes estimating the mental state from feature quantities based on eye movement. Patent Document 3 describes determining the reflexivity of eye movement under predetermined conditions. Patent Document 4 describes estimating the recovery transition based on quantified motion information of rehabilitation subjects.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

Summary of the Invention

Problems to be Solved by the Invention

[0007] Conventionally, the estimation of the patient's recovery degree has been performed by a medical staff or an expert visually evaluating or palpating the state of the patient performing a predetermined movement and quantifying the recovery status. It is also known to transmit the patient's movement video and the result of human body posture analysis as data, and for a medical staff or an expert to visually evaluate the data to quantify the recovery status of a patient located remotely. Further, Patent Document 1 describes that a medical information processing system quantifies the recovery status by analyzing the way of movement of the human body from a video of the patient's walking scene.

[0008]

[0009] One of the objects of the present invention is to quantitatively estimate the recovery degree without imposing a burden on the patient or the medical staff.

Means for Solving the Problems

[0010] To solve the above problems, in one aspect of the present invention, a recovery degree estimation device includes: image acquisition means for acquiring an image of a patient's eyeball; eyeball movement feature extraction means for extracting an eyeball movement feature, which is a feature of eyeball movement, based on the image; recovery degree estimation means for estimating the recovery degree of the patient from the eyeball movement feature using a pre-trained recovery degree estimation model; output means for outputting the degree of recovery; patient information storage means for storing a recovery record including one or more of the patient's past recovery degree history and rehabilitation content; The recovery degree estimation means estimates the recovery degree of the patient from the recovery record and the eye movement characteristics.

[0011] In another aspect of the present invention, a recovery degree estimation method is executed by a computer, and includes: acquiring an image of a patient's eyeball; extracting an eyeball movement feature, which is a feature of eyeball movement, based on the image; estimating the recovery degree of the patient from the eyeball movement feature using a pre-trained recovery degree estimation model; Execute the estimation process and Output the degree of recovery; Store a recovery record including one or more of the patient's past recovery degree history and rehabilitation content; The estimation process estimates the recovery degree of the patient from the recovery record and the eye movement characteristics.

[0012] In still another aspect of the present invention, a program includes: acquiring an image of a patient's eyeball; extracting an eyeball movement feature, which is a feature of eyeball movement, based on the image; Execute the estimation process estimating the recovery degree of the patient from the eyeball movement feature using a pre-trained recovery degree estimation model; and Output the degree of recovery; Cause a computer to execute a process of storing a recovery record including one or more of the patient's past recovery degree history and rehabilitation content; The estimation process estimates the recovery degree of the patient from the recovery record and the eye movement characteristics.

Advantages of the Invention

[0013] ​​​ According to the present invention, it is possible to quantitatively estimate the degree of recovery without imposing a burden on patients or medical staff.

Brief Description of the Drawings

[0014]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Modes for Carrying Out the Invention

[0015] Hereinafter, embodiments of the present invention will be described with reference to the drawings. [First Embodiment] (Configuration) Figure 1 shows a schematic configuration of a recovery degree estimation device according to a first embodiment of the present invention. The recovery degree estimation device 1 is connected to a camera 2. The camera 2 images the eyes of a patient (hereinafter simply referred to as "patient") who is the target of recovery degree estimation, and transmits a captured image D1 to the recovery degree estimation device 1. The camera 2 is assumed to use a high-speed camera that can image the eyes at a high speed, such as 1000 frames per second. The recovery degree estimation device 1 estimates the recovery degree of the patient by analyzing the captured image D1 and calculating an estimated recovery degree.

[0016] Figure 2 is a block diagram showing the hardware configuration of the recovery degree estimation device 1. As shown in the figure, the recovery degree estimation device 1 includes an interface 11, a processor 12, a memory 13, a recording medium 14, a display unit 15, and an input unit 16.

[0017] The interface 11 exchanges data with the camera 2. The interface 11 is used when receiving the captured image D1 generated by the camera 2. Also, the interface 11 is used when the recovery degree estimation device 1 exchanges data with a predetermined device connected by wire or wirelessly.

[0018] The processor 12 is a computer such as a CPU (Central Processing Unit), and controls the entire recovery degree estimation device 1 by executing a program prepared in advance. The memory 13 is composed of a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The memory 13 stores a program executed by the processor 12. Also, the memory 13 is used as a working memory during the execution of various processes by the processor 12.

[0019] The recording medium 14 is a non-volatile and non-temporary recording medium such as a disk-shaped recording medium or a semiconductor memory, and is configured to be detachable from the recovery degree estimation device 1. The recording medium 14 stores various programs executed by the processor 12. When the recovery degree estimation device 1 executes the recovery degree estimation process, the program recorded on the recording medium 14 is loaded into the memory 13 and executed by the processor 12.

[0020] The display unit 15 is, for example, an LCD (Liquid Crystal Display) or the like, and displays the estimated recovery degree and the like, which are the results of estimating the patient's recovery degree. Note that the display unit 15 may display the tasks of the third embodiment described later. The input unit 16 is used by operators such as medical staff and experts, such as a keyboard, a mouse, and a touch panel.

[0021] FIG. 3 is a block diagram showing the functional configuration of the recovery degree estimation device 1. Functionally, the recovery degree estimation device 1 includes an eye movement feature storage unit 21, a recovery degree estimation model update unit 22, a recovery degree correct answer information storage unit 23, a recovery degree estimation model storage unit 24, an image acquisition unit 25, an eye movement feature extraction unit 26, a recovery degree estimation unit 27, and an alert output unit 28. Note that the recovery degree estimation model update unit 22, the image acquisition unit 25, the eye movement feature extraction unit 26, the recovery degree estimation unit 27, and the alert output unit 28 are realized by the processor 12 executing a program. Further, the eye movement feature storage unit 21, the recovery degree correct answer information storage unit 23, and the recovery degree estimation model storage unit 24 are realized by the memory 13.

[0022] The recovery degree estimation device 1 generates and updates a recovery degree estimation model that learns the relationship between the eye movement characteristics of a patient and the recovery degree with reference to the eye movement. Specifically, the recovery degree estimation device 1 can be applied, for example, to the estimation of the recovery degree by rehabilitation from sequelae caused by cerebral infarction. As the learning algorithm, for example, any machine learning method such as a neural network, SVM (Support Vector Machine), or logistic regression can be used. Further, the recovery degree estimation device 1 estimates the recovery degree by calculating the estimated recovery degree of the patient from the eye movement characteristics of the patient using the recovery degree estimation model.

[0023] The eye movement feature storage unit 21 stores the eye movement features used as input data in the learning of the recovery degree estimation model. FIG. 4 shows an example of the eye movement features. The eye movement features are the features of human eye movement, and examples thereof include nystagmus information, deviation of the movement direction, deviation of the left and right movements, and visual field defect information.

[0024] As shown in FIG. 4(A), the nystagmus information is information related to the vibration of the eyes. Based on the nystagmus information, for example, abnormalities such as nystagmus caused by cerebral infarction can be detected. Specifically, the nystagmus information may be information regarding the xy coordinates of one point at the center of the pupil for each of the left and right eyes and the time-series change of the coordinates, or may be frequency information extracted by FFT (Fast Fourier Transform) conversion of the xy coordinates within an arbitrary time interval. Further, it may be information regarding the appearance frequency within a predetermined time of a predetermined movement such as microsaccade movement.

[0025] As shown in FIG. 4(B), the bias in the moving direction is information regarding the bias in the vertical or horizontal movement of the eyeball. Based on the bias in the moving direction, for example, abnormalities such as fixation paralysis caused by cerebral infarction can be detected. Specifically, the variance of the x-direction component and the variance of the y-direction component of the position (x, y) are calculated, and the ratio of the variances is used for determination. Alternatively, the variance of the x-direction component and the variance of the y-direction component of the temporal difference of the position of the velocity information are calculated, and the ratio of the variances is used for determination, thereby obtaining information regarding the quantitative bias in the moving direction. Further, the bias in the moving direction may be obtained by determining it based on the principal moment of inertia of the (x, y) position information or the contribution rate of the first principal component.

[0026] As shown in FIG. 4(C), the deviation in the left-right movement is information regarding the deviation in the eye movement of the left and right eyeballs. Based on the deviation in the left-right movement, for example, abnormalities such as strabismus caused by cerebral infarction can be detected. Specifically, the value obtained by integrating the angle formed by the moving directions of the left and right eyeballs respectively over the time axis is calculated, and it is determined that the larger the value, the larger the deviation. Alternatively, it is determined that the smaller the value obtained by integrating the inner product of the angles formed by the moving directions of the left and right eyeballs respectively, the larger the deviation, thereby obtaining information regarding the quantitative deviation in the left-right movement.

[0027] As shown in FIG. 4(D), the visual field defect information is information regarding the defect in the visual field. Based on the visual field defect information, for example, abnormalities such as fixation disorder caused by cerebral infarction can be detected. Specifically, by tracking a light spot or the like presented to the patient and calculating the area of the location where tracking failure occurs frequently, or by dividing the light spot display area into a virtual grid pattern and counting the number of grids with a high frequency of tracking failure, quantitative visual field defect information can be obtained.

[0028] The recovery degree correct answer information storage unit 23 stores correct answer information (correct answer labels) used in the learning process for learning the recovery degree estimation model. Specifically, the recovery degree correct answer information storage unit 23 stores the correct answer information of the recovery degree for each eye movement feature stored in the eye movement feature storage unit 21. The recovery degree can be arbitrarily applied, for example, BBS (Berg Balance Scale), TUG (Timed Up and Go test), FIM (Functional Independence Measure), etc.

[0029] The recovery degree estimation model update unit 22 learns the recovery degree estimation model using the pre-prepared learning data. Here, the learning data includes input data and correct answer data. The eye movement features stored in the eye movement feature storage unit 21 are used as the input data, and the correct answer information of the recovery degree stored in the recovery degree correct answer information storage unit 23 is used as the correct answer data. Specifically, the recovery degree estimation model update unit 22 acquires the eye movement features from the eye movement feature storage unit 21, and acquires the correct answer information of the recovery degree corresponding to the eye movement features from the recovery degree correct answer information storage unit 23. Next, the recovery degree estimation model update unit 22 calculates the estimated recovery degree of the patient from the acquired eye movement features using the recovery degree estimation model, and collates it with the correct answer information of the recovery degree. Then, the recovery degree estimation model update unit 22 updates the recovery degree estimation model so that the error between the recovery degree calculated by the recovery degree estimation model and the correct answer information of the recovery degree becomes small. The recovery degree estimation model update unit 22 overwrites and stores the updated recovery degree estimation model with improved estimation accuracy of the recovery degree in the recovery degree estimation model storage unit 24.

[0030] The recovery degree estimation model storage unit 24 stores the recovery degree estimation model learned and updated by the recovery degree estimation model update unit 22.

[0031] The image acquisition unit 25 acquires a captured image D1 of the patient's eyes supplied from the camera 2. When the captured image D1 captured by the camera 2 is collected and stored in a database or the like, the image acquisition unit 25 may acquire the captured image D1 from the database or the like.

[0032] The eye movement feature extraction unit 26 performs predetermined image processing on the captured image D1 acquired by the image acquisition unit 25, and extracts the eye movement features of the patient. Specifically, the eye movement feature extraction unit 26 extracts the time-series information of the vibration pattern of the eye in the captured image D1 as the eye movement features.

[0033] The recovery degree estimation unit 27 calculates the estimated recovery degree of the patient from the eye movement features extracted by the eye movement feature extraction unit 26 using the recovery degree estimation model. The calculated estimated recovery degree is stored in the memory 13 or the like in association with the information about the patient.

[0034] The alert output unit 28 refers to the memory 13 or the like, and when the estimated recovery degree of the patient deteriorates from the threshold value, outputs an alert to the patient to the display unit 15. The alert may be set for a period and output when the estimated recovery degree of the patient deteriorates from the threshold value within a predetermined period.

[0035] (Learning process) Next, the learning process by the recovery degree estimation device 1 will be described. FIG. 5 is a flowchart of the learning process by the recovery degree estimation device 1. This process is realized by the processor 12 shown in FIG. 2 executing a program prepared in advance.

[0036] First, the recovery degree estimation device 1 acquires the eye movement features from the eye movement feature storage unit 21, and acquires the correct answer information of the recovery degree for the eye movement features from the recovery degree correct answer information storage unit 23 (step S101). Next, the recovery degree estimation device 1 calculates the estimated recovery degree of the patient from the acquired eye movement features using the recovery degree estimation model, and collates it with the correct answer information of the recovery degree (step S102). Then, the recovery degree estimation device 1 updates the recovery degree estimation model so that the error between the estimated recovery degree calculated by the recovery degree estimation model and the correct answer information of the recovery degree becomes small (step S103). The recovery degree estimation device 1 updates the recovery degree estimation model so as to improve the estimation accuracy by repeating this process with different learning data.

[0037] (Recovery degree estimation process) Next, the recovery degree estimation process by the recovery degree estimation device 1 will be described. FIG. 6 is a flowchart of the recovery degree estimation process by the recovery degree estimation device 1. This process is realized by the processor 12 shown in FIG. 2 executing a program prepared in advance.

[0038] First, the recovery degree estimation device 1 acquires a captured image D1 of the patient's eyeball (step S201). Next, the recovery degree estimation device 1 extracts eye movement features from the acquired captured image D1 by image processing (step S202). Next, the recovery degree estimation device 1 calculates the estimated recovery degree of the patient from the extracted eye movement features using the recovery degree estimation model (step S203). The estimated recovery degree is presented to the patient, medical staff, etc. by any method. In this way, since the recovery degree estimation device 1 can estimate the patient's recovery degree even without the presence of medical staff or experts based on the captured image D1 of the eyeball, the burden on medical staff, etc. can be reduced. Also, since the daily recovery degree can be predicted even while sitting, it can be applied to patients who do not require hospital visits or are at risk of falling and have difficulty walking independently.

[0039] Note that the recovery degree estimation device 1 may store the calculated estimated recovery degree in the memory 13 or the like for each patient, and when the estimated recovery degree of the patient deteriorates from the threshold value, output an alert to the patient to the display unit 15 or the like.

[0040] As described above, according to the recovery degree estimation device 1 of the first embodiment, it is easy for the patient to quantitatively measure the estimated recovery degree every day at home or the like, and the daily recovery degree can be made objectively visible. Therefore, effects such as an increase in the amount of rehabilitation accompanying an improvement in the patient's rehabilitation motivation and an improvement in the quality of rehabilitation due to frequent modification of the rehabilitation plan can be expected, and the recovery effect can be improved. Also, it becomes possible to detect abnormalities such as signs of recurrence of cerebral infarction at an early stage without waiting for inspections or diagnoses by medical staff. Examples of industrial use of the recovery degree estimation device 1 include remote rehabilitation guidance and management.

[0041] [Second Embodiment] (Configuration) When estimating the recovery degree of a patient, the recovery degree estimation device 1x according to the second embodiment uses patient information about the patient, such as attributes and recovery records, in addition to eye movement characteristics. Note that since the schematic configuration and hardware configuration of the recovery degree estimation device are the same as those of the first embodiment, the description thereof is omitted.

[0042] FIG. 7 is a block diagram showing the functional configuration of the recovery degree estimation device 1x. Functionally, the recovery degree estimation device 1x includes an eye movement feature storage unit 31, a recovery degree estimation model update unit 32, a recovery degree correct answer information storage unit 33, a recovery degree estimation model storage unit 34, an image acquisition unit 35, an eye movement feature extraction unit 36, a recovery degree estimation unit 37, an alert output unit 38, and a patient information storage unit 39. Note that the recovery degree estimation model update unit 32, the image acquisition unit 35, the eye movement feature extraction unit 36, the recovery degree estimation unit 37, and the alert output unit 38 are realized by the processor 12 executing a program. Also, the eye movement feature storage unit 31, the recovery degree correct answer information storage unit 33, the recovery degree estimation model storage unit 34, and the patient information storage unit 39 are realized by the memory 13.

[0043] The recovery degree estimation device 1x according to the second embodiment generates and updates a recovery degree estimation model for estimating the recovery degree based on the eye movement characteristics and patient information of the patient. As the learning algorithm, for example, any machine learning method such as a neural network, SVM, or logistic regression may be used. Also, the recovery degree estimation device 1x estimates the recovery degree by calculating the estimated recovery degree of the patient from the eye movement characteristics and patient information of the patient using the recovery degree estimation model.

[0044] The patient information storage unit 39 stores patient information about the patient. The patient information is, for example, attributes such as gender and age, a history of recovery degree, a disease name, symptoms, and past patient recovery records such as records of rehabilitation content. The patient information storage unit 39 stores the patient information in association with the identification information of the patient.

[0045] The recovery degree correct answer information storage unit 33 stores the correct answer information of the recovery degree corresponding to the combination of the patient information and the eye movement characteristics.

[0046] Based on the pre-prepared learning data, the recovery degree estimation model updating unit 32 learns and updates the recovery degree estimation model. Here, the learning data includes input data and correct answer data. In the second embodiment, the eye movement features stored in the eye movement feature storage unit 31 and the patient information stored in the patient information storage unit 39 are used as the input data. In the recovery degree correct answer information storage unit 33, the correct answer information of the recovery degree corresponding to the combination of the eye movement features and the patient information is stored, and this is used as the correct answer data.

[0047] Specifically, the recovery degree estimation model updating unit 32 acquires the eye movement features from the eye movement feature storage unit 31 and acquires the patient information from the patient information storage unit 39. Also, the recovery degree estimation model updating unit 32 acquires the correct answer information of the recovery degree corresponding to the acquired patient information and eye movement features from the recovery degree correct answer information storage unit 33. Next, the recovery degree estimation model updating unit 32 calculates the estimated recovery degree of the patient from the eye movement features and the patient information using the recovery degree estimation model, and collates it with the correct answer information of the recovery degree. Then, the recovery degree estimation model updating unit 32 updates the recovery degree estimation model so that the error between the recovery degree calculated by the recovery degree estimation model and the correct answer information of the recovery degree becomes small. The updated recovery degree estimation model is stored in the recovery degree estimation model storage unit 34.

[0048] The recovery degree estimation unit 37 acquires the patient information of a certain patient from the patient information storage unit 39 and acquires the eye movement features of that patient from the eye movement feature extraction unit 36. Then, the recovery degree estimation unit 37 calculates the estimated recovery degree of the patient from the eye movement features and the patient information using the recovery degree estimation model. The calculated estimated recovery degree is stored in the memory 13 or the like in association with the identification information of the patient.

[0049] Note that since the eye movement feature storage unit 31, the recovery degree estimation model storage unit 34, the image acquisition unit 35, the eye movement feature extraction unit 36, and the alert output unit 38 are the same as those in the first embodiment, the description thereof is omitted.

[0050] (Learning process) Next, the learning process by the recovery degree estimation device 1x will be described. FIG. 8 is a flowchart of the learning process by the recovery degree estimation device 1x. This process is realized by the processor 12 shown in FIG. 2 executing a program prepared in advance.

[0051] First, the recovery degree estimation device 1x acquires the patient information of a certain patient from the patient information storage unit 39 and acquires the eye movement characteristics of that patient from the eye movement characteristic storage unit 31 (step S301). Next, the recovery degree estimation device 1x acquires the correct answer information of the recovery degree for the patient information and the eye movement characteristics from the recovery degree correct answer information storage unit 33 (step S302). Next, the recovery degree estimation device 1x calculates the estimated recovery degree of the patient from the eye movement characteristics and the patient information and collates it with the correct answer information of the recovery degree (step S303). Then, the recovery degree estimation device 1x updates the recovery degree estimation model so that the error between the estimated recovery degree calculated by the recovery degree estimation model and the correct answer information of the recovery degree becomes small (step S304). The recovery degree estimation device 1x updates the recovery degree estimation model so as to improve the estimation accuracy by repeating this process with different learning data.

[0052] (Recovery degree estimation process) Next, the recovery degree estimation process by the recovery degree estimation device 1x will be described. FIG. 9 is a flowchart of the recovery degree estimation process by the recovery degree estimation device 1x. This process is realized by the processor 12 shown in FIG. 2 executing a program prepared in advance.

[0053] First, the recovery degree estimation device 1x acquires the captured image D1 of the patient's eyes (step S401). Next, the recovery degree estimation device 1x extracts the eye movement characteristics from the acquired captured image D1 by image processing (step S402). Next, the recovery degree estimation device 1x acquires the patient information of that patient from the patient information storage unit 39 (step S403). Next, the recovery degree estimation device 1x calculates the estimated recovery degree of the patient from the extracted eye movement characteristics and the acquired patient information using the recovery degree estimation model (step S404). Then, the process ends. The estimated recovery degree is presented to the patient, medical staff, etc. by any method.

[0054] Note that the recovery degree estimation device 1x may store the calculated estimated recovery degree in a memory 13 or the like for each patient, and output an alert to the patient to the display unit 15 or the like when the estimated recovery degree of the patient deteriorates from a threshold value.

[0055] As described above, according to the recovery degree estimation device 1x of the second embodiment, since a recovery degree estimation model for estimating the recovery degree based on the eye movement characteristics and patient information is used, it is possible to estimate the recovery degree in consideration of the individuality and characteristics of each patient.

[0056] [Third Embodiment] (Configuration) When imaging the patient's eyes, the recovery degree estimation device 1y of the third embodiment presents a task. The task is a predetermined condition or problem related to eye movement. By presenting a task to the patient when imaging the eyes, the recovery degree estimation device 1y can capture an image from which it is easy to extract the eye movement characteristics necessary for estimating the recovery degree.

[0057] Note that, unlike the first and second embodiments, the recovery degree estimation device 1y of the third embodiment is assumed to incorporate a camera 2. Since the interface 11, the processor 12, the memory 13, the recording medium 14, the display unit 15, and the input unit 16 are the same as those in the first and second embodiments, the description thereof is omitted.

[0058] FIG. 10 is a block diagram showing the functional configuration of the recovery degree estimation device 1y. Functionally, the recovery degree estimation device 1y includes an eye movement feature storage unit 41, a recovery degree estimation model update unit 42, a recovery degree correct answer information storage unit 43, a recovery degree estimation model storage unit 44, an image acquisition unit 45, an eye movement feature extraction unit 46, a recovery degree estimation unit 47, an alert output unit 48, and a task presentation unit 49. Note that the recovery degree estimation model update unit 42, the image acquisition unit 45, the eye movement feature extraction unit 46, the recovery degree estimation unit 47, the alert output unit 48, and the task presentation unit 49 are realized by the processor 12 executing a program. Further, the eye movement feature storage unit 41, the recovery degree correct answer information storage unit 43, and the recovery degree estimation model storage unit 44 are realized by the memory 13.

[0059] The recovery degree estimation device 1y generates and updates a recovery degree estimation model that learns the relationship between the patient's eye movement characteristics and the recovery degree with reference to the eye movement. As the learning algorithm, for example, any machine learning method such as a neural network, SVM, or logistic regression may be used. Further, the recovery degree estimation device 1y presents a task related to eye movement to the patient, and acquires a captured image D1 obtained by imaging the eyes of the patient to whom the task has been presented. Then, the recovery degree estimation device 1y estimates the recovery degree by calculating the estimated recovery degree of the patient from the eye movement characteristics of the patient based on the acquired captured image D1 using the recovery degree estimation model.

[0060] The task presentation unit 49 presents a task for the patient on the display unit 15. The task is a predetermined condition or task related to eye movement, and can be arbitrarily set, for example, "look at a predetermined video with changes", "track a moving light point with the eyes", etc.

[0061] FIG. 11 is a specific example of the task "track a moving light point with the eyes". In the light point display area 50 shown in FIG. 11, the black circles are light points, and they move to the square 51 at the elapsed time of 1 second (t = 1), to the square 52 at the elapsed time of 2 seconds (t = 2), to the square 53 at the elapsed time of 3 seconds (t = 3), to the square 54 at the elapsed time of 4 seconds (t = 4), to the square 55 at the elapsed time of 5 seconds (t = 5), and to the square 56 at the elapsed time of 6 seconds (t = 6). The patient tracks the moving light point with the eyes. By presenting such a task, the camera 2 incorporated in the recovery degree estimation device 1y can easily capture an image including the patient's visual field defect information.

[0062] The image acquisition unit 45 acquires the captured image D1 by imaging the eyes of the patient that have moved along with the task by the camera 2 incorporated in the recovery degree estimation device 1y.

[0063] Note that the eye movement feature memory unit 41, the recovery degree estimation model update unit 42, the recovery degree correct answer information memory unit 43, the recovery degree estimation model memory unit 44, the eye movement feature extraction unit 46, the recovery degree estimation unit 47, and the alert output unit 48 are the same as those in the first embodiment, and thus the description thereof is omitted. Also, the learning process by the recovery degree estimation device 1y is the same as that in the first embodiment, and thus the description thereof is omitted.

[0064] (Recovery Degree Estimation Process) Next, the recovery degree estimation process by the recovery degree estimation device 1y will be described. FIG. 12 is a flowchart of the recovery degree estimation process by the recovery degree estimation device 1y. This process is realized by the processor 12 shown in FIG. 2 executing a program prepared in advance.

[0065] First, the recovery degree estimation device 1y presents a task to the patient using the display unit 15 or the like (step S501). Then, the recovery degree estimation device 1y captures the eyes of the patient to whom the task has been presented with the camera 2 and obtains a captured image D1 (step S502). Further, the recovery degree estimation device 1y extracts eye movement features from the obtained captured image D1 by image processing (step S503). Next, the recovery degree estimation device 1y calculates the estimated recovery degree of the patient from the extracted eye movement features using the recovery degree estimation model (step S504). The estimated recovery degree is presented to the patient, medical staff, etc. by any method. By presenting such a predetermined task, the recovery degree estimation device 1y can obtain a captured image D1 from which it is easy to extract eye movement features.

[0066] Note that the recovery degree estimation device 1y may store the calculated estimated recovery degree in the memory 13 or the like for each patient, and output an alert to the patient using the display unit 15 or the like when the estimated recovery degree of the patient deteriorates from the threshold value.

[0067] In the third embodiment, for convenience of explanation, the recovery degree estimation device 1y incorporates the camera 2 and presents tasks on the display unit 15. However, the present invention is not limited to this, and the recovery degree estimation device may not incorporate the camera 2 and may be connected to the camera 2 by wire or wirelessly as long as data can be exchanged. In this case, the recovery degree estimation device 1y outputs tasks for the patient to the camera 2 and acquires the captured image D1 captured by the camera 2.

[0068] Also, the recovery degree estimation device 1y in the third embodiment may use patient information in the same manner as the recovery degree estimation model described in the second embodiment. Further, the recovery degree estimation device 1 in the first embodiment and the recovery degree estimation device 1x in the second embodiment may present the tasks described in this embodiment.

[0069] [Fourth Embodiment] FIG. 13 is a block diagram showing the functional configuration of the recovery degree estimation device according to the fourth embodiment. The recovery degree estimation device 60 includes an image acquisition unit 61, an eye movement feature extraction unit 62, and a recovery degree estimation unit 63.

[0070] FIG. 14 is a flowchart of the recovery degree estimation process by the recovery degree estimation device 60. The image acquisition unit 61 acquires an image of the patient's eyes (step S601). The eye movement feature extraction unit 62 extracts eye movement features, which are features of eye movement, based on the image (step S602). The recovery degree estimation unit 63 estimates the patient's recovery degree from the eye movement features using a pre-trained recovery degree estimation model (step S603).

[0071] According to the recovery degree estimation device 60 of the fourth embodiment, the recovery degree of a patient with a predetermined disease can be estimated based on an image of the patient's eyes.

[0072] Some or all of the above embodiments may be described as follows in the following supplementary notes, but are not limited thereto.

[0073] [Supplementary Note 1] Image acquisition means for acquiring an image of a patient's eyeball; Eyeball movement feature extraction means for extracting an eyeball movement feature, which is a feature of the eyeball movement, based on the image; Degree of recovery estimation means for estimating the degree of recovery of the patient from the eyeball movement feature using a pre-trained degree of recovery estimation model; A degree of recovery estimation device comprising the above.

[0074] (Appendix 2) The degree of recovery estimation device according to Appendix 1, wherein the eyeball movement feature includes eyeball vibration information related to the vibration of the eyeball.

[0075] (Appendix 3) The degree of recovery estimation device according to Appendix 1 or 2, wherein the eyeball movement feature includes information related to at least one of a bias in the moving direction of the eyeball and a deviation in the left-right movement of the eyeball.

[0076] (Appendix 4) Further comprising task presentation means for presenting a task related to the eyeball movement to the patient, wherein the image acquisition means acquires an image of the patient's eyeball when the task is presented, and the eyeball movement feature extraction means extracts an eyeball movement feature in the task based on the image. The degree of recovery estimation device according to any one of Appendices 1 to 3.

[0077] (Appendix 5) The degree of recovery estimation device according to Appendix 4, wherein the eyeball movement feature includes visual field defect information related to a visual field defect.

[0078] (Appendix 6) Comprising patient information storage means for storing patient information related to at least one of the patient's attributes and the patient's past recovery records, wherein the degree of recovery estimation means estimates the degree of recovery of the patient from the patient information and the eyeball movement feature. The degree of recovery estimation device according to Appendix 1.

[0079] (Appendix 7) The recovery degree estimation device according to any one of Appendices 1 to 6, comprising alert output means for outputting an alert when the recovery degree of the patient deteriorates from a threshold value.

[0080] (Appendix 8) Obtain an image of the patient's eyeball, Based on the image, extract the eye movement characteristics, which are the characteristics of eye movement, A recovery degree estimation method for estimating the recovery degree of a patient from the eye movement characteristics using a recovery degree estimation model pre-trained by machine learning.

[0081] (Appendix 9) Obtain an image of the patient's eyeball, Based on the image, extract the eye movement characteristics, which are the characteristics of eye movement, A recording medium recording a program for causing a computer to execute a process of estimating the recovery degree of a patient from the eye movement characteristics using a recovery degree estimation model pre-trained by machine learning.

[0082] Although the present invention has been described with reference to the embodiments and examples above, the present invention is not limited to the above embodiments and examples. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

Description of Reference Numerals

[0083] 1, 1x, 1y Recovery degree estimation device 2 Camera 11 Interface 12 Processor 13 Memory 14 Recording medium 15 Display unit 16 Input unit 21, 31, 41 Eye movement feature storage unit 22, 32, 42 Recovery degree estimation model update unit 23, 33, 43 Recovery degree correct answer information storage unit 24, 34, 44 Recovery degree estimation model storage unit 25, 35, 45 Image acquisition unit 26, 36, 46 Ocular Movement Feature Extraction Unit 27, 37, 47 Recovery Degree Estimation Unit 28, 38, 48 Alert Output Unit 39 Patient Information Storage Unit 49 Task Presentation Unit

Claims

1. Image acquisition means for acquiring an image of a patient's eyeball; Eye movement feature extraction means for extracting an eye movement feature, which is a feature of eye movement, based on the image; Recovery degree estimation means for estimating the recovery degree of the patient from the eye movement feature using a pre-trained recovery degree estimation model; Output means for outputting the recovery degree; Patient information storage means for storing a recovery record including any one or more of the patient's past recovery degree history and rehabilitation content; and The recovery degree estimation means is a recovery degree estimation device that estimates the recovery degree of the patient from the recovery record and the eye movement feature.

2. The recovery degree estimation device according to claim 1, wherein the eye movement feature includes eye movement information related to the vibration of the eyeball.

3. The recovery degree estimation device according to claim 1 or 2, wherein the eye movement feature includes information related to any one or more of the bias in the moving direction of the eyeball and the deviation in the left-right movement of the eyeball.

4. Further comprising task presentation means for presenting a task related to the eye movement to the patient, The image acquisition means acquires an image of the patient's eyeball when the task is presented, The eye movement feature extraction means extracts an eye movement feature in the task based on the image, according to any one of claims 1 to 3.

5. The recovery degree estimation device according to claim 4, wherein the eye movement feature includes visual field defect information related to a visual field defect.

6. The patient information storage means further stores the attributes of the patient, The recovery degree estimation means estimates the recovery degree of the patient from the attributes of the patient and the eye movement feature, according to claim 1.

7. The recovery degree estimation device according to any one of claims 1 to 6, comprising alert output means for outputting an alert when the recovery degree of the patient deteriorates from a threshold value.

8. Executed by a computer, Acquire an image of a patient's eyeball, Extract an eye movement feature, which is a feature of eye movement, based on the image, Execute an estimation step of estimating the recovery degree of the patient from the eye movement feature using a pre-trained recovery degree estimation model, Output the recovery degree, Store a recovery record including any one or more of the patient's past recovery degree history and rehabilitation content, The estimation step is a recovery degree estimation method for estimating the recovery degree of the patient from the recovery record and the eye movement feature.

9. Obtain an image of the patient's eyeball, Based on the image, extract the eye movement characteristics, which are the characteristics of eye movement, Execute an estimation step of estimating the patient's recovery degree from the eye movement characteristics using a pre-trained recovery degree estimation model, Output the recovery degree, Cause the computer to execute a process of storing a recovery record including one or more of the patient's past recovery degree history and rehabilitation content, The estimation step is a program for estimating the patient's recovery degree from the recovery record and the eye movement characteristics.

Images