Information processing device, learning model creation device, information processing method, learning model creation method, and program

Abstract

For the subjects, information indicating the sphericity of the eye is obtained from fundus image data. [Solution] The information processing device comprises a receiving unit that receives fundus image information of a subject, a trained model that has learned the relationship between fundus image information of the eye and information indicating the sphericality of the eye, a processing unit that acquires information indicating the sphericality of the subject's eye based on the fundus image information of the subject received by the receiving unit, and an output unit that outputs the information indicating the sphericality of the subject's eye acquired by the processing unit.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, a learning model creation apparatus, an information processing method, a learning model creation method, and a program.

Background Art

[0002] It has been reported that values of biomarkers such as age, gender, smoking history, HbA1C, etc. can be predicted from fundus images, and many studies have been conducted thereafter. Techniques for estimating refractive errors from retinal fundus photographs are known (see, for example, Non-Patent Document 1). Also, techniques for estimating axial length based on fundus images are known (see, for example, Non-Patent Document 2).

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Non-Patent Document 2

[0004] Sphericity is typically measured using specialized equipment, based on the principle of measuring the magnification of the optical system combining the optical system of the eye and the measuring instrument. However, this test is generally not performed in routine health checkups, for example. Optical coherence tomography (OCT) can be used to determine the thickness of the retina from optical cross-sectional images of the retina, quantify the shape of the fundus, and then determine the sphericalness of the eye based on the resulting quantification of the fundus shape. However, determining the sphericalness of the eye is a time-consuming process. The development of multimodal AI (Artificial Intelligence) has made it possible to predict target variables from various parameters. Furthermore, explainable AI technology, which can show the parameters considered during the prediction process, is also entering the practical application stage. The present invention aims to provide an information processing device, a learning model creation device, an information processing method, a learning model creation method, and a program that can acquire information indicating the sphericity of the eye from fundus image information of a subject. This method is useful because it allows for the acquisition of fundus images for examinations such as diabetes and glaucoma during health checkups, and enables the determination of sphericity during these examinations. [Means for solving the problem]

[0005] (1) One aspect of the present invention is an information processing device comprising: a receiving unit for receiving fundus image information of a subject; a trained model that has been machine-learned to determine the relationship between fundus image information of the eye and information indicating the sphericality of the eye; a processing unit that acquires information indicating the sphericality of the subject's eye based on the fundus image information of the subject received by the receiving unit; and an output unit that outputs the information indicating the sphericality of the subject's eye acquired by the processing unit.

[0006] (2) In an information processing apparatus according to one embodiment of the present invention, the processing unit acquires information indicating the axial length of the eye of the subject based on a trained model that has been further trained by machine learning to obtain the relationship between the fundus image of the eye and the axial length of the eye, and the fundus image information of the subject received by the reception unit, and the output unit outputs the information indicating the axial length of the eye of the subject acquired by the processing unit.

[0007] (3) An information processing device according to one embodiment of the present invention further comprises an extraction unit that extracts a fundus image of a predetermined region including the macula and optic nerve head from a fundus image based on the fundus image information of a subject, and the processing unit acquires information indicating the sphericity of the subject's eye based on the trained model and the fundus image information of the predetermined region extracted by the extraction unit.

[0008] (4) In an information processing device according to one aspect of the present invention, the trained model includes a first trained model that has been trained to learn the relationship between fundus image information of the right eye and information indicating the sphericity of the right eye, and a second trained model that has been trained to learn the relationship between fundus image information of the left eye and information indicating the sphericity of the left eye.

[0009] (5) One aspect of the present invention is a learning model creation device comprising: a reception unit that receives a learning dataset in which fundus image information of the eye is included as learning data and information indicating the sphericality of the eye is included as training data; a processing unit that creates a learning model by machine learning the relationship between fundus image information of the eye and information indicating the sphericality of the eye, with fundus image information of the eye as an explanatory variable and information indicating the sphericality of the eye as an objective variable, based on the learning dataset received by the reception unit; and an output unit that outputs the learning model created by the processing unit.

[0010] (6) In an information processing apparatus according to one embodiment of the present invention, the receiving unit receives a training dataset which includes fundus image information of the eye as training data and further includes information indicating the axial length of the eye as training data, and the processing unit creates a learning model based on the training dataset received by the receiving unit, using the fundus image information of the eye as an explanatory variable and the information indicating the axial length of the eye as an objective variable, by performing machine learning on the relationship between the fundus image information of the eye and the information indicating the axial length of the eye.

[0011] (7) One aspect of the present invention is an information processing method performed by a computer, which receives fundus image information of a subject, acquires information indicating the sphericality of the subject's eye based on a trained model that has learned the relationship between fundus image information of the eye and information indicating the sphericality of the eye, and the received fundus image information of the subject, and outputs the acquired information indicating the sphericality of the subject's eye.

[0012] (8) One aspect of the present invention is a method for creating a learning model executed by a computer, the method comprising: receiving a learning dataset which includes fundus image information of an eye as learning data and information indicating the sphericity of the eye as training data; creating a learning model by machine learning the relationship between fundus image information of an eye and information indicating the sphericity of the eye, using the fundus image information of an eye as an explanatory variable and the information indicating the sphericity of the eye as an objective variable, and outputting the created learning model.

[0013] (9) One aspect of the present invention is a program that causes a computer to receive fundus image information of a subject, acquires information indicating the sphericality of the subject's eye based on a trained model that has learned the relationship between fundus image information of the eye and information indicating the sphericality of the eye, and the fundus image information of the subject that has been received, and outputs the acquired information indicating the sphericality of the subject's eye.

[0014] (10) One aspect of the present invention is a program that causes a computer to accept a training dataset in which fundus image information of an eye is included as training data and information indicating the sphericality of the eye is included as training data, and based on the accepted training dataset, causes the computer to create a learning model by machine learning the relationship between fundus image information of an eye and information indicating the sphericality of the eye, with fundus image information of an eye as an explanatory variable and information indicating the sphericality of the eye as an objective variable, and outputs the created learning model. [Effects of the Invention]

[0015] According to the present invention, it is possible to provide an information processing device, a learning model creation device, an information processing method, a learning model creation method, and a program that can acquire information indicating the sphericity of the eye from fundus image information for a subject. [Brief explanation of the drawing]

[0016] [Figure 1] This figure shows an example of an information processing device according to this embodiment. [Figure 2] This figure shows an example of a fundus image of the eye. [Figure 3] This is a flowchart showing an example of the operation of the information processing device according to the embodiment. [Figure 4] This figure shows an example of a learning model creation device according to this embodiment. [Figure 5] This flowchart shows an example of the operation of the learning model creation device of this embodiment. [Figure 6] This figure shows an example of an information processing device, modified example 1 of the embodiment. [Figure 7]It is a flowchart showing an example of the operation of the information processing apparatus according to the first modification of the embodiment. [Figure 8] It is a diagram showing an example of the information processing apparatus according to the second modification of the embodiment. [Figure 9] It is a flowchart showing an example of the operation of the information processing apparatus according to the second modification of the embodiment. [Figure 10] It is a diagram showing an example of the learning model creation apparatus according to the second modification of the embodiment. [Figure 11] It is a flowchart showing an example of the operation of the learning model creation apparatus according to the second modification of the embodiment.

Mode for Carrying Out the Invention

[0017] Hereinafter, an information processing apparatus, a learning model creation apparatus, an information processing method, a learning model creation method, and a program according to the embodiment will be described with reference to the drawings. The embodiments described below are merely examples, and the embodiments to which the present invention is applied are not limited to the following embodiments. In all the drawings for explaining the embodiments, those having the same function are denoted by the same reference numerals, and repeated explanations are omitted. In addition, “based on XX” as used in the present application means “at least based on XX”, and includes cases where it is based on another element in addition to XX. Also, “based on XX” is not limited to the case where XX is directly used, and includes cases where it is based on something obtained by performing an operation or processing on XX. “XX” is an arbitrary element (for example, arbitrary information).

[0018] (Embodiment) (Information Processing Apparatus) FIG. 1 is a diagram showing an example of the information processing apparatus according to the present embodiment. The information processing apparatus 100 according to the present embodiment receives subject-related information. The subject-related information includes subject identification information and fundus image information of the subject's eyes.

[0019] The information processing device 100 acquires information indicating the sphericity of the subject's eye based on the fundus image information of the eye included in the received subject-related information and a trained model. Here, the trained model is one that has been machine-learned to determine the relationship between the fundus image information of the eye and the information indicating the sphericity of the eye. The information processing device 100 outputs subject identification information and information indicating the sphericality of the subject's eye.

[0020] The information processing device 100 is implemented by a device such as a personal computer, server, smartphone, tablet computer, or industrial computer. The information processing device 100 comprises an input unit 102, a reception unit 104, a processing unit 106, an output unit 108, and a storage unit 110. The input unit 102 receives information. For example, the input unit 102 may have an operating unit such as a keyboard and a mouse. In this case, the input unit 102 receives information corresponding to the operations performed by the user on the operating unit. As another example, the input unit 102 may receive information from an external device. This external device may be, for example, a portable storage medium. Subject-related information is received by the input unit 102.

[0021] The reception unit 104 acquires subject-related information from the input unit 102. The reception unit 104 acquires subject identification information and fundus image information of the eye included in the acquired subject-related information, and accepts the acquired subject identification information and fundus image information of the eye.

[0022] Fundus images are taken by shining light into the eyeball through the pupil and photographing the back of the eyeball (the part closest to the brain). Fundus images allow us to observe the macula, optic nerve head, retina, and other structures. Fundus images are generally taken in color, but monochrome or images with more than three primary colors are also acceptable. Furthermore, images taken with a simple optical system capable of capturing the macula and optic nerve head, which are crucial for predicting sphericity and axial length, can also be useful. For example, even without a full-fledged fundus camera used for disease diagnosis, images taken with an optical system similar to an autorefractometer can sometimes be used for prediction. Furthermore, the images used here can also be obtained from scanning optical systems. Figure 2 shows an example of a fundus image of the eye. The fundus image of the eye includes the macula EF01, the optic nerve head EF02, the veins EF03, and the arteries EF04. Return to Figure 1 and continue the explanation.

[0023] The processing unit 106 acquires subject identification information and fundus image information of the eye from the reception unit 104. The processing unit 106 is equipped with a trained model 107. The trained model 107 has been machine-learned to understand the relationship between fundus image information of the eye and information indicating the sphericality of the eye. The method for creating the trained model 107 will be described later. The processing unit 106 inputs the acquired fundus image information of the eye into the trained model 107 and acquires information indicating the sphericality of the eye output by the trained model 107 for the input fundus image information of the eye. The sphericality of the eye is defined as the reciprocal of the focal length expressed in meters. In this embodiment, sphericality and refractive error are synonymous.

[0024] The output unit 108 obtains subject identification information and information indicating the sphericity of the eye from the processing unit 106. The output unit 108 outputs the obtained subject identification information and information indicating the sphericity of the eye. For example, the output unit 108 may output subject identification information and information indicating the sphericity of the eye as audio, or it may output subject identification information and information indicating the sphericity of the eye by displaying them on a display unit (not shown). Furthermore, the output unit 108 may associate subject identification information with information indicating the sphericity of the eye and store it in the storage unit 110.

[0025] All or part of the input unit 102, the receiving unit 104, the processing unit 106, and the output unit 108 are functional units (hereinafter referred to as software functional units) that are realized by a processor such as a CPU (Central Processing Unit) executing a program stored in the memory unit 110. Furthermore, all or part of the input unit 102, reception unit 104, processing unit 106, and output unit 108 may be implemented by hardware such as an LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), or FPGA (Field-Programmable Gate Array), or by a combination of software functions and hardware.

[0026] (Operation of the information processing device 100) Figure 3 is a flowchart showing an example of the operation of the information processing device according to the embodiment. (Step S1-1) The input unit 102 acquires subject-related information. (Step S2-1) The reception unit 104 acquires subject-related information from the input unit 102. The reception unit 104 acquires subject identification information and fundus image information of the eye included in the acquired subject-related information, and accepts the acquired subject identification information and fundus image information of the eye.

[0027] (Step S3-1) The processing unit 106 obtains subject identification information and fundus image information of the eye from the reception unit 104. The processing unit 106 inputs the acquired fundus image information of the eye into the trained model 107 and obtains information indicating the sphericity of the eye output by the trained model 107 for the input fundus image information of the eye. (Step S4-1) The output unit 108 obtains subject identification information and information indicating the sphericity of the eye from the processing unit 106. The output unit 108 outputs the obtained subject identification information and information indicating the sphericity of the eye.

[0028] In the embodiment described above, the processing unit 106 obtains information indicating the sphericality of the subject's eye by inputting the fundus image information of the subject's eye into a trained model 107, which contains the fundus image information of the subject's eye, for which the sphericality of the eye has actually been obtained. Hereinafter, the fundus image information of the eye involved in the generation of the trained model 107 (subjects for whom the sphericality of the eye has actually been obtained and for whom the fundus image information of the eye has been stored) will be referred to as the model subject.

[0029] The creation of the trained model 107 will now be described. The trained model 107 is created by the trained model creation device. In other words, the trained model creation device creates the trained model 107. Note that the information processing device 100 may include the trained model creation device. In other words, the information processing device 100 may create the trained model 107.

[0030] Figure 4 shows an example of a learning model creation device according to this embodiment. The learning model creation device 200 according to this embodiment is implemented by a device such as a personal computer, server, smartphone, tablet computer, or industrial computer. The learning model creation device 200 uses a learning dataset in which fundus image information of the eye of a subject to be modeled is used as an input sample and information indicating the sphericity of the eye of the subject is used as an output sample to train a learning model (the model that will become the trained model 107), and creates the trained model 107.

[0031] For example, the learning model creation device 200 constructs a trained model 107 using algorithms such as CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory), Random Forest, SVM (Support Vector Machine), and neural networks. Input samples are the data input to the input layer during training of the learning model. Output samples are the correct data (training data) used to compare with the output values ​​output from the output layer during training of the learning model.

[0032] The learning model creation device 200 includes an input unit 202, a reception unit 204, a processing unit 206, an output unit 208, and a storage unit 210. The input unit 202 receives information. For example, the input unit 202 may have an operation unit such as a keyboard and a mouse. In this case, the input unit 202 receives information corresponding to the operations performed by the user on the operation unit. As another example, the input unit 202 may receive information from an external device. This external device may be, for example, a portable storage medium. The input unit 202 receives a training dataset.

[0033] The reception unit 204 obtains the training dataset from the input unit 202 and accepts the obtained training dataset. The training dataset contains input samples and output samples, and the input samples and output samples are paired. The training dataset consists of multiple pairs.

[0034] The processing unit 206 calculates the error between the output value output from the output layer of the learning model 207 (which is obtained by inputting the input sample to the input layer) and the corresponding output sample (training data) for each pair. It then modifies the parameters of the learning model 207 (trains the learning model 207) to minimize the error and creates the trained model 107. For example, the processing unit 206 may perform transfer learning using the learning model 207 that has been pretrained using data such as ImageNet. As described above, the trained model 107 is received from the output unit 208 via a network or medium to the information processing device 100, and acquired by the processing unit 106. If the learning model creation device 200 is included in the information processing device 100, the processing unit 106 acquires the trained model 107 from the learning model creation device 200.

[0035] All or part of the input unit 202, the receiving unit 204, the processing unit 206, and the output unit 208 are functional units (hereinafter referred to as software functional units) that are realized, for example, by a processor such as a CPU executing a program stored in the memory unit 210. Furthermore, all or part of the input unit 202, reception unit 204, processing unit 206, and output unit 208 may be implemented by hardware such as an LSI, ASIC, or FPGA, or by a combination of software functions and hardware.

[0036] (Operation of the learning model creation device 200) Figure 5 is a flowchart showing an example of the operation of the learning model creation device of this embodiment. (Step S1-2) The input unit 202 acquires the training dataset. (Step S2-2) The reception unit 204 obtains the training dataset from the input unit 202 and accepts the obtained training dataset.

[0037] (Step S3-2) The processing unit 206 obtains the training dataset from the reception unit 204. For all pairs of input and output samples included in the training dataset, the processing unit 206 inputs the input sample into the input layer of the training model 207, calculates the error between the output value output from the output layer and the corresponding output sample (training data), and modifies the parameters of the training model 207 to minimize the error (trains the training model 207). (Step S4-2) The output unit 208 obtains the learning model 207 from the processing unit 206. The output unit 208 outputs the obtained learning model 207.

[0038] In the embodiment described above, the trained model 107 may be one that has learned through machine learning the relationship between information indicating at least one of the following: intraocular pressure, interpupillary distance, height, and corneal curvature radius, in addition to the fundus image information of the eye, and information indicating the sphericity of the eye. In this case, the information processing device 100 may have a processing unit 106 that inputs fundus image information of the eye and information indicating at least one of intraocular pressure, interpupillary distance, height, and corneal curvature radius to a trained model 107, and then acquires information indicating the sphericity of the eye output by the trained model 107 based on the input fundus image information of the eye and information indicating at least one of intraocular pressure, interpupillary distance, height, and corneal curvature radius.

[0039] In the embodiment described above, the learning model creation device 200 may train a learning model (the model that will become the learned model 107) using a learning dataset in which, in addition to fundus image information of the eye of the subject to be modeled, information indicating at least one of intraocular pressure, interpupillary distance, height, and corneal curvature radius, and information indicating the sphericality of the eye are used as input samples, and information indicating the sphericality of the eye of the subject is used as an output sample.

[0040] In the embodiment described above, the trained model 107 may be configured to include a first trained model that has been trained to learn the relationship between the fundus image information of the right eye and the information indicating the sphericity of the right eye, and a second trained model that has been trained to learn the relationship between the fundus image information of the left eye and the information indicating the sphericity of the left eye. In this case, the information processing device 100 may have the processing unit 106 input the right eye's fundus coordinate information included in the fundus image information of the eye to a first trained model, and then acquire information indicating the sphericity of the right eye output by the first trained model based on the input right eye's fundus image information. Alternatively, the processing unit 106 may input the left eye's fundus coordinate information included in the fundus image information of the eye to a second trained model, and then acquire information indicating the sphericity of the left eye output by the second trained model based on the input left eye's fundus image information.

[0041] In the embodiment described above, the learning model creation device 200 may use a learning dataset in which the fundus image information of the right eye of the subject to be modeled is used as an input sample and information indicating the sphericity of the subject's right eye is used as an output sample, and a learning dataset in which the fundus image information of the left eye of the subject to be modeled is used as an input sample and information indicating the sphericity of the subject's left eye is used as an output sample to train a first learning model (the model on which the first learning model is based) and a second learning model (the model on which the second learning model is based), respectively, and create a learned model 107 that includes the first learning model and the second learning model.

[0042] In the embodiment described above, the trained model 107 may be one that has learned the relationship between the fundus image information of the eye and the information indicating the degree of astigmatism and the astigmatism axis through machine learning. In this case, the information processing device 100 may have a processing unit 106 that inputs fundus image information of the eye to a trained model 107, and then acquires information indicating the degree of astigmatism and information indicating the axis of astigmatism of the eye output by the trained model 107 based on the input fundus image information of the eye.

[0043] In the embodiment described above, the learning model creation device 200 may use a learning dataset in which fundus image information of the eye of the subject to be modeled is used as an input sample, and information indicating the degree of astigmatism and information indicating the axis of astigmatism of the subject's eye is used as an output sample to train a learning model (the model that will become the learning model 107) and create the learning model 107.

[0044] In the embodiment described above, the trained model 107 may be configured to include a first trained model that has been machine-trained to learn the relationship between the fundus image information of the right eye and the information indicating the degree of astigmatism and the astigmatism axis of the right eye, and a second trained model that has been machine-trained to learn the relationship between the fundus image information of the left eye and the information indicating the degree of astigmatism and the astigmatism axis of the left eye. In this case, the processing unit 106 of the information processing device 100 may input the right eye's fundus coordinate information included in the fundus image information of the eye to a first trained model, and then acquire information indicating the degree of astigmatism and information indicating the astigmatism axis of the right eye output by the first trained model for the input right eye's fundus image information. Alternatively, the processing unit 106 may input the left eye's fundus coordinate information included in the fundus image information of the eye to a second trained model, and then acquire information indicating the degree of astigmatism and information indicating the astigmatism axis of the left eye output by the second trained model for the input left eye's fundus image information.

[0045] In the embodiment described above, the learning model creation device 200 may use a learning dataset in which the fundus image information of the right eye of the subject to be modeled is used as an input sample and information indicating the degree of astigmatism and the astigmatism axis of the subject's right eye is used as an output sample, and a learning dataset in which the fundus image information of the left eye of the subject to be modeled is used as an input sample and information indicating the degree of astigmatism and the astigmatism axis of the subject's left eye is used as an output sample to train a first learning model (the model on which the first learning model is based) and a second learning model (the model on which the second learning model is based), respectively, and create a learned model 107 that includes the first learning model and the second learning model.

[0046] In the embodiment described above, the trained model 107 may be one that has learned the relationship between the fundus image information of the eye, the information indicating the sphericity of the eye, and the information indicating the degree of astigmatism and the astigmatism axis through machine learning. In this case, the information processing device 100 may have a processing unit 106 that inputs fundus image information of the eye to a trained model 107, and then acquires information indicating the sphericity of the eye, information indicating the astigmatism degree of the eye, and information indicating the astigmatism axis, which are output by the trained model 107, based on the input fundus image information of the eye.

[0047] In the embodiment described above, the learning model creation device 200 may use a learning dataset in which fundus image information of the eye of the subject to be modeled is used as an input sample, and information indicating the sphericity of the subject's eye, information indicating the degree of astigmatism, and information indicating the axis of astigmatism are output samples to train a learning model (the model that will become the learning model 107) and create the learning model 107.

[0048] In the embodiment described above, the trained model 107 may be configured to include a first trained model that has been machine-trained to learn the relationship between the fundus image information of the right eye and the information indicating the sphericity of the right eye, the astigmatism degree, and the astigmatism axis, and a second trained model that has been machine-trained to learn the relationship between the fundus image information of the left eye and the information indicating the sphericity of the left eye, the astigmatism degree, and the astigmatism axis. In this case, the processing unit 106 of the information processing device 100 may input the right eye's fundus coordinate information contained in the fundus image information of the eye to a first trained model, and then acquire information indicating the sphericity of the right eye, information indicating the astigmatism degree, and information indicating the astigmatism axis, which are output by the first trained model, based on the input right eye's fundus image information. Alternatively, the processing unit 106 may input the left eye's fundus coordinate information contained in the fundus image information of the eye to a second trained model, and then acquire information indicating the sphericity of the left eye, information indicating the astigmatism degree, and information indicating the astigmatism axis, which are output by the second trained model, based on the input left eye's fundus image information.

[0049] In the embodiment described above, the learning model creation device 200 may use a learning dataset in which the fundus image information of the right eye of the subject to be modeled is used as an input sample and information indicating the sphericity, astigmatism degree, and astigmatism axis of the subject's right eye is used as an output sample, and a learning dataset in which the fundus image information of the left eye of the subject to be modeled is used as an input sample and information indicating the sphericity, astigmatism degree, and astigmatism axis of the subject's left eye is used as an output sample to train a first learning model (the model on which the first learned model is based) and a second learning model (the model on which the second learned model is based), respectively, and create a learned model 107 that includes the first learned model and the second learned model.

[0050] According to the information processing device of this embodiment, the information processing device 100 receives fundus image information of a subject and can acquire information indicating the sphericality of the subject's eye based on a trained model 107 that has been machine-learned to determine the relationship between fundus image information of the eye and information indicating the sphericality of the eye, and the received fundus image information of the subject. Therefore, it is possible to acquire information indicating the sphericality of the eye for a subject. By using the acquired information indicating the sphericality of the eye, it is possible to predict the myopia of the subject. Fundus images are acquired for examinations of diabetes and glaucoma in medical checkups, etc., and it is beneficial to be able to know the sphericality in such examinations.

[0051] According to the information processing device of this embodiment, the information processing device 100 receives fundus image information of a subject and can acquire information indicating the sphericality of the subject's right eye and left eye based on a first trained model that has learned the relationship between the fundus image information of the right eye and information indicating the sphericality of the right eye, a second trained model that has learned the relationship between the fundus image information of the left eye and information indicating the sphericality of the left eye, and the received fundus image information of the subject. Thus, it can acquire information indicating the sphericality of the right eye and information indicating the sphericality of the left eye for each subject.

[0052] According to the learning model creation device of this embodiment, the learning model creation device 200 can create a learning model 207 by machine learning the relationship between the fundus image information of the eye and the information indicating the sphericity of the eye, based on a training dataset, with the fundus image information of the eye as the explanatory variable and the information indicating the sphericity of the eye as the target variable.

[0053] According to the learning model creation device of this embodiment, the learning model creation device 200 can create a first learning model by machine learning the relationship between the fundus image information of the right eye and the information indicating the sphericity of the right eye, based on a training dataset, with the fundus image information of the right eye as the explanatory variable and the information indicating the sphericity of the right eye as the objective variable. Furthermore, the learning model creation device 200 can create a second learning model by machine learning the relationship between the fundus image information of the left eye and the information indicating the sphericity of the left eye, based on a training dataset, with the fundus image information of the left eye as the explanatory variable and the information indicating the sphericity of the left eye as the objective variable.

[0054] (Modification of Embodiment 1) The information processing device 300 of the modified embodiment 1 will now be described. Figure 6 shows an example of an information processing device of the modified embodiment 1. The information processing device 300 of the modified embodiment 1 differs from the information processing device 100 of the embodiment in that it extracts a fundus image of a predetermined region, including the macula and optic nerve head, from the fundus image based on the fundus image information of the subject.

[0055] The information processing device 300 is implemented by a device such as a personal computer, server, smartphone, tablet computer, or industrial computer. The information processing device 300 comprises an input unit 302, a reception unit 304, an extraction unit 305, a processing unit 306, an output unit 308, and a storage unit 310. The input unit 302, the receiving unit 304, and the output unit 308 can be replaced by the input unit 102, the receiving unit 104, and the output unit 108, respectively, so their descriptions are omitted. The extraction unit 305 acquires subject identification information and fundus image information from the reception unit 104, and extracts fundus images of a predetermined region, including the macula and optic nerve head, from the fundus images based on the acquired fundus image information. Here, the fundus image of the predetermined region is a narrower region than the fundus image acquired from the reception unit 104.

[0056] The processing unit 306 acquires subject identification information and fundus image information of a predetermined region from the extraction unit 305. The processing unit 306 is equipped with a trained model 307. The trained model 307 was created by machine learning to determine the relationship between fundus image information of the eye and information indicating the sphericity of the eye. The processing unit 306 inputs the acquired fundus image information of a predetermined region into the trained model 307 to obtain information indicating the sphericity of the eye.

[0057] All or part of the input unit 302, reception unit 304, extraction unit 305, processing unit 306, and output unit 308 are functional units (hereinafter referred to as software functional units) that are realized, for example, by a processor such as a CPU executing a program stored in the memory unit 110. All or part of the input unit 302, reception unit 304, extraction unit 305, processing unit 306, and output unit 308 may be realized by hardware such as an LSI, ASIC, or FPGA, or by a combination of software functional units and hardware.

[0058] (Operation of the information processing device 300) Figure 7 is a flowchart showing an example of the operation of the information processing device in a modified example of the embodiment 1. (Steps S1-3) The input unit 302 acquires subject-related information. (Step S2-3) The reception unit 304 acquires subject-related information from the input unit 302. The reception unit 304 receives subject identification information and fundus image information of the eye, which are included in the acquired subject-related information.

[0059] (Step S3-3) The extraction unit 305 acquires subject identification information and fundus image information of the eye from the reception unit 304, and based on the acquired fundus image information of the eye, extracts fundus images of a predetermined region of the eye, including the macula and optic nerve head, from the fundus image of the eye. (Step S4-3) The processing unit 306 acquires subject identification information and fundus image information of a predetermined region of the eye from the extraction unit 305. The processing unit 306 inputs the acquired fundus image information of the predetermined region of the eye into the trained model 307 and acquires information indicating the sphericity of the eye output by the trained model 307 for the input fundus image information of the predetermined region of the eye.

[0060] (Step S5-3) The output unit 308 obtains subject identification information and information indicating the sphericity of the eye from the processing unit 306. The output unit 308 outputs the obtained subject identification information and information indicating the sphericity of the eye.

[0061] In the modified embodiment 1 described above, the processing unit 306 obtains information indicating the sphericality of the subject's eye by inputting the fundus image information of the subject's eye into a trained model 307, which contains the fundus image information of a subject whose eye sphericality has actually been acquired and whose fundus image information has already been stored. Hereinafter, the fundus image information of the eye involved in the generation of the trained model 107 (a subject whose eye sphericality has actually been acquired and whose fundus image information has already been stored) will be referred to as the model subject. The method for generating the pre-trained model 307 can be applied using the method described above for generating the pre-trained model 107, so the explanation will be omitted.

[0062] In the modified embodiment 1 described above, the trained model 307 may be configured to include a first trained model that has been trained to learn the relationship between the fundus image information of the right eye and the information indicating the sphericity of the right eye, and a second trained model that has been trained to learn the relationship between the fundus image information of the left eye and the information indicating the sphericity of the left eye. In this case, the processing unit 306 may input the right eye's fundus coordinate information included in a fundus image of a predetermined region including the macula and optic nerve head, extracted from the fundus image, into the first trained model, and obtain information indicating the sphericity of the right eye output by the first trained model based on the input right eye's fundus image information. Alternatively, the processing unit 106 may input the left eye's fundus coordinate information included in a fundus image of a predetermined region including the macula and optic nerve head, extracted from the fundus image, into the second trained model, and obtain information indicating the sphericity of the right eye output by the second trained model based on the input left eye's fundus image information.

[0063] According to the information processing device of the modified embodiment 1, the information processing device 300 extracts fundus images of a predetermined region including the macula and optic nerve head from the fundus image based on the subject's fundus image information, and can obtain information indicating the sphericality of the subject's eye based on the trained model and the extracted fundus image information of the predetermined region. Since information indicating the sphericality of the subject's eye can be obtained based on fundus images of a predetermined region including the macula and optic nerve head extracted from the fundus image, accuracy can be improved compared to the case where information indicating the sphericality of the subject's eye can be obtained based on the fundus image.

[0064] (Modified embodiment 2) A modified example of the embodiment, the information processing device 400 of which is described below. Figure 8 shows an example of an information processing device, modified example 2 of the embodiment. The information processing device 400 of the modified embodiment 2 differs from the information processing device 100 of the embodiment in that it acquires information indicating the axial length of the eye of a subject based on a trained model that has further machine-learned the relationship between the fundus image of the eye, the sphericity of the eye, and the axial length of the eye, as well as the fundus image information of the subject that has been received.

[0065] The information processing device 400 is implemented by a device such as a personal computer, server, smartphone, tablet computer, or industrial computer. The information processing device 400 comprises an input unit 402, a reception unit 404, a processing unit 406, an output unit 408, and a storage unit 410. Since the input unit 402 and the reception unit 304 can be replaced by the input unit 102 and the reception unit 104 respectively, their descriptions will be omitted.

[0066] The processing unit 406 acquires subject identification information and fundus image information of the eye from the reception unit 404. The processing unit 406 is equipped with a trained model 407. The trained model 407 was created by machine learning the relationship between fundus image information of the eye, information indicating the sphericity of the eye, and information indicating the axial length of the eye. The processing unit 406 inputs the fundus image information into the trained model 407 and obtains information indicating the sphereness of the eye and information indicating the axial length of the eye.

[0067] All or part of the input unit 402, the receiving unit 404, the processing unit 406, and the output unit 408 are functional units (hereinafter referred to as software functional units) that are realized, for example, by a processor such as a CPU executing a program stored in the memory unit 410. All or part of the input unit 402, the receiving unit 404, the processing unit 406, and the output unit 408 may be realized by hardware such as an LSI, ASIC, or FPGA, or by a combination of software functional units and hardware.

[0068] (Operation of the information processing device 400) Figure 9 is a flowchart showing an example of the operation of the information processing device in a modified example of the embodiment 2. (Steps S1-4) The input unit 402 acquires subject-related information. (Step S2-4) The reception unit 404 acquires subject-related information from the input unit 402. The reception unit 404 receives subject identification information and fundus image information of the eye, which are included in the acquired subject-related information.

[0069] (Step S3-4) The processing unit 406 obtains subject identification information and fundus image information of the eye from the reception unit 404. The processing unit 406 inputs the acquired fundus image information of the eye into the trained model 407 and obtains information indicating the sphericity of the eye and information indicating the axial length of the eye, which are output by the trained model 407 based on the input fundus image information of the eye. (Step S4-4) The output unit 408 obtains subject identification information, information indicating the sphericity of the eye, and information indicating the axial length of the eye from the processing unit 406. The output unit 408 outputs the obtained subject identification information, information indicating the sphericity of the eye, and information indicating the axial length of the eye.

[0070] In the modified embodiment described above, the processing unit 406 obtains information indicating the sphericality of the subject's eye and information indicating the axial length of the eye by inputting the fundus image information of the subject's eye into a trained model 407, which contains fundus image information of the subject's eye that has been stored for a subject from whom the sphericality of the eye and the axial length of the eye have actually been obtained. Hereinafter, fundus image information of the eye involved in the generation of the trained model 407 (subjects from whom the sphericality of the eye and the axial length of the eye have actually been obtained and from whom fundus image information of the subject's eye has been stored) will be referred to as the model subject.

[0071] The generation of the trained model 407 will now be described. The trained model 407 is created by the trained model creation device. In other words, the model creation device creates the trained model 407. Note that the information processing device 400 may include the model creation device. In other words, the information processing device 400 may create the trained model 407. Figure 10 shows an example of a learning model creation device according to a modified example of the embodiment 2. The learning model creation device 500 according to the modified example of the embodiment is implemented by a device such as a personal computer, server, smartphone, tablet computer, or industrial computer.

[0072] The learning model creation device 500 uses a learning dataset in which fundus image information of the eye of a subject to be modeled is used as input samples, and information indicating the sphericity of the eye and information indicating the axial length of the eye of the subject are used as output samples to train a learning model (the model that will become the trained model 407), and creates the trained model 407. For example, the learning model creation device 500 constructs a trained model 407 using algorithms such as CNN, RNN, LSTM, Random Forest, SVM, and Neural Network. Input samples are the data input to the input layer during training of the learning model. Output samples are the ground truth data (training data) used to compare with the output values ​​output from the output layer during training of the learning model.

[0073] The learning model creation device 500 includes an input unit 502, a receiving unit 504, a processing unit 506, an output unit 508, and a storage unit 510. Input unit 502 can apply input unit 202. The training dataset is input to input unit 502. Reception unit 504 retrieves the training dataset from input unit 502. Input samples and output samples are paired, and the training dataset consists of multiple pairs.

[0074] The processing unit 506 calculates the error between the output value output from the output layer of the learning model 507 (which is obtained by inputting the input sample to the input layer) and the corresponding output sample (training data) for each pair. It then modifies the parameters of the learning model 507 (trains the learning model 507) to minimize the error and creates the trained model 407. For example, the processing unit 506 may perform transfer learning using the learning model 507 that has been pre-trained using data such as ImageNet.

[0075] As described above, the trained model 407 is received from the output unit 508 via a network or medium to the information processing device 400, and acquired by the processing unit 406. If the learning model creation device 500 is included in the information processing device 400, the processing unit 406 acquires the trained model 407 from the learning model creation device 500. All or part of the processing unit 506 is a functional unit (hereinafter referred to as the software functional unit) that is realized, for example, by a processor such as a CPU executing a program stored in the memory unit 510. All or part of the processing unit 506 may be realized by hardware such as an LSI, ASIC, or FPGA, or by a combination of the software functional unit and hardware.

[0076] (Operation of the learning model creation device 500) Figure 11 is a flowchart showing an example of the operation of a learning model creation device in a modified example of the embodiment 2. (Steps S1-5) The input unit 502 acquires the training dataset. (Step S2-5) The reception unit 504 obtains the training dataset from the input unit 502. The reception unit 504 accepts the obtained training dataset.

[0077] (Steps S3-5) The processing unit 506 obtains the training dataset from the reception unit 504. For all pairs of input and output samples included in the training dataset, the processing unit 506 inputs the input sample into the input layer of the training model 507, calculates the error between the output value output from the output layer and the corresponding output sample (training data), and modifies the parameters of the training model 507 to minimize the error (trains the training model 507). (Steps S4-5) The output unit 508 obtains the learning model 507 from the processing unit 506. The output unit 508 outputs the obtained learning model 507.

[0078] According to the information processing device of the modified embodiment 2, the information processing device 400 can acquire information indicating the sphericality of the subject's eye and information indicating the axial length of the eye based on a trained model that has been further machine-learned to show the relationship between the fundus image of the eye and the axial length of the eye, and the fundus image information of the subject that has been received from the information processing device 100. In this way, information indicating the sphericality of the eye and information indicating the axial length of the eye can be acquired for the subject. By using the acquired information indicating the sphericality of the eye and the axial length of the eye, the subject's myopia can be predicted. Fundus images are acquired for the examination of diabetes and glaucoma in medical checkups, etc., and it is useful to know the sphericality and axial length of the eye in such examinations.

[0079] According to the learning model creation device of this embodiment, the learning model creation device 500 receives a learning dataset in the learning model creation device 200, which includes fundus image information of the eye as learning data and further includes information indicating the axial length of the eye as training data. Based on the received learning dataset, the learning model can be created by machine learning the relationship between fundus image information of the eye, information indicating the sphericity of the eye, and information indicating the axial length of the eye, with fundus image information of the eye as explanatory variables and information indicating the sphericity of the eye and information indicating the axial length of the eye as target variables.

[0080] <Example Configuration> As an example configuration, the information processing device comprises a receiving unit that receives fundus image information of a subject, a trained model that has learned the relationship between fundus image information of the eye and information indicating the sphericality of the eye, a processing unit that acquires information indicating the sphericality of the subject's eye based on the fundus image information of the subject received by the receiving unit, and an output unit that outputs the information indicating the sphericality of the subject's eye acquired by the processing unit.

[0081] As one example configuration, the processing unit acquires information indicating the axial length of the subject's eye based on a trained model that has further machine-learned the relationship between the fundus image of the eye and the axial length of the eye, and the fundus image information of the subject received by the reception unit. The output unit outputs the information indicating the axial length of the subject's eye acquired by the processing unit.

[0082] As an example configuration, the system further includes an extraction unit that extracts fundus images of a predetermined region, including the macula and optic nerve head, from the fundus image based on the subject's fundus image information. The processing unit acquires information indicating the sphericity of the subject's eye based on a trained model and the fundus image information of the predetermined region extracted by the extraction unit.

[0083] As an example configuration, the trained model includes a first trained model that has learned the relationship between the fundus image information of the right eye and the information indicating the sphericity of the right eye, and a second trained model that has learned the relationship between the fundus image information of the left eye and the information indicating the sphericity of the left eye.

[0084] Although embodiments of the present invention and modifications thereof have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments and modifications thereof, and includes design changes and the like that do not depart from the spirit of the present invention. For example, modifications 1 and 2 of the embodiment may be combined.

[0085] Alternatively, computer programs for realizing the functions of the information processing device 100, information processing device 300, information processing device 400, learning model creation device 200, and learning model creation device 500 described above may be recorded on a computer-readable recording medium, and the programs recorded on this recording medium may be loaded into a computer system and executed. The term "computer system" here may include hardware such as an operating system and peripheral devices.

[0086] Furthermore, "computer-readable recording media" refers to writable non-volatile memory such as flexible disks, magneto-optical disks, ROMs, and flash memory, portable media such as DVDs (Digital Versatile Disks), and storage devices such as hard disks built into computer systems.

[0087] Furthermore, "computer-readable recording media" also includes volatile memory (such as DRAM (Dynamic Random Access Memory)) within computer systems that act as servers or clients when programs are transmitted via networks such as the Internet or communication lines such as telephone lines, which retain programs for a certain period of time.

[0088] Furthermore, the above program may be transmitted from a computer system that stores the program in a memory device or the like to another computer system via a transmission medium or by transmission waves within the transmission medium. Here, the "transmission medium" used to transmit the program refers to a medium that has the function of transmitting information, such as a network (communication network) like the Internet or a communication line (communication line) like a telephone line.

[0089] Furthermore, the above program may be intended to implement some of the functions described above. It may also be a so-called differential file (differential program) that can implement the aforementioned functions in combination with programs already recorded in the computer system. [Explanation of symbols]

[0090] 100, 300, 400… Information processing unit; 102, 302, 402… Input unit; 104, 304, 404… Reception unit; 305… Extraction unit; 106, 306, 406… Processing unit; 107, 307, 407… Trained model; 108, 308, 408… Output unit; 110, 310, 410… Storage unit; 200, 500… Trained model creation device; 202, 502… Input unit; 204, 504… Reception unit; 206, 506… Processing unit; 207, 507… Trained model; 208, 508… Output unit; 210, 510… Storage unit

Claims

1. A reception area that receives fundus image information from the subject, A pre-trained model that has been machine-learned to determine the relationship between fundus image information of the eye and information indicating the sphericality of the eye, and a processing unit that acquires information indicating the sphericality of the subject's eye based on the fundus image information of the subject received by the reception unit, The processing unit outputs information indicating the sphericity of the subject's eye acquired by the processing unit. An information processing device equipped with the following features.

2. The processing unit acquires information indicating the axial length of the subject's eye based on a trained model that has further machine-learned the relationship between the fundus image of the eye and the axial length of the eye, and the fundus image information of the subject received by the reception unit. The information processing apparatus according to claim 1, wherein the output unit outputs information indicating the axial length of the eye of the subject acquired by the processing unit.

3. Based on the subject's fundus image information, an extraction unit extracts fundus images of a predetermined region, including the macula and optic nerve head, from the fundus image. Furthermore, The information processing apparatus according to claim 1, wherein the processing unit acquires information indicating the sphericity of the subject's eye based on the trained model and the fundus image information of a predetermined region extracted by the extraction unit.

4. The information processing apparatus according to claim 1, wherein the trained model includes a first trained model that has been machine-learned to determine the relationship between fundus image information of the right eye and information indicating the sphericity of the right eye, and a second trained model that has been machine-learned to determine the relationship between fundus image information of the left eye and information indicating the sphericity of the left eye.

5. A reception unit that accepts a training dataset in which fundus image information of the eye is included as training data and information indicating the sphericity of the eye is included as training data, A processing unit creates a learning model by machine learning the relationship between the fundus image information and the information indicating the sphericity of the eye, using the fundus image information as an explanatory variable and the information indicating the sphericity of the eye as an objective variable, based on the training dataset received by the reception unit. The output unit outputs the learning model created by the processing unit. A learning model creation device equipped with the following features.

6. The reception unit receives a training dataset which includes fundus image information of the eye as training data and further includes information indicating the axial length of the eye as training data. The learning model creation device according to claim 5, wherein the processing unit creates a learning model by machine learning the relationship between the fundus image information of the eye and the information indicating the axial length of the eye, using the fundus image information of the eye as an explanatory variable and the information indicating the axial length of the eye as an objective variable, based on the learning dataset received by the reception unit.

7. A method of information processing performed by a computer, We accept fundus image information from the subjects. Based on a trained model that has been machine-learned to determine the relationship between fundus image information of the eye and information indicating the sphericality of the eye, and the fundus image information of the subject received, information indicating the sphericality of the subject's eye is obtained. An information processing method that outputs information indicating the sphericity of the subject's eye obtained.

8. A method for creating a learning model that a computer can execute, We accept a training dataset that includes fundus image information of the eye as training data and information indicating the sphericity of the eye as training data. Based on the training dataset received, a learning model is created by machine learning the relationship between the fundus image information and the information indicating the sphericality of the eye, with the fundus image information of the eye as the explanatory variable and the information indicating the sphericality of the eye as the dependent variable. A method for creating a learning model, which outputs the created learning model.

9. On the computer, The subject's fundus image information is received, Based on a trained model that has learned the relationship between fundus image information of the eye and information indicating the sphericality of the eye, and the fundus image information of the subject that has been received, information indicating the sphericality of the subject's eye is obtained. A program that outputs information indicating the sphericity of the subject's eye, which has been acquired.

10. On the computer, The system accepts a training dataset that includes fundus image information of the eye as training data and information indicating the sphericity of the eye as training data. Based on the training dataset received, a learning model is created by machine learning the relationship between the fundus image information and the information indicating the sphericality of the eye, with the fundus image information of the eye as the explanatory variable and the information indicating the sphericality of the eye as the dependent variable. A program that outputs the aforementioned learning model that was created.

Images