Ophthalmologic system

Abstract

Provided is an ophthalmologic system that makes it possible to easily determine the probability of an eye disease when examination image data are input. An ophthalmologic system (S1) comprises a reference image data classification unit (20), a reference image conversion data generation unit (30), an examination image conversion data generation unit (40), and a data comparison unit (50). The reference image conversion data generation unit (30) is a trained AI model trained to extract features from eye image data, and generates reference image conversion data from each set of reference image data using a first encoder (31) that converts features extracted through a plurality of computational layers into embedding data. The examination image conversion data generation unit (40) generates examination image conversion data from one set of newly acquired examination image data using a second encoder (41) that is the same as the first encoder (31). The data comparison unit (50) compares the examination image conversion data with the reference image conversion data, and calculates the similarity between the two sets of image conversion data.

Description

Ophthalmology System

[0001] The present disclosure relates to ophthalmic systems.

[0002] Non-Patent Document 1 describes a knowledge-based universal model of the retina developed from a collection of 37 public datasets. This public dataset includes 284,660 fundus images and 96 different categories (disease types). The basic model consists of an image encoder and a text encoder, which are trained symmetrically on paired image and text descriptors.

[0003] Patent Document 1 describes a medical information processing system that uses AI (artificial intelligence) to detect eye diseases (such as glaucoma and age-related macular degeneration) from images obtained during ophthalmic examinations. The system accepts a patient's fundus retinal tomographic image (hereinafter referred to as "fundus OCT image") and eye information (diagnosis, visual field test data, and intraocular pressure value), and uses an artificial intelligence engine to classify the fundus OCT image into two or more categories, including a glaucoma category and a non-glaucoma category. A category is selected based on the eye information, and a determination is made as to whether the classification result based on the fundus OCT image matches the selection result based on the eye information. If there is no match, the fundus OCT image is classified into a unique category, and machine learning is performed based on the fundus OCT images classified into the unique category. Note that OCT is an abbreviation for "Optical Coherence Tomography."

[0004] “A Foundation LAnguage-Image model of the Retina (FLAIR): Encoding expert knowledge in text supervision” (arXiv:2308.07898v1[cs.CV] 15 Aug 2023)

[0005] Japanese Patent Application Laid-Open No. 2022-116134

[0006] Currently, the diagnosis of eye diseases is mainly performed by ophthalmologists visually assessing the type and severity of eye diseases based on fundus images, fundus OCT images, etc., displayed / captured by ophthalmic devices. However, the accuracy of determining the type and severity of eye diseases varies depending on the experience of the ophthalmologist making the assessment. Meanwhile, significant technological developments have been made in discriminative AI, such as image recognition, and generative AI, which generates text, images, data, etc. Against this technological background, there is a demand for an ophthalmic system that can easily determine the possibility of eye diseases simply by inputting test image data.

[0007] In contrast, the technology described in Non-Patent Document 1 is a technology that configures a basic model using an image encoder and a text encoder. The technology described in Patent Document 1 is a technology that classifies fundus OCT images using an artificial intelligence engine. Therefore, these prior art technologies cannot meet the demand for easily determining the possibility of eye disease by simply inputting test image data.

[0008] The present disclosure has been made in light of the above-mentioned problems, and aims to provide an ophthalmologic system that can easily determine the possibility of eye disease when test image data is input.

[0009] To solve the above-mentioned problems, an ophthalmologic system disclosed herein determines the possibility of eye disease based on eye image data. The ophthalmologic system includes a reference image data classification unit, a reference image transformation data generation unit, a test image transformation data generation unit, and a data comparison unit. The reference image data classification unit groups a large number of image data stored in a database by type of eye disease to generate reference image data for each group. The reference image transformation data generation unit is a trained AI model trained to extract features from eye image data, and generates reference image transformation data from each of the reference image data using a first encoder that converts the extracted features through multiple computational layers into embedded data. The test image transformation data generation unit uses a second encoder identical to the first encoder to generate test image transformation data from a newly acquired piece of test image data. The data comparison unit compares the test image transformation data with the reference image transformation data and calculates the similarity between the two image transformation data.

[0010] The ophthalmology system according to the present disclosure can easily determine the possibility of eye disease when examination image data is input.

[0011] FIG. 1 is a system configuration diagram showing an ophthalmologic system of embodiment 1. FIG. 2 is an explanatory diagram showing generation of embedded data by an encoder. FIG. 3 is an explanatory diagram showing data comparison between reference image vector values ​​and test image vector values. FIG. 4 is an explanatory diagram showing an example of a display screen that displays information on determining the possibility of eye disease. FIG. 5 is an explanatory diagram showing another example of a display screen that displays the progression of the degree of eye disease. FIG. 6 is an explanatory diagram showing experimental results of determination sensitivity using the ophthalmologic system of embodiment 1. FIG. 7 is a system configuration diagram showing an ophthalmologic system of embodiment 2. FIG. 8 is a detailed explanatory diagram showing details of optimization processing of reference image conversion data. FIG. 9 is an explanatory diagram showing experimental results of determination sensitivity using the ophthalmologic system of embodiment 2. FIG. 10 is an explanatory diagram showing the state of stage movement due to image data optimization.

[0012] An embodiment for carrying out the ophthalmologic system of the present disclosure will be described below based on Embodiments 1 and 2 shown in the drawings.

[0013] The ophthalmological systems of embodiments 1 and 2 are applied to predict and determine eye diseases and eye disease stages that can be discovered by inputting detected image data such as fundus images and fundus OCT images obtained from an ophthalmological device. Embodiment 1

[0014] [System Configuration (Fig. 1)] The ophthalmic system S1 of the first embodiment determines the possibility of ophthalmic disease based on image data acquired from an ophthalmic device. The ophthalmic device referred to here is not limited to the ophthalmic device 10 shown in Fig. 1 that acquires examination image data, but also includes various ophthalmic devices in various locations, such as a fundus camera capable of acquiring fundus images, an OCT device capable of acquiring fundus OCT images, and a slit lamp microscope capable of acquiring anterior segment images. Furthermore, ophthalmic diseases that can be determined include diabetic retinopathy (DR), branch retinal vein occlusion (BRE), age-related macular degeneration (AMD), drusen (precursor to age-related macular degeneration), tessellated fundus, optic disc cupping, and the like (see Fig. 4). Other ophthalmic diseases include glaucoma, retinal detachment, and branch retinal vein occlusion, and these may also be included.

[0015] As shown in Figure 1, the ophthalmologic system S1 has a reference image data classification unit 20, a reference image conversion data generation unit 30, an examination image conversion data generation unit 40, a data comparison unit 50, a judgment information processing unit 60, and a screen display unit 70.

[0016] The reference image data classification unit 20 groups a large number of image data stored in the database by type of eye disease to generate group-specific reference image data RI(A), RI(B), etc. For example, the reference image data for each group may be grouped such that reference image data RI(A) in group A is assigned to diabetic retinopathy and reference image data RI(B) in group B is assigned to glaucoma. Furthermore, the reference image data for each group may be grouped such that reference image data RI(A) in group A is assigned to diabetic retinopathy and reference image data RI(B) in group B is assigned to disease-free (healthy). In this way, the groups of reference image data may include a disease-free, healthy group as a disease type.

[0017] The reference image data classification unit 20 classifies the reference image data RI(A), RI(B), etc. by group into a plurality of stages for each type of disease, ranging from mild to severe eye disease, including a no-eye disease stage. For example, if the group of reference image data RI(A) is a DR group due to diabetic retinopathy, the reference image data is subdivided into five stages ranging from mild to severe, including healthy, including no-eye disease, stage 1, stage 2, stage 3, and stage 4 (see FIG. 6 ).

[0018] The reference image transformation data generation unit 30 is a trained AI model trained to extract features from eye image data, and uses a first encoder 31 that converts the features into embedded data through multiple nonlinear calculation layers. The reference image transformation data generation unit 30 inputs each of the reference image data RI(A), RI(B), etc. to the first encoder 31, which then generates multiple reference image vector values ​​R1, R2, R3, ..., Rn as multiple reference image transformation data RN(A), RN(B), etc., using the first encoder 31. The reference image vector values ​​R1, R2, R3, ..., Rn are vector values ​​representing feature vectors generated by compressing each of the reference image data RI(A), RI(B), etc. input to the first encoder 31. The first encoder 31 then outputs the multiple reference image vector values ​​R1, R2, R3, ..., Rn to the data comparison unit 50. Note that the vector values ​​refer to the component values ​​of a vector.

[0019] The test image conversion data generation unit 40 is a trained AI model trained to extract features from eye image data. It uses a second encoder 41 identical to the first encoder 31, which converts image data into embedded data through multiple nonlinear computation layers. The test image conversion data generation unit 40 inputs, for example, test image data PI newly acquired from the ophthalmologic device 10 to the second encoder 41, and generates multiple test image vector values ​​P1, P2, P3, ..., Pn as test image conversion data PN using the second encoder 41. The second encoder 41 then outputs the multiple test image vector values ​​P1, P2, P3, ..., Pn to the data comparison unit 50. Here, the test image data PI may be a fundus image or a fundus OCT image obtained by an examination. The test image vector values ​​P1, P2, P3, ..., Pn are vector values ​​representing feature vectors generated by compressing the test image data PI input to the second encoder 41. The plurality of reference image vector values ​​R1, R2, R3, ..., Rn and the plurality of inspection image vector values ​​P1, P2, P3, ..., Pn are all embedded data, and the details of the embedded data will be described later with reference to FIG.

[0020] The data comparison unit 50 compares one piece of inspection image transformation data PN from the inspection image transformation data generation unit 40 with multiple pieces of reference image transformation data RN(A), RN(B), etc. from the reference image transformation data generation unit 30, and calculates the similarity between the two pieces of image transformation data. Here, the inspection image transformation data PN is data based on multiple inspection image vector values ​​P1, P2, P3, ..., Pn. Each of the multiple reference image transformation data RN(A), RN(B), etc. is data based on multiple reference image vector values ​​R1, R2, R3, ..., Rn. The similarity between the two pieces of image transformation data is calculated by a scalar product of vector multiplication between paired values ​​of the reference image vector values ​​R1, R2, R3, ..., Rn and the inspection image vector values ​​P1, P2, P3, ..., Pn, and a similarity calculation value SC is obtained as the calculation result. Details of the data comparison will be described later with reference to FIG. 3.

[0021] The determination information processing unit 60 receives the calculated similarity values ​​SC from the data comparison unit 50 and processes the determination information necessary for determining the possibility of an eye disease. The determination information processing by the determination information processing unit 60 includes a calculation process for calculating an average similarity value AS by averaging the multiple calculated similarity values ​​SC within the same group, and a selection process for selecting the maximum similarity value MS from the multiple calculated similarity values ​​SC within the same group. In addition to the calculation process and the selection process, the determination information processing by the determination information processing unit 60 also includes a process for acquiring test image data PI, a process for acquiring reference image data RI(A), RI(B), etc., which has a high calculated similarity value SC, and a process for acquiring past similarity data of the same patient.

[0022] The screen display unit 70 has a display screen 71 that displays the determination information output from the determination information processing unit 60. The screen display unit 70 uses, for example, a display unit of a personal computer connected to the ophthalmologic apparatus 10. Details of the content displayed on the display screen 71 will be described later with reference to FIGS. 4 and 5.

[0023] The ophthalmologic system S1 may be configured as a system that can acquire information for determining the possibility of eye disease by inputting test image data indicating whether or not an eye disease is present, through operation of a terminal device capable of exchanging information with the ophthalmologic apparatus 10. Specifically, the ophthalmologic system S1 may be configured using edge computing or cloud computing. Furthermore, the ophthalmologic system S1 may be configured using a combination of edge computing and cloud computing. In the case of a combined system, the combined system reduces the storage capacity and processing load of the terminal devices, such as a server or a personal computer. In a combined system, for example, the first encoder 31 and the second encoder 41, which handle massive amounts of big data and require training, such as re-learning, over time, can be shared by cloud computing.

[0024] [Details of embedded data generation (Figure 2)] Figure 2 shows the process of generating multiple reference image conversion data RN(A), RN(B), etc. by the first encoder 31, and the process of generating inspection image conversion data PN by the second encoder 41, and the details of embedded data generation will be explained based on Figure 2.

[0025] The reference image transformation data generator 30 generates a plurality of reference image transformation data RN(A), RN(B), etc. Each of the reference image transformation data RN(A), RN(B), etc. is a plurality of reference image vector values ​​R1, R2, R3, ..., Rn representing feature vectors obtained by compressing the reference image data RI(A), RI(B), etc. input to the first encoder 31 (see FIG. 3).

[0026] The inspection image conversion data generator 40 generates one inspection image conversion data PN, which is a plurality of inspection image vector values ​​P1, P2, P3, ..., Pn representing feature vectors obtained by compressing the inspection image data PI input to the second encoder 41 (see FIG. 3).

[0027] First, one reference image data RI or one inspection image data PI is image data made up of, for example, 512 (number of vertical pixels) x 512 (number of horizontal pixels) x 3 (three primary colors of RGB) pixels.

[0028] The first encoder 31 and the second encoder 41 are one type of neural network mechanism, called an autoencoder, and also known as a data generation model. The first encoder 31 and the second encoder 41 are configured with an input layer, an intermediate layer, and an output layer, and extract features from the input image data through the multi-layered intermediate layer. In other words, the first encoder 31 and the second encoder 41 extract features, which are the most useful information from the image data, and perform a compression process that discards the remaining image data.

[0029] The first encoder 31 and the second encoder 41 have multiple nonlinear computation layers in the intermediate layer, extract features from input image data, and convert the extracted features into embedded data through the multiple nonlinear computation layers. The embedded data output from the first encoder 31 and the second encoder 41 are multiple image vector values ​​obtained by converting feature vectors obtained by compressing the image data into vector values, as shown on the right side of FIG. 2 . The multiple image vector values ​​are reference image vector values ​​R1, R2, R3, ..., Rn and test image vector values ​​P1, P2, P3, ..., Pn, and these vector values ​​are represented as, for example, 512 vector values ​​resulting from compression. Here, "embedded" refers to a numerical representation of real-world objects (e.g., images, text) that machine learning systems (ML systems) and artificial intelligence systems (AI systems) use to understand complex knowledge domains like humans.

[0030] [Details of Data Comparison (FIG. 3)] FIG. 3 shows a data comparison between reference image vector values ​​R1, R2, R3, ..., Rn and inspection image vector values ​​P1, P2, P3, ..., Pn, and details of the data comparison will be explained based on FIG. 3.

[0031] The reference image transformation data generation unit 30 and the inspection image transformation data generation unit 40 generate the same number (e.g., n = 512) of reference image vector values ​​R1, R2, R3, ..., Rn and inspection image vector values ​​P1, P2, P3, ..., Pn. The data comparison unit 50 then calculates the similarity by calculating the scalar product of vector multiplication between paired output values ​​of the reference image vector values ​​R1, R2, R3, ..., Rn and the inspection image vector values ​​P1, P2, P3, ..., Pn. The paired output values ​​refer to pairs such as vector value R1 and vector value P1, vector value R2 and vector value P2, etc.

[0032] Therefore, the calculation formula for the scalar product (inner product) of vector multiplication in the data comparison unit 50 is expressed as follows: ΣRi·Pi=R1·P1+R2·P2+ ... +Rn·Pn=|Ri||Pi|cos θ, where i = 1 to n. The reason for the above calculation formula is that the scalar product (inner product) of vectors a and b can be expressed as (a, b) = a1·b1 + a2·b2 = |a||b|cos θ, where a = (a1, a2) and b = (b1, b2) when vectors a and b on a plane are written in terms of components.

[0033] The magnitude of the scalar product of vector multiplication calculated by the data comparison unit 50 represents the similarity between the inspection image data PI and the reference image data RI(A), RI(B), etc. That is, cos θ in the above calculation formula becomes 1 when the paired vector values ​​P1 and R1 match and θ=0, becomes 0 when θ=π / 2, and becomes -1 when θ=π. Therefore, if the similarity between the inspection image data PI and the reference image data RI(A), RI(B), etc. is high, cos θ will be close to 1, and the scalar product will be a large value. If the similarity is low, the scalar product will be a small positive number including zero, or a negative number.

[0034] The calculation of the scalar product of vector multiplication is repeated between the inspection image vector values ​​P1, P2, P3, ..., Pn and all of the multiple reference image vector values ​​R1, R2, R3, ..., Rn, thereby measuring the similarity between the inspection image data PI and the reference image data RI(A), RI(B), etc., among all of the reference image data RI(A), RI(B), etc., contained in the multiple groups.

[0035] Here, the similarity calculation value calculated by the data comparison unit 50 is converted into a similarity calculation value (-1.0 to +1.0) using, for example, an activation function, which makes it easy to calculate the judgment information and understand the similarity. Note that the similarity calculation value calculated by the data comparison unit 50 may be output as is to the judgment information processing unit 60, and the value may be converted using an activation function at the input unit of the judgment information processing unit 60.

[0036] [Detailed display of eye disease assessment information (Figures 4 and 5)] Figure 4 is an example of a display screen that displays information for assessing the possibility of eye disease, and Figure 5 is an example of a display screen that displays the progression of the severity of eye disease. The detailed display of eye disease assessment information will be explained based on Figures 4 and 5.

[0037] The screen display unit 70 receives necessary information from the determination information processing unit 60 and displays information useful for determining eye diseases on the display screen 71. As shown in FIG. 4 , an example of the screen display unit 70 has a test image display unit 71 a, a similar reference image display unit 71 b, and a similarity calculation value display unit 71 c on the display screen 71.

[0038] The test image display unit 71a displays an eye image based on test image data PI acquired by the ophthalmologic apparatus 10 and used to determine the possibility of an eye disease.

[0039] The similar reference image display unit 71b displays an eye image based on reference image data RI having a high similarity to the test image data PI, among reference image data RI(A), RI(B), etc. Note that the similar reference image display unit 71b allows the user to select, by tapping the switching operation unit 71d, the reference image with the highest similarity, the reference image with the second highest similarity, or the reference image with the third highest similarity as the reference image data RI having a high similarity.

[0040] The similarity calculation value display section 71c displays the average similarity AS and the maximum similarity MS for each group, such as the DR group, in the form of a comparison table. For example, in the case of the DR group, the average similarity value for the related groups is 0.662, and the maximum similarity value is 0.973.

[0041] 5, another example of the screen display unit 70 has a similarity transition graph display section 71e that displays a similarity transition graph in which past and current similarities are graphed on a time axis on the display screen 71 when past similarity data for the same patient exists. Here, the past and current similarities may be represented by an average similarity value AS calculated as the average of multiple similarity calculation values ​​SC within the same group, or may be represented by a maximum similarity value MS selected from multiple similarity calculation values ​​within the same group.

[0042] In the similarity transition graph displayed in the similarity transition graph display unit 71 e, in the case where the eye disease is progressing, the similarity to the reference image data RI for the eye disease increases over time, as shown by the similarity transition characteristic U indicated by the thick solid line in Fig. 5. On the other hand, in the case where the eye disease is improved by the application of treatment, the similarity to the reference image data RI for the eye disease decreases over time, as shown by the similarity transition characteristic D indicated by the thin solid line in Fig. 5.

[0043] [Eye Disease Diagnosis Function of Ophthalmologic System S1] In the eye disease diagnosis function of the ophthalmologic system S1, the reference image data classification unit 20 divides and classifies the grouped reference image data RI(A), RI(B), etc. into groups by type of eye disease. The grouped reference image data RI(A), RI(B), etc. are further subdivided into a plurality of stages for each type of eye disease, ranging from a mild eye disease stage to a severe eye disease stage, including a no-eye-disease stage. Next, the reference image transformation data generation unit 30 uses a first encoder 31 to generate a plurality of reference image vector values ​​R1, R2, R3, ..., Rn, which are reference image transformation data, from each of the subdivided reference image data RI(A), RI(B), etc., divided into a plurality of stages.

[0044] Meanwhile, in the test image transformation data generation unit 40, by using a second encoder 41 that is the same as the first encoder 31, a plurality of test image vector values ​​P1, P2, P3, ..., Pn, which are test image transformation data, are generated from one test image data PI newly acquired from the ophthalmologic apparatus 10. The number of the test image vector values ​​P1, P2, P3, ..., Pn is the same as the number of reference image vector values ​​R1, R2, R3, ..., Rn (for example, 512).

[0045] Next, the data comparison unit 50 calculates similarity using the input multiple inspection image vector values ​​P1, P2, P3, ..., Pn and multiple reference image vector values ​​R1, R2, R3, ..., Rn. To calculate similarity, the data comparison unit 50 calculates a scalar product of vector multiplication between paired values ​​of the multiple reference image vector values ​​R1, R2, R3, ..., Rn and the multiple inspection image vector values ​​P1, P2, P3, ..., Pn. The scalar product of vector multiplication is used as a similarity calculation value SC, which numerically represents the degree of similarity. The similarity calculation value SC is output to the judgment information processing unit 60.

[0046] The determination information processing unit 60 calculates an average similarity value AS for each group and selects a maximum similarity value MS for each group as determination information for determining the possibility of an eye disease based on the calculated similarity values ​​SC input from the data comparison unit 50. Various pieces of determination information including the average similarity value AS and the maximum similarity value MS are output to a screen display unit 70 having a display screen 71.

[0047] When the display screen 71 of the screen display unit 70 selects information display for determining the possibility of eye disease, the display screen 71 displays a test image display area 71a, a similar reference image display area 71b, and a similarity calculation value display area 71c, as shown in FIG. 4 . By viewing the display screen 71 shown in FIG. 4 , an eye disease assessor, such as an ophthalmologist, can determine which eye disease group has the highest similarity among the groups and thereby determine whether a certain eye disease is present or suspected. For example, in the case of the DR group on the display screen 71 shown in FIG. 4 , the average similarity value AS of the related group is 0.662, which is higher than the other groups, and the maximum similarity value MS is 0.973, which is higher than the other groups. Therefore, the eye disease assessor can determine that the eye of the patient from whom the test image data PI was acquired is highly suspected of having diabetic retinopathy. Furthermore, the eye disease assessor can visually compare the test image displayed in the test image display area 71a with the reference image displayed in the similar reference image display area 71b to confirm whether their determination that diabetic retinopathy is highly suspected is correct.

[0048] Furthermore, when past similarity data for the same patient exists and information display for determining the progression of the severity of an eye disease is selected, a similarity progression graph display section 71e is displayed on the display screen 71 of the screen display unit 70, as shown in the display screen 71 of Fig. 5. Therefore, when the similarity progression graph display section 71e displays the similarity progression characteristic U shown by the thick solid line in Fig. 5, an eye disease evaluator such as an ophthalmologist can determine that the eye disease is progressing because the similarity for a certain eye disease is increasing over time. On the other hand, when the similarity progression graph display section 71e displays the similarity progression characteristic D shown by the thin solid line, an eye disease evaluator can determine that the eye disease is improving due to the application of treatment because the similarity for a certain eye disease is decreasing over time.

[0049] [Confirmation of Eye Disease Determination Sensitivity (FIG. 6)] FIG. 6 shows the experimental results of the determination sensitivity using the ophthalmologic system S1 of the first embodiment, and the confirmation of the eye disease determination sensitivity will be described with reference to FIG.

[0050] As described above in the eye disease determination function, when determining that there is a high suspicion of a certain eye disease, when determining that the eye disease is progressing, or when determining that the eye disease is improving, high sensitivity of the eye disease prediction stage is a condition for accurate determination. Therefore, the inventor conducted an experiment to confirm the sensitivity of the eye disease prediction stage of the ophthalmologic system S1 of embodiment 1.

[0051] In the experiment, 366 pieces of image data were prepared as image data of eyes belonging to the DR group due to diabetic retinopathy. The ophthalmologic system S1 of embodiment 1 determined the predicted eye disease stage from the 366 pieces of image data. Meanwhile, a group of experts, including experienced ophthalmologists, determined the true eye disease stage for the 366 pieces of image data. Then, the results of the predicted eye disease stage determination were plotted on the horizontal axis, and the results of the true eye disease stage determination were plotted on the vertical axis, and the 366 pieces of image data were sorted into regions to which they belonged, as shown in the lower right of Figure 6.

[0052] The experimental results showed that the predicted eye disease stage and the true eye disease stage matched for 312 of the 366 image data. In other words, the experimental results showed that the sensitivity was 1.0 for the 312 image data. However, the experimental results showed that the predicted eye disease stage and the true eye disease stage did not match for the remaining 54 image data, and the sensitivity was less than 1.0 for the 54 image data.

[0053] Furthermore, the experimental results showed that of the 197 image data pieces with an eye disease prediction stage of stage 0 (healthy), the predicted eye disease stage matched the true eye disease stage for 196 image data pieces. However, when the predicted eye disease stages were stage 1, stage 2, stage 3, and stage 4, the degree of agreement between the predicted eye disease stage and the true eye disease stage decreased compared to stage 0.

[0054] As a result, the experimental results showed that the total sensitivity of the degree of agreement between the predicted stage of eye disease and the true stage of eye disease was 0.85, confirming that this sensitivity can determine the stage of eye disease. In addition, the experimental results showed a high specificity of 0.98.

[0055] Other experimental results using the ophthalmology system S1 are shown in the upper part of Figure 6, which shows the average similarity values ​​for the same stage and different stages for the DR group. The average similarity value for the same stage is calculated by performing vector multiplication between two data sets within the same stage and calculating the average value. The average similarity value for different stages is calculated by performing vector multiplication between each two data sets in different stages and calculating the average value. The average similarity values ​​for the same stage are 0.74 for the ocular disease-free stage, 0.69 for stage 1, 0.75 for stage 2, 0.74 for stage 3, and 0.69 for stage 4. The average similarity values ​​between different stages are 0.06 between the ocular disease-free stage and stage 1, 0.65 between stages 1 and 2, 0.67 between stages 2 and 3, and 0.65 between stages 3 and 4.

[0056] [Advantages of Ophthalmologic System S1] The ophthalmologic system S1 of the first embodiment has the following advantages.

[0057] (1) The ophthalmologic system S1 determines the possibility of eye disease based on eye image data and includes a reference image data classification unit 20, a reference image conversion data generation unit 30, a test image conversion data generation unit 40, and a data comparison unit 50. The reference image data classification unit 20 groups a large number of image data stored in a database by type of eye disease to generate reference image data RI(A), RI(B), etc. for each group. The reference image conversion data generation unit 30 is a trained AI model trained to extract features from eye image data. It uses a first encoder 31 that converts the extracted features through multiple calculation layers into embedded data to generate reference image conversion data RN(A), RN(B), etc. from each of the reference image data RI(A), RI(B), etc. The test image conversion data generation unit 40 uses a second encoder 41 identical to the first encoder 31 to generate test image conversion data PN from a newly acquired single test image data PI. The data comparison unit 50 compares the test image transformation data PN with the reference image transformation data RN(A), RN(B), etc., and calculates the similarity between the two image transformation data. This ophthalmologic system S1 compares two image transformation data generated using the encoders 31 and 41 based on a trained AI model and calculates the similarity, so that when test image data PI is input, the possibility of eye disease can be easily determined.

[0058] (2) The reference image data classification unit 20 has data for each group of reference image data RI(A), RI(B), etc., which are subdivided into multiple stages for each type of eye disease, ranging from mild eye disease stages to severe eye disease stages, including a stage without eye disease. When the test image data PI is input, this ophthalmologic system S1 can determine not only the possibility of eye disease but also the eye disease stage, which indicates the degree of progression of the eye disease. Furthermore, since the determined eye disease stages include mild eye disease stages, this can contribute to early detection and early treatment of eye diseases and improved medical efficiency.

[0059] (3) The reference image transformation data generation unit 30 converts the reference image transformation data RN(A), RN(B), etc. into a plurality of reference image vector values ​​R1, R2, R3, ..., Rn representing feature vectors generated by compressing the reference image data input to the first encoder 31. The test image transformation data generation unit 40 converts the test image transformation data PN into a plurality of test image vector values ​​P1, P2, P3, ..., Pn representing feature vectors generated by compressing the test image data PI input to the second encoder 41. This ophthalmologic system S1 can convert the reference image transformation data RN(A), RN(B), etc. and the test image transformation data PN into vector values ​​that are easy to handle when performing data comparison calculations.

[0060] (4) The reference image transformation data generation unit 30 and the test image transformation data generation unit 40 generate the same number of reference image vector values ​​R1, R2, R3, ..., Rn and test image vector values ​​P1, P2, P3, ..., Pn, respectively. The data comparison unit 50 obtains a similarity calculation value SC by taking the scalar product of the reference image vector values ​​R1, R2, R3, ..., Rn and the test image vector values ​​P1, P2, P3, ..., Pn. In this ophthalmologic system S1, the data comparison unit 50 can obtain the similarity calculation value SC, which serves as determination information for image similarity when assessing eye diseases, by a simple calculation process, namely, the scalar product of vector multiplication.

[0061] (5) The ophthalmologic system S1 includes a determination information processing unit 60 that receives the calculated similarity values ​​SC from the data comparison unit 50 and processes determination information necessary for determining the possibility of eye disease. The determination information processing unit 60 calculates an average similarity value AS by averaging multiple calculated similarity values ​​SC within the same group as one piece of determination information. When determining eye disease, the ophthalmologic system S1 uses the average similarity value AS as a determination criterion and can determine whether or not there is a suspicion of eye disease based on the magnitude of the average similarity value AS.

[0062] (6) The determination information processing unit 60 calculates the maximum similarity value MS, which is the largest among the multiple similarity calculation values ​​SC within the same group, as one piece of determination information. When determining eye disease, this ophthalmologic system S1 uses the maximum similarity value MS as a determination criterion and can determine whether an eye disease is suspected based on the group that produced the maximum similarity value MS.

[0063] (7) The ophthalmologic system S1 includes a screen display unit 70 having a display screen 71 for displaying the assessment information output from the assessment information processing unit 60. The screen display unit 70 includes a test image display unit 71a for displaying an eye image based on the test image data PI, a similar reference image display unit 71b for displaying an eye image based on data with a high degree of similarity among the reference image data RI(A), RI(B), etc., and a similarity calculation value display unit 71c for displaying the average similarity AS and the highest similarity MS for each group. When assessing an eye disease, the ophthalmologic system S1 can display image information and numerical information useful for assessing an eye disease together on a single display screen 71.

[0064] (8) The ophthalmologic system S1 includes a screen display unit 70 having a display screen 71 that displays the determination information output from the determination information processing unit 60. When past similarity data for the same patient exists, the screen display unit 70 has a similarity transition graph display unit 71e that displays a similarity transition graph, which graphs past similarities and current similarities on a time axis, on the display screen 71. When determining an eye disease, the ophthalmologic system S1 can determine whether the eye disease is progressing or whether the eye disease has improved as a result of treatment by looking at the displayed similarity transition graph. Embodiment 2

[0065] The second embodiment is a configuration in which an optimizer that optimizes image conversion data output from the encoder is added to the system configuration of the first embodiment.

[0066] [System Configuration (Fig. 7)] As shown in Fig. 7, the ophthalmologic system S2 of the second embodiment includes a reference image data classification unit 20, a reference image conversion data generation unit 30, a reference image data optimization processing unit 80, a test image conversion data generation unit 40, a test image data optimization processing unit 90, and a data comparison unit 50'. The ophthalmologic system S2 of the second embodiment also includes the determination information processing unit 60 and the screen display unit 70 shown in Fig. 1, but these are not shown in Fig. 7.

[0067] The reference image data optimization processing unit 80 is provided between the reference image transformation data generation unit 30 and the data comparison unit 50'. This reference image data optimization processing unit 80 uses a first optimizer 81 that assigns different weights to each of multiple embedding data through training that increases the similarity between the same data groups and decreases the similarity between different data groups. Here, the optimizer has the function of converting the embedding data from the reference image transformation data generation unit 30 into different embedding data.

[0068] The training objective of the first optimizer 81 is to increase the similarity between the same data groups and decrease the similarity between different data groups. In other words, when the same data groups are grouped, the training objective is to increase the similarity within the group and decrease the similarity between the groups. Furthermore, when the same data groups are grouped into the same stage, the training objective is to increase the similarity within each stage in each group and decrease the similarity between each stage. Aiming for this objective, the first optimizer 81 performs training in machine learning, thereby assigning different weighting coefficients to each of the multiple reference image vector values ​​R1, R2, R3, ..., Rn, which are embedded data. In other words, the reference image data optimization processing unit 80 uses the first optimizer 81 to optimize reference image transformation data RN(A), RN(B), etc. into optimized reference image transformation data RNO(A), RNO(B), etc.

[0069] The inspection image data optimization processing unit 90 is provided between the inspection image conversion data generation unit 40 and the data comparison unit 50'. This inspection image data optimization processing unit 90 uses a second optimizer 91, which is the same as the first optimizer 81, for each of the multiple embedded data. Therefore, the second optimizer 91 performs processing to apply different weighting coefficients obtained using the first optimizer 81 to each of the multiple inspection image vector values ​​P1, P2, P3, ..., Pn, which are the embedded data. In other words, the inspection image data optimization processing unit 90 uses the second optimizer 91 to optimize the inspection image conversion data PN into optimized inspection image conversion data PNO.

[0070] The data comparison unit 50' is a comparison unit that compares the optimized inspection image conversion data PNO with the optimized reference image conversion data RNO(A), RNO(B), etc., and calculates the similarity between the two image conversion data. Note that the reference image data classification unit 20, reference image conversion data generation unit 30, and inspection image conversion data generation unit 40 are the same as the reference image data classification unit 20, reference image conversion data generation unit 30, and inspection image conversion data generation unit 40 of the first embodiment, and therefore their explanations will be omitted.

[0071] [Reference image transformation data optimization process (Figure 8)] Figure 8 shows details of the reference image transformation data optimization process that increases similarity within a group and decreases similarity between groups, and the reference image transformation data optimization process will be explained based on Figure 8.

[0072] The first optimizer 81 performs training by machine learning with the objective of increasing similarity within a group and decreasing similarity between groups. For this reason, when a plurality of reference image vector values ​​R1, R2, R3, ..., Rn, which are embedded data, are input, the first optimizer 81 assigns different weights to each of the plurality of reference image vector values ​​R1, R2, R3, ..., Rn, and assigns different weighting coefficients to each of the plurality of reference image vector values ​​R1, R2, R3, ..., Rn.

[0073] 8, the multiple reference image vector values ​​R1, R2, R3, ..., Rn in group A and group B have small variations in similarity values ​​within each group. On the other hand, group A and group B have large numerical differences, so they can be clearly distinguished.

[0074] The first optimizer 81 performs training by machine learning with the objective of increasing the similarity within each stage in each group and decreasing the similarity between each stage. Therefore, when the first optimizer 81 receives multiple reference image vector values ​​R1, R2, R3, ..., Rn as embedded data, it assigns different weights to each of the multiple reference image vector values ​​R1, R2, R3, ..., Rn, and assigns different weighting coefficients to each of the multiple reference image vector values ​​R1, R2, R3, ..., Rn.

[0075] Therefore, for example, the multiple reference image vector values ​​R1, R2, R3, ..., Rn of group A in stages 0 to 4 have small variations in similarity values ​​within each stage. On the other hand, the multiple reference image vector values ​​R1, R2, R3, ..., Rn of group A have large numerical differences between each of stages 0 to 4, so they can be clearly distinguished.

[0076] [Confirmation of the sensitivity of eye disease determination (FIGS. 9 and 10)] FIG. 8 shows the experimental results of the sensitivity of determination using the ophthalmologic system S2 of embodiment 2, and FIG. 9 shows the state of stage movement due to image data optimization. The confirmation of the sensitivity of eye disease determination will be explained based on FIGS. 9 and 10.

[0077] The experimental results using the ophthalmology system S2 show the average similarity values ​​for the same stage and the average similarity values ​​for different stages in the DR group, as shown in the upper part of Figure 9. The average similarity values ​​for the same stage increased from 0.74 to 0.98 for the stage without ocular disease, increased from 0.69 to 0.94 for stage 1, increased from 0.75 to 0.79 for stage 2, increased from 0.74 to 0.93 for stage 3, and increased from 0.69 to 0.91 for stage 4. In other words, it was confirmed that the ophthalmology system S2 increased the similarity within each stage in the DR group through data optimization. Furthermore, the average similarity between different stages decreased from 0.06 to -0.16 between the stage without ocular disease and stage 1, decreased from 0.65 to -0.28 between stage 1 and stage 2, decreased from 0.67 to -0.28 between stage 2 and stage 3, and decreased from 0.65 to -0.17 between stage 3 and stage 4. In other words, it was confirmed that the ophthalmology system S2 reduces the similarity between each stage in the DR group due to data optimization compared to the ophthalmology system S1.

[0078] The experiment shown in Fig. 6 of the above-mentioned embodiment 1 was also conducted in the same manner in embodiment 2. Then, with the judgment results of the predicted eye disease stage on the horizontal axis and the judgment results of the true eye disease stage on the vertical axis, the 366 pieces of image data were divided into areas to which they belong, as shown in the lower right of Fig. 9.

[0079] The experimental results showed that the predicted eye disease stage and the true eye disease stage matched for 318 of the 366 image data. In other words, the experimental results showed a sensitivity of 1.0 for the 318 image data, six more than for the ophthalmic system S1. The experimental results showed that the predicted eye disease stage and the true eye disease stage did not match for 54 image data in the ophthalmic system S1, whereas the predicted eye disease stage and the true eye disease stage did not match for 48 image data, six fewer than the ophthalmic system S2. The breakdown of the six missing image data is shown in Figure 10 . In the ophthalmic system S2, optimization processing of the comparison data shifted six image data to positions where the predicted eye disease stage and the true eye disease stage matched, as indicated by the arrows on the right side of Figure 10 . The reason why the predicted eye disease stage and the true eye disease stage did not match for the 48 image data is, for example, due to dirt adhering to the lens of the ophthalmic device capturing the images, resulting in the dirt being captured in the image data.

[0080] As a result, the experimental results showed that the total sensitivity of the agreement between the predicted eye disease stage and the true eye disease stage was 0.88 (>0.85), confirming that the eye disease stage can be determined with a sensitivity of 88%.In addition, the experimental results showed a high specificity of 0.98 (=0.98).

[0081] [Effects of Ophthalmologic System S2] The ophthalmologic system S2 of the second embodiment has the following effects in addition to the effects (1) to (8) of the first embodiment.

[0082] (9) Between the reference image conversion data generation unit 30 and the data comparison unit 50', there is a reference image data optimization processing unit 80 that optimizes reference image conversion data RN(A), RN(B), etc. into optimized reference image conversion data RNO(A), RNO(B), etc. The reference image data optimization processing unit 80 uses a first optimizer 81 that assigns different weights to multiple embedded data through training to increase the similarity between the same data groups and decrease the similarity between different data groups. Between the inspection image conversion data generation unit 40 and the data comparison unit 50', there is an inspection image data optimization processing unit 90 that optimizes inspection image conversion data PN into optimized inspection image conversion data PNO. The inspection image data optimization processing unit 90 uses a second optimizer 91 that is the same as the first optimizer 81. The data comparison unit 50' is a comparison unit that compares the optimized inspection image conversion data PNO with the optimized reference image conversion data RNO(A), RNO(B), etc., and calculates the similarity between the two image conversion data. By adding the first optimizer 81 and the second optimizer 91, the ophthalmic system S2 has higher sensitivity in determining what eye disease a patient has, compared to the ophthalmic system S1 which does not have an optimizer, and can improve the accuracy of predicting eye diseases.

[0083] The ophthalmic system S1 of Embodiment 1 and the ophthalmic system S2 of Embodiment 1 have been described above with reference to the drawings. However, the specific configuration of the ophthalmic system of the present disclosure is not limited to Embodiments 1 and 2, and design changes and additions are permitted as long as they do not deviate from the gist of the invention according to each claim in the scope of the claims.

[0084] In the first and second embodiments, the reference image data classifier 20 subdivides the reference image data for each group into a plurality of stages, ranging from a mild eye disease stage to a severe eye disease stage, including a stage without eye disease, for each type of disease. However, the reference image data classifier is not limited to subdividing the reference image data into a plurality of stages in addition to the type of disease. For example, the reference image data classifier may simply classify the reference image data into groups for each type of disease and determine whether there is a possible eye disease.

[0085] In the first and second embodiments, the reference image transformation data generation unit 30 and the test image transformation data generation unit 40 are configured to use a plurality of image vector values ​​representing feature vectors generated by compressing image data as image transformation data. However, the reference image transformation data generation unit and the test image transformation data generation unit are not limited to the example in which a plurality of image vector values ​​are used as image transformation data. In other words, the image transformation data may be embedded data obtained by converting feature quantities extracted through a plurality of calculation layers.

[0086] In the first and second embodiments, the determination information processing unit 60 acquires the average similarity value AS and the maximum similarity value MS as the determination information. However, the determination information processing unit is not limited to acquiring the average similarity value and the maximum similarity value as the determination information. In other words, the determination information may be any information useful for determining the possibility of an eye disease based on the calculated similarity value.

[0087] In the first and second embodiments, the screen display unit 70 includes a test image display unit 71 a, a similar reference image display unit 71 b, a similarity calculation value display unit 71 c, and a similarity transition graph display unit 71 e on the display screen 71. However, the screen display unit is not limited to having the above display units on the display screen. For example, the screen display unit may display other graphs, such as bar graphs, on the display screen. CROSS-REFERENCE TO RELATED APPLICATIONS

[0088] This application claims priority based on Japanese Patent Application No. 2024-104645, filed with the Japan Patent Office on June 28, 2024, the entire disclosure of which is incorporated herein by reference in its entirety.

Claims

1. An ophthalmology system that determines the possibility of eye disease based on eye image data, comprising: a reference image data classification unit that groups a large number of image data stored in a database by type of eye disease and creates reference image data for each group; a reference image transformation data generation unit that uses a first encoder, which is a learned AI model trained to extract features from eye image data, to convert the features extracted through multiple calculation layers into embedded data, and generates reference image transformation data from each of the reference image data; a test image transformation data generation unit that uses a second encoder that is the same as the first encoder, and generates test image transformation data from a single piece of newly acquired test image data; and a data comparison unit that compares the test image transformation data with the reference image transformation data and calculates the similarity between the two image transformation data.

2. An ophthalmological system according to claim 1, characterized in that the reference image data classification unit has, as the reference image data for each group, data subdivided into a plurality of stages for each type of disease, ranging from mild eye disease stages to severe eye disease stages, including a stage without eye disease.

3. An ophthalmic system according to claim 2, characterized in that the reference image transformation data generation unit sets the reference image transformation data as a plurality of reference image vector values ​​representing feature vectors generated by compressing the reference image data input to the first encoder, and the test image transformation data generation unit sets the test image transformation data as a plurality of test image vector values ​​representing feature vectors generated by compressing the test image data input to the second encoder.

4. An ophthalmological system according to claim 3, wherein the reference image transformation data generation unit and the test image transformation data generation unit generate the same number of reference image vector values ​​and test image vector values, respectively, and the data comparison unit obtains a similarity calculation value by the scalar product of the reference image vector value and the test image vector value.

5. An ophthalmological system according to claim 4, further comprising a judgment information processing unit that receives the calculated similarity value from the data comparison unit and processes judgment information necessary for determining the possibility of an eye disease, and the judgment information processing unit calculates an average similarity value as one piece of judgment information by averaging a plurality of calculated similarity values ​​within the same group.

6. An ophthalmological system according to claim 5, wherein the judgment information processing unit calculates the maximum similarity value among a plurality of the similarity calculation values ​​within the same group as one of the judgment information.

7. An ophthalmological system according to claim 6, comprising a screen display unit having a display screen for displaying the judgment information output from the judgment information processing unit, wherein the screen display unit has, on the display screen, a test image display unit for displaying an eye image based on the test image data, a similar reference image display unit for displaying an eye image based on data with a high degree of similarity among the reference image data, and a similarity calculation value display unit for displaying the average similarity value and the highest similarity value for each group.

8. An ophthalmological system according to claim 6, further comprising a screen display unit having a display screen for displaying the judgment information output from the judgment information processing unit, wherein the screen display unit has a similarity transition graph display unit for displaying a similarity transition graph on the display screen, in the case where past similarity data for the same patient exists, which graphs past similarity and current similarity on a time axis.

9. An ophthalmological system according to any one of claims 1 to 8, characterized in that: between the reference image conversion data generation unit and the data comparison unit, there is a reference image data optimization processing unit that uses a first optimizer that assigns different weights to multiple embedded data through training that increases the similarity between the same data groups and decreases the similarity between different data groups, and that optimizes the reference image conversion data into optimized reference image conversion data; between the test image conversion data generation unit and the data comparison unit, there is an inspection image data optimization processing unit that uses a second optimizer that is the same as the first optimizer, and that optimizes the inspection image conversion data into optimized inspection image conversion data; and the data comparison unit is a comparison unit that compares the optimized inspection image conversion data with the optimized reference image conversion data and calculates the similarity between the two image conversion data.

Images