Ophthalmic image processing device, ophthalmic image processing program, and ophthalmic image processing system

Abstract

A control unit of this ophthalmic image processing device acquires an ophthalmic image captured by an ophthalmic image imaging device. The control unit acquires evaluation information indicating a degree of appropriateness for acquiring medical data for the acquired ophthalmic image, said medical data being at least one of an analysis result pertaining to a disease of the eye being examined, an analysis result pertaining to the structure of the eye being examined, or a converted image obtained by converting the acquired ophthalmic image. The control unit acquires the medical data on the basis of the acquired ophthalmic image. The control unit changes the method for acquiring the medical data in accordance with whether the evaluation information for the ophthalmic image satisfies a criterion.

Description

[Technical field]

[0001] The present disclosure relates to an ophthalmic image processing device, an ophthalmic image processing program, and an ophthalmic image processing system used for processing ophthalmic images of a subject's eye. [Background technology]

[0002] In recent years, a technique for acquiring various medical data based on an ophthalmic image of a subject's eye has been proposed. For example, in the technique disclosed in Non-Patent Document 1, an analysis result regarding the boundaries of each layer of tissue shown in an ophthalmic image is acquired based on the ophthalmic image. It is also conceivable to acquire an analysis result regarding a disease of the subject's eye or a converted image obtained by converting an ophthalmic image based on the captured ophthalmic image. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Yufan He, Aaron Carass, et al. “Topology guaranteed segmentation of the human retina from OCT using convolutional neural networks.”arXiv:1803.05120, 14 Mar 2018 Summary of the Invention

[0004] When acquiring medical data based on an ophthalmic image, the captured ophthalmic image may be an ophthalmic image that is not suitable for acquiring medical data. When medical data is acquired based on an inappropriate ophthalmic image, the reliability of the medical data is reduced, and the medical data is often not useful to doctors and the like.

[0005] A typical object of the present disclosure is to provide an ophthalmic image processing device, an ophthalmic image processing program, and an ophthalmic image processing system that are capable of appropriately acquiring medical data based on an ophthalmic image.

[0006] An ophthalmic image processing device provided by a typical embodiment of the present disclosure is an ophthalmic image processing device that processes ophthalmic images, which are images of tissues of a test eye, and a control unit of the ophthalmic image processing device includes an image acquisition step of acquiring an ophthalmic image captured by an ophthalmic image capturing device, and an evaluation information acquisition step of acquiring evaluation information indicating the suitability of the ophthalmic image acquired in the image acquisition step for acquiring at least any one of medical data, such as an analysis result regarding a disease of the test eye, an analysis result regarding a structure of the test eye, and a converted image obtained by converting the acquired ophthalmic image; Mathematical models trained by machine learning algorithms The ophthalmologic image acquired in the image acquisition step. By entering A medical data acquisition step of acquiring the medical data; the mathematical model is trained with a plurality of training data sets, the training data being input ophthalmic images captured by an ophthalmic image capturing device and the medical data corresponding to the input training data, and the training data being output; In the medical data acquisition step, the control unit determines whether the evaluation information acquired in the evaluation information acquisition step is , which is set based on a plurality of the input training data used to train the mathematical model. The method of acquiring the medical data is changed depending on whether the criteria are met.

[0007] An ophthalmic image processing program provided by a typical embodiment of the present disclosure is an ophthalmic image processing program executed by an ophthalmic image processing device that processes ophthalmic images, which are images of the tissues of a test eye, and the ophthalmic image processing program is executed by a control unit of the ophthalmic image processing device to perform an image acquisition step of acquiring an ophthalmic image captured by an ophthalmic image capturing device, and an evaluation information acquisition step of acquiring evaluation information indicating the suitability of the ophthalmic image acquired in the image acquisition step for acquiring at least any one of medical data, such as an analysis result regarding a disease of the test eye, an analysis result regarding the structure of the test eye, and a converted image obtained by converting the acquired ophthalmic image; Mathematical models trained by machine learning algorithms The ophthalmologic image acquired in the image acquisition step. By entering a medical data acquisition step of acquiring the medical data by the ophthalmologic image processing device; the mathematical model is trained with a plurality of training data sets, the training data being input ophthalmic images captured by an ophthalmic image capturing device and the medical data corresponding to the input training data, and the training data being output; In the medical data acquisition step, the evaluation information acquired in the evaluation information acquisition step is , which is set based on a plurality of the input training data used to train the mathematical model.Depending on whether the criteria are met, the method of acquiring the medical data is changed.

[0008] An ophthalmic image processing system provided by a typical embodiment of the present disclosure is an ophthalmic image processing system that processes ophthalmic images, which are images of tissues of a test eye, and the ophthalmic image processing system includes an OCT device that captures ophthalmic images, which are tomographic images of the tissues, by receiving interference light between a reference light and reflected light of a measurement light irradiated onto the tissues of the test eye, and a control unit of the ophthalmic image processing system includes an image acquisition step of acquiring an ophthalmic image captured by the OCT device, and an evaluation information acquisition step of acquiring evaluation information indicating the suitability of the ophthalmic image acquired in the image acquisition step for acquiring at least any one of medical data, such as an analysis result regarding a disease of the test eye, an analysis result regarding a structure of the test eye, and a converted image obtained by converting the acquired ophthalmic image; a medical data acquisition step of acquiring the medical data by inputting the ophthalmic image acquired in the image acquisition step into a mathematical model trained by a machine learning algorithm, the mathematical model being trained by a plurality of training data sets in which ophthalmic images captured by an ophthalmic image capturing device are used as input training data and the medical data corresponding to the input training data are used as output training data, and the control unit further The evaluation information acquired in the evaluation information acquisition step , which is set based on a plurality of the input training data used to train the mathematical model. an addition step of performing a process of additionally photographing the same part as the part photographed with the ophthalmologic image when the criterion is not satisfied, and performing an averaging process of the additionally photographed image with respect to the ophthalmologic image; and in the medical data acquisition step, The ophthalmic image on which the arithmetic averaging process has been performed is input into a mathematical model trained by a machine learning algorithm to obtain the medical data. R .

[0009] According to the ophthalmic image processing device, the ophthalmic image processing program, and the ophthalmic image processing system according to the present disclosure, medical data can be appropriately acquired based on an ophthalmic image.

[0010] The control unit of the ophthalmic image processing device exemplified in the present disclosure executes an image acquiring step, an evaluation information acquiring step, and a medical data acquiring step. In the image acquiring step, the control unit acquires an ophthalmic image captured by the ophthalmic image capturing device. In the evaluation information acquiring step, the control unit acquires evaluation information indicating the suitability for acquiring medical data for the ophthalmic image acquired in the image acquiring step. The medical data is at least one of an analysis result regarding a disease in the subject's eye, an analysis result regarding the structure of the subject's eye, and data of a converted image obtained by converting the ophthalmic image. In the medical data acquiring step, the control unit acquires medical data based on the ophthalmic image acquired in the image data acquiring step. In the medical data acquiring step, the control unit changes a method of acquiring medical data depending on whether or not the evaluation information of the ophthalmic image satisfies a criterion.

[0011] According to the ophthalmic image processing device exemplified in the present disclosure, evaluation information indicating the suitability for acquiring medical data is acquired for the ophthalmic image, and the method of acquiring the medical data is changed according to the evaluation information. Thus, the medical data is appropriately acquired from the ophthalmic image according to the evaluation information of the ophthalmic image.

[0012] In addition, various parameters related to the image quality of an ophthalmic image can be appropriately used for the evaluation information of an ophthalmic image. For example, the signal strength of an ophthalmic image or an index indicating the quality of a signal (e.g., SSI (Signal Strength Index) or QI (Quality Index) etc.) may be used as the evaluation information of an ophthalmic image. In addition, at least one of the ratio of the noise level to the signal level of an image (SNR (Signal to Noise Ratio), background noise level, image contrast, etc.) may be used as the evaluation information of an ophthalmic image. In addition, the shooting conditions when an ophthalmic image is shot by an ophthalmic image shooting device (e.g., at least one of the conditions related to scanning and exposure time when an ophthalmic image is shot by scanning light on tissue) may be used as the evaluation information. In addition, when an ophthalmic image is acquired by shooting multiple images of the same part and performing an addition process (e.g., an averaging process) of the multiple images, the number of images used in the addition process (hereinafter referred to as "the number of images added") may be used as the evaluation information.

[0013] In addition, the specific contents of the medical data acquired based on the ophthalmic image can be appropriately selected. For example, an automatic analysis result indicating whether or not the subject eye has any disease may be acquired as medical data from the ophthalmic image. In this case, for example, the probability of the presence of each disease may be acquired as an analysis result. In addition, the contents of the analysis result of the structure of the subject eye can be appropriately selected. For example, an analysis result of at least one of the tissue layers and the boundary of the layers may be acquired as medical data from a tomographic image of the tissue of the subject eye (e.g., the fundus, etc.). In addition, an analysis result of the fundus blood vessels may be acquired as medical data from the fundus image of the subject eye. In this case, the analysis result may include an analysis result (discrimination result) of the artery and vein of the fundus. In addition, a method of acquiring a converted image obtained by converting an ophthalmic image can be appropriately selected. For example, data of a high-quality image may be acquired as data of the converted image from an ophthalmic image of low image quality. In this case, data of the high-quality image may be acquired, for example, by reducing noise in the ophthalmic image or by improving the contrast of the ophthalmic image. In addition, a high-resolution image obtained by increasing the resolution of the ophthalmic image may be acquired as the converted image. In addition, an image with improved visibility of an ophthalmic image, or an image in which a specific structure shown in the ophthalmic image is emphasized, may be acquired as a converted image.

[0014] In the medical data acquisition step, the control unit may acquire medical data by inputting the ophthalmologic image into a mathematical model trained by a machine learning algorithm. In this case, by using a mathematical model trained by a plurality of training data, useful medical data can be easily acquired even in cases where it is difficult to appropriately acquire medical data using a function that does not use a machine learning algorithm.

[0015] Furthermore, when using a machine learning algorithm, the greater the difference between the quality of the ophthalmic image used to train the mathematical model and the quality of the ophthalmic image input to the mathematical model, the lower the reliability of the medical data output from the mathematical model. However, the ophthalmic image processing device of this embodiment can change the method of acquiring medical data according to evaluation information indicating the suitability of the ophthalmic image (e.g., image quality, etc.). Therefore, medical data useful to doctors and the like can be more appropriately acquired.

[0016] However, the control unit may acquire medical data from the ophthalmological image without using a machine learning algorithm in the medical data acquisition step. For example, even when the above-mentioned medical data is acquired by performing image processing of the ophthalmological image using a function without using a machine learning algorithm, at least a part of the technology exemplified in the present disclosure can be applied.

[0017] The control unit may also output the ophthalmic image to another device (e.g., a server) in which a program for realizing the mathematical model is stored. The device to which the ophthalmic image is output may obtain medical data by inputting the ophthalmic image input from the ophthalmic image processing device to the mathematical model, and output the obtained medical data to the ophthalmic image processing device. The control unit may obtain the medical data input from the device to which the ophthalmic image is output.

[0018] When a machine learning algorithm is used, the mathematical model may be trained with a plurality of training data sets, in which ophthalmic images captured by an ophthalmic image capturing device are used as input training data, and medical data corresponding to the input training data are used as output training data. The criteria for the evaluation information may be set based on the plurality of input training data used to train the mathematical model. In this case, the criteria for the evaluation information are appropriately set based on the plurality of input training data used to train the mathematical model. For example, when determining whether the image quality of the ophthalmic image input to the mathematical model is close to the image quality of the input training data, a more appropriate judgment is made using the set criteria.

[0019] A specific method for setting the standard of the evaluation information based on the input training data can also be appropriately selected. For example, a value indicating the distribution of the evaluation information of a plurality of input training data (e.g., the average value and the standard deviation σ, etc.) may be obtained, and the boundary of whether the evaluation information falls within a predetermined range (e.g., within ±3σ, etc.) set from the obtained value may be set as the standard.

[0020] However, it is also possible to change the method of setting the criteria for the evaluation information. For example, a fixed criterion may be set in advance. When information indicating whether medical data has been appropriately acquired from each of a plurality of ophthalmological images is obtained, the criterion for the evaluation information may be set based on the relationship between the evaluation information of each ophthalmological image and the information indicating whether medical data has been appropriately acquired. The criterion may also be set according to an instruction input by a user.

[0021] Also, a criterion for comparing evaluation information of an ophthalmic image (i.e., a first criterion for judging suitability for acquiring medical data) and a criterion for judging whether or not a captured ophthalmic image is suitable as an image for a user to observe (i.e., a second criterion for judging whether or not the capture was successful) may be set separately. In this case, whether or not an ophthalmic image is suitable is more appropriately judged. Also, the control unit may notify the user of information indicating whether or not an ophthalmic image satisfies the first criterion and information indicating whether or not an ophthalmic image satisfies the second criterion, either simultaneously or separately. In this case, the user can appropriately grasp whether or not an ophthalmic image is suitable depending on the situation.

[0022] In the medical data acquisition step, the control unit may acquire the usage data by inputting information on the evaluation information of the ophthalmological image together with the ophthalmological image into the mathematical model. In this case, the medical data is output from the mathematical model after taking into account the evaluation information of the ophthalmological image. Therefore, the reliability of the output medical data is improved.

[0023] When the evaluation information is input to the mathematical model, the mathematical model may be trained in advance by a large number of input training data with different evaluation information. In this case, medical data is appropriately output from the mathematical model according to an algorithm corresponding to the evaluation information of the input ophthalmic image.

[0024] Furthermore, the information regarding the evaluation information input into the mathematical model may be the evaluation information itself, or may be information obtained from the evaluation information (for example, information indicating whether the evaluation information meets a standard, etc.).

[0025] In the medical data acquisition step, if the evaluation information of the ophthalmic image does not satisfy a standard, the control unit may execute an evaluation information improvement process on the ophthalmic image to bring the evaluation information of the ophthalmic image closer to the standard. The control unit may acquire medical data based on the ophthalmic image on which the evaluation information improvement process has been performed. In this case, even if the evaluation information (e.g., image quality, etc.) of the ophthalmic image acquired from the ophthalmic image capturing device does not satisfy a standard, the evaluation information improvement process is performed and medical data is acquired from the ophthalmic image. Thus, the reliability of the acquired medical data is appropriately improved.

[0026] The specific contents of the evaluation information improvement process can be appropriately selected. For example, at least one of known image processes such as noise removal and sharpening may be performed on the ophthalmic image. Furthermore, the evaluation information improvement process may be performed on the ophthalmic image using a machine learning algorithm. Furthermore, the control unit may acquire data such as an image of the same subject's eye or various measurement results acquired by an apparatus other than the ophthalmic image capturing apparatus that captured the ophthalmic image, and may execute the evaluation information improvement process using the acquired data.

[0027] The control unit may output an instruction to additionally capture the same part as the part from which the ophthalmic image was captured, if the evaluation information of the ophthalmic image does not satisfy the criteria in the medical data acquisition step. The control unit may acquire medical data based on the additionally captured ophthalmic image. In this case, the ophthalmic image of the same part is recaptured so that the evaluation information satisfies the criteria. Therefore, the reliability of the acquired medical data is appropriately improved.

[0028] In the medical data acquisition step, if the evaluation information of the ophthalmic image does not satisfy the criterion, the control unit may output an instruction to additionally capture the same part as the part from which the ophthalmic image was captured. The control unit may acquire medical data based on an ophthalmic image obtained by averaging the additionally captured images. In this case, medical data is acquired based on an ophthalmic image whose evaluation information has been brought closer to the criterion by the averaging process. Thus, the reliability of the acquired medical data is appropriately improved. Note that the control unit may execute the imaging process of the same part and the averaging process of the additionally captured images until the evaluation information satisfies the criterion. In this case, the reliability of the medical data is further improved.

[0029] The control unit may acquire a plurality of ophthalmological images of the same tissue of the same subject's eye in the image acquiring step. The control unit may acquire medical data based on at least one of the plurality of ophthalmological images whose evaluation information satisfies a criterion in the medical data acquiring step. In this case, an ophthalmological image used to acquire medical data is appropriately selected from the plurality of ophthalmological images of the same part based on the evaluation information. Thus, the reliability of the acquired medical data is appropriately improved.

[0030] The control unit may output warning information to a user when the evaluation information of the ophthalmic image does not satisfy a standard in the medical data acquisition step. In this case, the user can appropriately make various decisions such as whether or not a diagnosis or additional examination is necessary after understanding that the evaluation information of the ophthalmic image from which the medical data is acquired does not satisfy a standard.

[0031] A specific method for outputting the warning information may be appropriately selected. For example, the control unit may perform a process of acquiring medical data based on the ophthalmic image only when the evaluation information of the ophthalmic image satisfies a criterion, and may output the warning information without performing a process of acquiring medical data based on the ophthalmic image when the evaluation information does not satisfy the criterion. In this case, the user can make various decisions after understanding from the warning information that the evaluation information of the ophthalmic image does not satisfy the criterion (that is, that the reliability of the acquired medical data may be low). Furthermore, the control unit may output the warning information when the evaluation information does not satisfy the criterion, and may also accept an input of an instruction from the user as to whether or not to acquire medical data from the ophthalmic image whose evaluation information does not satisfy the criterion. When an instruction to acquire medical data is input, the control unit may acquire medical data based on the ophthalmic image whose evaluation information does not satisfy the criterion.

[0032] Furthermore, even if the evaluation information of the ophthalmologic image does not satisfy the criteria, the control unit may acquire medical data based on the ophthalmologic image and output the acquired medical data accompanied by warning information. In this case, the user can easily understand from the warning information that the medical data has been acquired based on the ophthalmologic image whose evaluation information does not satisfy the criteria.

[0033] The devices that execute the image acquiring step, the evaluation information acquiring step, and the medical data acquiring step may be appropriately selected. For example, a control unit of a personal computer (hereinafter referred to as "PC") may execute all of the image acquiring step, the evaluation information acquiring step, and the medical data acquiring step. That is, the control unit of the PC may acquire an ophthalmic image from an ophthalmic image capturing device, and perform an acquisition process of the evaluation information and an acquisition process of the medical data based on the acquired ophthalmic image. Also, a control unit of the ophthalmic image capturing device may execute all of the image acquiring step, the evaluation information acquiring step, and the medical data acquiring step. Also, control units of a plurality of devices (e.g., an ophthalmic image capturing device and a PC, etc.) may cooperate to execute the image acquiring step, the evaluation information acquiring step, and the medical data acquiring step. [Brief description of the drawings]

[0034] [Figure 1] 1 is a block diagram showing a schematic configuration of a mathematical model construction device 1, an ophthalmic image processing device 21, and ophthalmic image photographing devices 11A and 11B. [Diagram 2] 2 is a flowchart of a mathematical model construction process executed by the mathematical model construction device 1. [Diagram 3] 13 is a diagram showing an example of input training data when an analysis result of the structure of the fundus tissue of a test eye is output to a mathematical model. FIG. [Figure 4] 13 is a diagram showing an example of output training data when the analysis results of the structure of the fundus tissue of the subject's eye are output to a mathematical model. FIG. [Diagram 5] 13A and 13B are diagrams illustrating an example of input training data and output training data when a transformed image is output to a mathematical model. [Figure 6] 2 is a flowchart of ophthalmologic image processing executed by the ophthalmologic image processing system 100. [Figure 7] 4 is a flowchart of a medical data acquisition process executed by the ophthalmologic image processing system 100 of the first embodiment. [Figure 8] 10 is a flowchart of a medical data acquisition process executed by the ophthalmologic image processing system 100 of the second embodiment. [Figure 9] 13 is a flowchart of a medical data acquisition process executed by the ophthalmologic image processing system 100 of the third embodiment. [Figure 10] 13 is a flowchart of a medical data acquisition process executed by the ophthalmologic image processing system 100 of the fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0035] (Device configuration) A typical embodiment of the present disclosure will be described below with reference to the drawings. As shown in FIG. 1, in this embodiment, a mathematical model construction device 1, an ophthalmic image processing device 21, and ophthalmic image capturing devices 11A and 11B are used. The mathematical model construction device 1 constructs a mathematical model by training the mathematical model using a machine learning algorithm. A program for realizing the constructed mathematical model is stored in a storage device 24 of the ophthalmic image processing device 21. The ophthalmic image processing device 21 inputs an ophthalmic image to the mathematical model to obtain medical data. The ophthalmic image capturing devices 11A and 11B capture ophthalmic images, which are images of tissues of a subject's eye.

[0036] As an example, a personal computer (hereinafter referred to as "PC") is used as the mathematical model construction device 1 of this embodiment. Although details will be described later, the mathematical model construction device 1 constructs a mathematical model by training the mathematical model using an ophthalmic image (hereinafter referred to as "training ophthalmic image") acquired from the ophthalmic image capturing device 11A and medical data corresponding to the training ophthalmic image. However, a device that can function as the mathematical model construction device 1 is not limited to a PC. For example, the ophthalmic image capturing device 11A may function as the mathematical model construction device 1. Furthermore, control units of a plurality of devices (for example, a CPU of a PC and a CPU 13A of the ophthalmic image capturing device 11A) may cooperate to construct a mathematical model.

[0037] In the present embodiment, the ophthalmic image processing system 100 that acquires medical data based on an ophthalmic image includes an ophthalmic image processing device 21, which is a PC, and an ophthalmic image capturing device 11B. However, the configuration of the ophthalmic image processing system can be changed. For example, one of the ophthalmic image processing device 21 and the ophthalmic image capturing device 11B may function as an ophthalmic image processing system by itself. The ophthalmic image processing device 21 may be a device other than a PC (for example, a tablet terminal or a mobile terminal such as a smartphone). The control units of a plurality of devices may cooperate to function as the ophthalmic image processing device 21. A device (for example, a server, etc.) different from the ophthalmic image processing device 21 may input an ophthalmic image to a mathematical model to acquire medical data. In this case, the ophthalmic image processing device 21 may acquire medical data by outputting an ophthalmic image to a device in which a program that realizes the mathematical model is stored, and inputting medical data from the device.

[0038] In addition, in this embodiment, a CPU is used as an example of a controller that performs various processes. However, it goes without saying that a controller other than a CPU may be used for at least a part of the various devices. For example, a GPU may be used as a controller to speed up the processing.

[0039] The mathematical model construction device 1 will be described. The mathematical model construction device 1 is disposed, for example, at a manufacturer that provides an ophthalmic image processing device 21 or an ophthalmic image processing program to a user. The mathematical model construction device 1 includes a control unit 2 that performs various control processes, and a communication I / F 5. The control unit 2 includes a CPU 3 that is a controller that manages control, and a storage device 4 that can store programs, data, and the like. The storage device 4 stores a mathematical model construction program for executing a mathematical model construction process (see FIG. 2) described later. In addition, the communication I / F 5 connects the mathematical model construction device 1 to other devices (for example, an ophthalmic image capturing device 11A and an ophthalmic image processing device 21, etc.).

[0040] The mathematical model construction device 1 is connected to an operation unit 7 and a display device 8. The operation unit 7 is operated by a user to input various instructions to the mathematical model construction device 1. For example, at least one of a keyboard, a mouse, a touch panel, etc. can be used for the operation unit 7. Note that a microphone or the like for inputting various instructions may be used together with or instead of the operation unit 7. The display device 8 displays various images. For the display device 8, various devices capable of displaying images (for example, at least one of a monitor, a display, a projector, etc.) can be used. Note that, in this disclosure, "image" includes both still images and moving images.

[0041] The mathematical model construction device 1 can acquire data of ophthalmic images (hereinafter, may be simply referred to as "ophthalmic images") from the ophthalmic image capturing device 11A. The mathematical model construction device 1 may acquire data of ophthalmic images from the ophthalmic image capturing device 11A by at least one of wired communication, wireless communication, a removable storage medium (e.g., USB memory), and the like.

[0042] The ophthalmic image processing device 21 will be described. The ophthalmic image processing device 21 is disposed in, for example, a facility (for example, a hospital or a medical examination facility) where a diagnosis or examination is performed on a subject. The ophthalmic image processing device 21 includes a control unit 22 that performs various control processes, and a communication I / F 25. The control unit 22 includes a CPU 23 that is a controller that manages control, and a storage device 24 that can store programs, data, and the like. The storage device 24 stores an ophthalmic image processing program for executing ophthalmic image processing (see FIG. 6) described later. The ophthalmic image processing program includes a program for realizing a mathematical model constructed by the mathematical model construction device 1. The communication I / F 25 connects the ophthalmic image processing device 21 to other devices (for example, the ophthalmic image capturing device 11B and the mathematical model construction device 1, etc.).

[0043] The ophthalmologic image processing device 21 is connected to an operation unit 27 and a display device 28. As with the operation unit 7 and the display device 8 described above, various devices can be used for the operation unit 27 and the display device 28.

[0044] The ophthalmological image processing device 21 can acquire ophthalmological images from the ophthalmological image capturing device 11B. The ophthalmological image processing device 21 may acquire ophthalmological images from the ophthalmological image capturing device 11B by at least one of wired communication, wireless communication, a removable storage medium (e.g., USB memory), etc. The ophthalmological image processing device 21 may also acquire a program or the like that realizes the mathematical model constructed by the mathematical model construction device 1 via communication or the like.

[0045] The ophthalmic image capturing devices 11A and 11B will be described. As an example, in this embodiment, a case will be described in which an ophthalmic image capturing device 11A that provides ophthalmic images to the mathematical model construction device 1 and an ophthalmic image capturing device 11B that provides ophthalmic images to the ophthalmic image processing device 21 are used. However, the number of ophthalmic image capturing devices used is not limited to two. For example, the mathematical model construction device 1 and the ophthalmic image processing device 21 may acquire ophthalmic images from a plurality of ophthalmic image capturing devices. Also, the mathematical model construction device 1 and the ophthalmic image processing device 21 may acquire ophthalmic images from one common ophthalmic image capturing device.

[0046] In the present embodiment, an OCT device is exemplified as the ophthalmic imaging device 11 (11A, 11B). However, an ophthalmic imaging device other than an OCT device (for example, a scanning laser ophthalmoscope (SLO), a fundus camera, a Scheimpflug camera, or a corneal endothelial cell imaging device (CEM)) may be used.

[0047] The ophthalmic image capturing device 11 (11A, 11B) includes a control unit 12 (12A, 12B) that performs various control processes, and an ophthalmic image capturing section 16 (16A, 16B). The control unit 12 includes a CPU 13 (13A, 13B) that is a controller that manages control, and a storage device 14 (14A, 14B) that can store programs, data, and the like. When the ophthalmic image capturing device 11 executes at least a part of the ophthalmic image processing (see FIG. 4) described below, it goes without saying that at least a part of the ophthalmic image processing program for executing the ophthalmic image processing is stored in the storage device 14.

[0048] The ophthalmologic image capturing unit 16 includes various components necessary for capturing an ophthalmologic image of the subject's eye. The ophthalmologic image capturing unit 16 of this embodiment includes an OCT light source, a branching optical element that branches the OCT light emitted from the OCT light source into measurement light and reference light, a scanning unit that scans the measurement light, an optical system that irradiates the subject's eye with the measurement light, and a light receiving element that receives a composite light of the light reflected by the tissue of the subject's eye and the reference light.

[0049] The ophthalmologic image capturing device 11 can capture two-dimensional tomographic images and three-dimensional tomographic images of the fundus of the subject's eye. In detail, the CPU 13 captures two-dimensional tomographic images (see Figs. 3 and 5) of a cross section intersecting the scan line by scanning the OCT light (measurement light) on the scan line. The CPU 13 can also capture three-dimensional tomographic images of tissue by two-dimensionally scanning the OCT light. For example, the CPU 13 acquires a plurality of two-dimensional tomographic images by scanning the measurement light on each of a plurality of scan lines at different positions within a two-dimensional area when the tissue is viewed from the front. Next, the CPU 13 acquires a three-dimensional tomographic image by combining the captured two-dimensional tomographic images.

[0050] Furthermore, the CPU 13 scans the same part of the tissue (on the same scan line in this embodiment) with the measurement light multiple times to capture multiple ophthalmic images of the same part. The CPU 13 performs averaging on multiple ophthalmic images of the same part to obtain an averaged image in which the influence of speckle noise is suppressed. The averaging may be performed, for example, by averaging pixel values ​​of pixels at the same position among multiple ophthalmic images. The more images are subjected to the averaging process, the easier it is to suppress the influence of speckle noise, but the longer the imaging time. Note that the ophthalmic image capturing device 11 performs tracking processing to make the scanning position of the OCT light follow the movement of the subject's eye while capturing multiple ophthalmic images of the same part.

[0051] (Mathematical model construction process) 2 to 5, a description will be given of a mathematical model construction process executed by the mathematical model construction device 1. The mathematical model construction process is executed by the CPU 3 in accordance with a mathematical model construction program stored in the storage device 4.

[0052] In the mathematical model construction process, a mathematical model is trained by a training dataset, thereby constructing a mathematical model that outputs medical data based on an ophthalmic image. The training dataset includes input data (input training data) and output data (output training data). The mathematical model can output various medical data. The type of training dataset used to train the mathematical model is determined according to the type of medical data to be output by the mathematical model. An example of the relationship between the type of medical data to be output and the training dataset used to train the mathematical model will be described below.

[0053] First, a case will be described in which an ophthalmic image is input to the mathematical model to output an analysis result of the structure of the tissue of the subject's eye to the mathematical model. In this case, the mathematical model is trained using an ophthalmic image of the tissue of the subject's eye as input training data and medical data indicating the structure of the tissue captured in the input training data (ophthalmic image) as output training data. FIGS. 3 and 4 show examples of input training data and output training data in a case in which an analysis result of the structure of the fundus tissue of the subject's eye is output to the mathematical model as medical data. The input training data 30 shown in FIG. 3 is a two-dimensional tomographic image of the fundus tissue. A plurality of layers in the fundus tissue are shown in the input training data 30. Moreover, the output training data 31 shown in FIG. 4 shows the position of the structure of the tissue captured in the input training data 30. As an example, the output training data 31 shown in FIG. 4 includes data of labels 32A to 32F indicating the positions of six boundaries captured in the input training data 30 (see FIG. 3). As a result, when an ophthalmologic image is input to the trained mathematical model, an analysis result of at least one of tissue layers and boundaries is output as medical data. In this embodiment, data of labels 32A to 32F in the output training data 31 is generated by an operator operating the operation unit 7 while looking at the boundaries in the input training data 30. However, the method of generating the label data can be changed.

[0054] In addition, it is also possible to output the analysis results of the structure of the tissue of the subject's eye other than the layers and boundaries to the mathematical model. For example, the analysis results of the fundus blood vessels of the subject's eye can be output to the mathematical model. Furthermore, the analysis results of the fundus arteries and veins can be output to the mathematical model. In this case, for example, the mathematical model may be trained using an ophthalmic image showing the fundus blood vessels as input training data, and data showing the positions of the blood vessels (arteries and veins) shown in the input training data as output training data. It is also possible to output the analysis results of the macula or fovea in the fundus to the mathematical model. The input training data does not need to be a tomographic image of the tissue of the subject's eye, and may be a two-dimensional front image or the like.

[0055] Next, a case will be described in which an ophthalmic image is input to a mathematical model, and a converted image obtained by converting the input image is output to the mathematical model. As described above, the converted image indicates at least one of an image in which noise of the input ophthalmic image is reduced, an image in which the resolution of the ophthalmic image is increased, an image in which the visibility of the ophthalmic image is improved, and an image in which a specific structure in the ophthalmic image is emphasized. As an example, in this embodiment, an ophthalmic image of the tissue of the subject eye is used as input training data, and an ophthalmic image of the same part having higher image quality (e.g., reduced noise) than the input training data is used as output training data to train the mathematical model. FIG. 5 shows an example of input training data and output training data when a converted image is output to the mathematical model. In the example shown in FIG. 5, the CPU 3 obtains a set 40 of a plurality of training ophthalmic images 400A to 400X obtained by photographing the same part of the tissue. The CPU 3 uses a part of the plurality of training ophthalmic images 400A to 400X in the set 40 (a number smaller than the number used for arithmetic averaging of the output training data described later) as input training data. Furthermore, the CPU 3 acquires, as output training data, an arithmetic average image 41 of the multiple training ophthalmological images 400A-400X in the set 40. When a mathematical model is trained using the input training data and output training data exemplified in Fig. 5, an ophthalmological image is input to the trained mathematical model, and a high-quality converted image in which the influence of speckle noise is suppressed is output.

[0056] It is also possible to change the method of generating high-quality output training data. For example, the output training data may be generated by improving the image quality of the input training data by a process other than averaging. The type of output training data may be appropriately selected according to the type of converted image to be output by the mathematical model. The input training data is not limited to tomographic images of tissue.

[0057] Next, a case will be described in which an ophthalmic image is input to a mathematical model, and an automatic analysis result indicating whether or not the subject's eye has a disease is output from the mathematical model. In this case, for example, an ophthalmic image of the tissue of the subject's eye is used as input training data, and data indicating the presence or absence of a disease in the photographed subject shown in the input training data is used as output training data to train the mathematical model.

[0058] The mathematical model construction process will be described with reference to Fig. 2. The CPU 3 acquires data of an ophthalmic image (training ophthalmic image) captured by the ophthalmic image capturing device 11A as input training data (S1). In this embodiment, the data of the training ophthalmic image is generated by the ophthalmic image capturing device 11A and then acquired by the mathematical model construction device 1. However, the CPU 3 may acquire the data of the training ophthalmic image by acquiring a signal (e.g., an OCT signal) that is the basis for generating the training ophthalmic image from the ophthalmic image capturing device 11A and generating the training ophthalmic image based on the acquired signal.

[0059] Next, the CPU 3 acquires evaluation information of the training ophthalmic image acquired in S1, and stores it in the storage device 4 (S2). The evaluation information indicates the degree of suitability (suitability) of the ophthalmic image for acquiring the above-mentioned medical data. In general, the better the image quality of the ophthalmic image, the easier it is to acquire appropriate medical data from the ophthalmic image. Therefore, in the present disclosure, as an example, a value related to the image quality of the ophthalmic image is used as the evaluation information of the ophthalmic image. For example, the signal strength of the ophthalmic image, or an index indicating the quality of the signal (e.g., Signal Strength Index (SSI) or Quality Index (QI)) may be used as the evaluation information of the ophthalmic image. In addition, at least one of the ratio of the noise level to the signal level of the image (SNR (Signal to Noise Ratio), the background noise level, the image contrast, etc.) may be used as evaluation information for the ophthalmic image. In addition, the shooting conditions when the ophthalmic image is captured by the ophthalmic image capturing device 11A (for example, at least one of the scanning conditions and the exposure time when capturing an ophthalmic image by scanning light on tissue) may be used as evaluation information. In addition, when an ophthalmic image is obtained by performing the above-mentioned averaging process, the number of images used in the averaging process (hereinafter referred to as the "number of images added") may be used as evaluation information.

[0060] Next, the CPU 3 acquires medical data corresponding to the input training data acquired in S1 as output training data (S3). An example of the correspondence between the input training data and the output training data (medical data) is as described above.

[0061] Next, the CPU 3 executes training of the mathematical model using the training data set by a machine learning algorithm (S4). Commonly known machine learning algorithms include, for example, neural networks, random forests, boosting, and support vector machines (SVMs).

[0062] Neural networks are a method to mimic the behavior of biological neural networks. Examples of neural networks include feedforward neural networks, radial basis function (RBF) networks, spiking neural networks, convolutional neural networks, recurrent neural networks (recurrent neural networks, feedback neural networks, etc.), and probabilistic neural networks (Boltzmann machines, Bayesian networks, etc.).

[0063] Random forest is a method to generate a large number of decision trees by learning based on randomly sampled training data. When using random forest, the branches of multiple decision trees that have been trained in advance as classifiers are traced, and the average (or majority vote) of the results obtained from each decision tree is taken.

[0064] Boosting is a method to generate a strong classifier by combining multiple weak classifiers. A strong classifier is constructed by sequentially training simple weak classifiers.

[0065] SVM is a method for constructing a two-class pattern classifier using linear input elements. For example, SVM learns the parameters of the linear input elements based on the criterion of finding the margin-maximizing hyperplane that maximizes the distance from each data point from the training data (hyperplane separation theorem).

[0066] A mathematical model refers to, for example, a data structure for predicting the relationship between input data and output data. A mathematical model is constructed by being trained using a training data set. As described above, a training data set is a set of input training data and output training data. For example, the correlation data (e.g., weights) between each input and output is updated by training.

[0067] In this embodiment, a multi-layered neural network is used as the machine learning algorithm. The neural network includes an input layer for inputting data, an output layer for generating data to be predicted, and one or more hidden layers between the input layer and the output layer. A plurality of nodes (also called units) are arranged in each layer. In detail, in this embodiment, a convolutional neural network (CNN), which is a type of multi-layered neural network, is used. However, other machine learning algorithms may be used. For example, generative adversarial networks (GAN), which utilize two competing neural networks, may be adopted as the machine learning algorithm.

[0068] The processes of S1 to S4 are repeated until the construction of the mathematical model is completed (S5: NO). When the construction of the mathematical model is completed (S5: YES), the mathematical model construction process ends. The program and data for realizing the constructed mathematical model are incorporated into the ophthalmologic image-processing device 21.

[0069] (Ophthalmology Image Processing) With reference to Figs. 6 to 10, the ophthalmic image processing performed by the ophthalmic image processing system 100 will be described. In the ophthalmic image processing, medical data on the subject eye captured in the ophthalmic image is acquired based on the ophthalmic image. In this embodiment, the ophthalmic image processing device 21 included in the ophthalmic image processing system 100 executes the ophthalmic image processing illustrated in Figs. 6 to 10. However, as described above, the CPU of another device included in the ophthalmic image processing system 100 (e.g., the CPU 13B of the ophthalmic image capturing device 11B, etc.) may execute the ophthalmic image processing. In addition, a plurality of control units included in the ophthalmic image processing system 100 (e.g., the CPU 23 of the ophthalmic image processing device 21 and the CPU 13B of the ophthalmic image capturing device 11B) may cooperate to execute the ophthalmic image processing. The ophthalmic image processing of this embodiment is executed by the CPU 23 according to an ophthalmic image processing program stored in the storage device 24.

[0070] As shown in Fig. 6, the CPU 23 sets a criterion for evaluation information of an ophthalmic image (S11). Although details will be described later, the CPU 23 changes the method of acquiring medical data depending on whether the evaluation information of the ophthalmic image for acquiring medical data satisfies the criterion. In this embodiment, when determining whether the ophthalmic image for acquiring medical data is appropriate for acquiring medical data, the evaluation information of the ophthalmic image is compared with the criterion.

[0071] As an example, in S1 of the present embodiment, a criterion for the evaluation information is set based on a plurality of input training data used in training the mathematical model. The evaluation information of the input training data used in training the mathematical model is stored in the above-mentioned mathematical model construction process (S2 in FIG. 2). The CPU 23 sets the criterion based on the stored evaluation information of the input training data. A specific method for setting the criterion can be appropriately selected. As an example, in the present embodiment, the CPU 23 acquires values ​​(average value and standard deviation σ) indicating the distribution of the evaluation information of a plurality of input training data, and sets a boundary value indicating whether the evaluation information falls within a predetermined range (for example, within ±3σ) set from the acquired value as the criterion. However, it is also possible to change the method for setting the criterion. For example, the CPU 23 may set the criterion according to an instruction input by a user. A control unit of a device other than the ophthalmologic image processing device 21 (for example, the CPU 3 of the mathematical model construction device 1, etc.) may set the criterion for the evaluation information. In addition, a fixed criterion may be set in advance. In this embodiment, a criterion for comparing evaluation information of ophthalmic images (i.e., a first criterion for judging suitability for acquiring medical data) and a criterion for judging whether a captured ophthalmic image is suitable as an image for a user to observe (i.e., a second criterion for judging whether the capture was successful) are set separately. The CPU 23 can notify the user (for example, display on the display device 28) information indicating whether the ophthalmic image satisfies the first criterion and information indicating whether the ophthalmic image satisfies the second criterion simultaneously or separately.

[0072] Next, the CPU 23 acquires an ophthalmic image of the subject's eye captured by the ophthalmic image capturing device 11B (S12). The ophthalmic image acquired in S12 is the same type of image as the ophthalmic image used as input training data in training the mathematical model. That is, when a two-dimensional tomographic image is used as input training data, the ophthalmic image acquired in S12 is also a two-dimensional tomographic image. When a two-dimensional front image is used as input training data, the ophthalmic image acquired in S12 is also a two-dimensional front image. Note that in S12, a plurality of ophthalmic images of the same subject's eye may be acquired, or a single ophthalmic image may be acquired.

[0073] Next, the CPU 23 acquires evaluation information about the ophthalmic image acquired in S12 (S13). As described above, the evaluation information indicates the degree (suitability) of whether the ophthalmic image is suitable for acquiring medical data. In this embodiment, as described above, various values ​​related to the image quality of the ophthalmic image can be used as the evaluation information.

[0074] Next, the CPU 23 acquires medical data based on the ophthalmologic image acquired in S12 (S14). Here, in S14, the method of acquiring medical data is changed depending on whether the evaluation information acquired in S13 satisfies a criterion. Hereinafter, the medical data acquisition process in each of the first to fourth embodiments will be described in detail with reference to Figs. 7 to 10.

[0075] (First embodiment) The medical data acquisition process executed by the ophthalmic image processing system 100 of the first embodiment will be described with reference to Fig. 7. In the first embodiment, information on the evaluation information of the ophthalmic image is input to a mathematical model together with the ophthalmic image, thereby improving the reliability of the medical data. Furthermore, when the evaluation information of the ophthalmic image does not satisfy a standard, warning information is output to the user.

[0076] First, the CPU 23 inputs information on the evaluation information and the ophthalmic image to the mathematical model (S21). The information on the evaluation information may be the evaluation information itself acquired in S3 of FIG. 6, or may be information obtained from the evaluation information acquired in S3 (for example, information indicating whether the evaluation information meets a standard). Note that the mathematical model in the first embodiment is trained in advance by a large number of input training data with different evaluation information in the mathematical model construction process (see FIG. 2). When the information on the evaluation information and the ophthalmic image are input to the mathematical model, the mathematical model outputs medical data according to an algorithm corresponding to the evaluation information of the ophthalmic image. Therefore, there is little difference between the evaluation information of the input training data used to construct the algorithm and the evaluation information of the ophthalmic image input to the mathematical model. Therefore, the reliability of the medical data is easily improved.

[0077] The CPU 23 acquires medical data output from the mathematical model (S22). The CPU 23 determines whether the evaluation information of the ophthalmological image input to the mathematical model in S21 meets the criteria (S23). If the evaluation information meets the criteria (S23: YES), the reliability of the medical data acquired in S22 is often high. Therefore, the CPU 23 outputs the medical data acquired in S22 as is (S24). On the other hand, if the evaluation information does not meet the criteria (S23: NO), the reliability of the medical data acquired in S22 may be low. Therefore, the CPU 23 outputs the medical data acquired in S22 together with warning information for warning the user (S25). As a result, the user can easily understand from the warning information that the medical data has been acquired based on the ophthalmological image whose evaluation information does not meet the criteria.

[0078] Second embodiment With reference to FIG. 8, the medical data acquisition process executed by the ophthalmic image processing system 100 of the second embodiment will be described. In the second embodiment, when the evaluation information of the ophthalmic image does not satisfy the criteria, an evaluation information improvement process (image quality improvement process in the present embodiment as an example) is executed on the ophthalmic image. This improves the reliability of the medical data. Furthermore, when the evaluation information of the ophthalmic image does not satisfy the criteria, warning information is output to the user before the medical data is acquired. Furthermore, when the evaluation information of the ophthalmic image does not satisfy the criteria, an additional image of the same part of the examinee's eye is captured, and the ophthalmic image is acquired again.

[0079] First, the CPU 23 determines whether the evaluation information of the ophthalmic image satisfies a criterion (S31). If the criterion is satisfied (S31: YES), the ophthalmic image is input to a mathematical model (S38), and medical data output from the mathematical model is acquired and output (S39).

[0080] If the evaluation information does not satisfy the standard (S31: NO), the CPU 23 executes an evaluation information improvement process for the ophthalmic image from which medical data is obtained, to bring the evaluation information closer to the standard (S32). In this embodiment, the CPU 23 executes an image quality improvement process for improving image quality as an example of the evaluation information improvement process. For example, at least one of a well-known noise removal process, a sharpening process, etc. may be adopted for the image quality improvement process. Furthermore, the evaluation information improvement process for the ophthalmic image may be performed using a machine learning algorithm. Furthermore, the CPU 23 may execute the evaluation information improvement process for the ophthalmic image using an image of the same subject eye or various measurement results acquired by a device other than the ophthalmic image capturing device 11B (in this embodiment, an ophthalmic imaging device other than the OCT device).

[0081] The CPU 23 acquires the evaluation information of the ophthalmic image that has been subjected to the evaluation information improvement process, and judges whether the evaluation information satisfies a criterion (S33). If the criterion is satisfied (S33: YES), the ophthalmic image is input to a mathematical model (S38), and medical data output from the mathematical model is acquired and output (S39).

[0082] If the evaluation information does not satisfy the criteria (S33: NO), the CPU 23 outputs warning information to the user (S34) without performing the process of acquiring medical data (S38, S39). The warning information may be output by displaying it on the display device 28 or by sound.

[0083] Next, the CPU 23 judges whether or not an instruction to acquire medical data from an ophthalmic image whose evaluation information does not satisfy the standard has been input by the user (S35). In this embodiment, when the evaluation information of the ophthalmic image does not satisfy the standard, the user can input an instruction to acquire medical data from the ophthalmic image that does not satisfy the standard, or an instruction to perform additional photography of an ophthalmic image of the same subject eye, via the operation unit 27, etc. If an instruction to acquire medical data has been input (S35: YES), the ophthalmic image is input to a mathematical model (S38), and medical data output from the mathematical model is acquired and output (S39). On the other hand, if an instruction to perform additional photography has been input (S35: NO), the CPU 23 outputs an instruction to additionally photograph the same site to the ophthalmic image capturing device 11B (S36). The CPU 23 reacquires the additionally photographed ophthalmic image (S37). Thereafter, the process returns to S31. If an ophthalmic image whose evaluation information approaches the standard is reacquired in S37, the reliability of the medical data acquired in S38 and S39 is improved.

[0084] Third embodiment The medical data acquisition process executed by the ophthalmologic image processing system 100 of the third embodiment will be described with reference to Fig. 9. In the third embodiment, when the evaluation information of the ophthalmologic image does not satisfy the standard, an additional image of the same part of the subject's eye is captured, and an arithmetic averaging process is performed on the original ophthalmologic image. As a result, the reliability of the acquired medical data is improved.

[0085] First, the CPU 23 determines whether the evaluation information of the ophthalmic image satisfies a criterion (S41). If the criterion is satisfied (S41: YES), the ophthalmic image is input to a mathematical model (S44), and medical data output from the mathematical model is acquired and output (S45).

[0086] If the evaluation information of the ophthalmic image does not satisfy the criteria (S41: NO), the CPU 23 outputs an instruction to the ophthalmic image taking device 11B to additionally take an image of the same part as the photographed part of the original ophthalmic image (the image acquired in S12 of FIG. 6) (S42). Next, the CPU 23 performs an averaging process of the additionally taken ophthalmic image on the original ophthalmic image (S42). As described above, the more images used in the averaging process, the higher the image quality. The averaging process may be performed by the ophthalmic image taking device 11B or the ophthalmic image processing device 21. The process returns to S41, and the additional photographing (S42) and the averaging process (S43) are repeated until the evaluation information satisfies the criteria. If the evaluation information satisfies the criteria (S41: YES), medical data is acquired by the ophthalmic image generated by the averaging process and having improved evaluation information (S44, S45).

[0087] (Fourth embodiment) A medical data acquisition process executed by the ophthalmic image processing system 100 of the fourth embodiment will be described with reference to Fig. 10. In the fourth embodiment, medical data is acquired based on an ophthalmic image whose evaluation information satisfies a criterion, among a plurality of ophthalmic images obtained by photographing the same tissue of the same subject's eye. In the fourth embodiment, a plurality of ophthalmic images obtained by photographing the same tissue of the same subject's eye are acquired in the process of S12 in Fig. 6.

[0088] First, the CPU 23 judges whether or not there is an ophthalmic image whose evaluation information satisfies the criteria among the multiple ophthalmic images acquired in S12 of Fig. 6 (S51). If there is no ophthalmic image whose evaluation information satisfies the criteria (S51: NO), output of warning information (for example, see S34 of Fig. 8) and processing of additional photography (for example, see S35 to S37 of Fig. 8) are appropriately executed. If there is an ophthalmic image whose evaluation information satisfies the criteria (S51: YES), the CPU 23 inputs at least one of the ophthalmic images that satisfies the criteria to a mathematical model (S52). Medical data output from the mathematical model is acquired and output (S53).

[0089] The techniques disclosed in the above embodiments are merely examples. Therefore, the techniques exemplified in the above embodiments can be modified. First, it is possible to execute only a part of the techniques exemplified in the above embodiments. For example, in the medical data acquisition process in the first embodiment (see FIG. 7), a process (S21) of inputting information on evaluation information together with an ophthalmological image into a mathematical model and a process (S25) of outputting warning information are both executed. However, only one of S21 and S25 may be executed. Similarly, in the medical data acquisition process in the second embodiment (see FIG. 8), an evaluation information improvement process (S31, S32), a process (S33, S34) of outputting warning information, and an additional photographing process (S35 to S37) are executed. However, only one or two of these processes may be executed.

[0090] It is also possible to combine and execute a plurality of techniques exemplified in each of the first to fourth embodiments. For example, before the ophthalmic image is input to the mathematical model in S21 in the first embodiment (see FIG. 7), the evaluation information improvement process exemplified in S31 and S32 in the second embodiment (see FIG. 8) may be executed. In the above embodiments, the evaluation information of the entire ophthalmic image is compared with a criterion, and then various processes are executed. However, it may be determined whether or not the evaluation information of a part of the ophthalmic image (may be pixels) satisfies the criterion. In other words, it may be determined whether or not the evaluation information satisfies the criterion for each region of the ophthalmic image, and then various processes are executed.

[0091] The process of acquiring an ophthalmologic image in S12 of Fig. 6 is an example of an "image acquiring step". The process of acquiring evaluation information of the ophthalmologic image in S13 of Fig. 6 is an example of an "evaluation information acquiring step". The medical data acquiring process shown in Figs. 7 to 10 is an example of a "medical data acquiring step". The averaging process shown in S41 to S43 of Fig. 9 is an example of an "addition step". [Explanation of symbols]

[0092] 11A, 11B Ophthalmic imaging device 13A, 13B CPU 14A,14B Storage device 21 Ophthalmological image processing device 23 CPU 24 Storage device 100 Ophthalmic image processing system

Claims

1. An ophthalmic image processing device that processes an ophthalmic image which is an image of tissue of a subject's eye, The control unit of the ophthalmologic image processing device an image acquiring step of acquiring an ophthalmic image captured by an ophthalmic image capturing device; an evaluation information acquisition step of acquiring evaluation information indicating suitability for acquiring at least any one of medical data of an analysis result on a disease of the subject's eye, an analysis result on a structure of the subject's eye, and a converted image obtained by converting the acquired ophthalmic image, for the ophthalmic image acquired in the image acquisition step; a medical data acquisition step of acquiring the medical data by inputting the ophthalmic image acquired in the image acquisition step into a mathematical model trained by a machine learning algorithm; Run the mathematical model is trained with a plurality of training data sets, the training data being input ophthalmic images captured by an ophthalmic image capturing device and the medical data corresponding to the input training data, and the training data being output; An ophthalmologic image processing device characterized in that, in the medical data acquisition step, the control unit changes the method of acquiring the medical data depending on whether the evaluation information acquired in the evaluation information acquisition step satisfies a standard set based on a plurality of input training data used to train the mathematical model.

2. 2. The ophthalmologic image processing device according to claim 1, The ophthalmological image processing device, characterized in that in the medical data acquisition step, the control unit acquires the medical data by inputting information regarding the evaluation information of the ophthalmological image into the mathematical model together with the ophthalmological image.

3. 3. The ophthalmologic image processing device according to claim 1, The control unit, in the medical data acquisition step, When the evaluation information of the ophthalmological image does not satisfy a standard, an evaluation information improvement process is performed on the ophthalmological image to bring the evaluation information of the ophthalmological image closer to the standard; 2. An ophthalmologic image processing apparatus comprising: an ophthalmologic image processing device that acquires the medical data based on the ophthalmologic image that has been subjected to the evaluation information improvement processing.

4. 4. An ophthalmologic image processing apparatus according to claim 1, The control unit, in the medical data acquisition step, outputting an instruction to additionally photograph the same region as the region in which the ophthalmologic image was photographed when the evaluation information of the ophthalmologic image does not satisfy a criterion; An ophthalmologic image processing apparatus, comprising: an ophthalmologic image processing device for acquiring the medical data based on an additionally captured ophthalmologic image.

5. 4. An ophthalmologic image processing apparatus according to claim 1, The control unit, in the medical data acquisition step, outputting an instruction to additionally photograph the same region as the region in which the ophthalmologic image was photographed when the evaluation information of the ophthalmologic image does not satisfy a criterion; An ophthalmologic image processing apparatus, comprising: an ophthalmologic image processing device for acquiring the medical data based on an ophthalmologic image obtained by averaging additionally captured images.

6. An ophthalmic image processing program executed by an ophthalmic image processing device for processing an ophthalmic image which is an image of tissue of a subject's eye, The ophthalmologic image processing program is executed by a control unit of the ophthalmologic image processing device, an image acquiring step of acquiring an ophthalmic image captured by an ophthalmic image capturing device; an evaluation information acquisition step of acquiring evaluation information indicating suitability for acquiring at least any one of medical data of an analysis result on a disease of the subject's eye, an analysis result on a structure of the subject's eye, and a converted image obtained by converting the acquired ophthalmic image, for the ophthalmic image acquired in the image acquisition step; a medical data acquisition step of acquiring the medical data by inputting the ophthalmic image acquired in the image acquisition step into a mathematical model trained by a machine learning algorithm; and causing the ophthalmologic image processing device to execute the above. the mathematical model is trained with a plurality of training data sets, the training data being input ophthalmic images captured by an ophthalmic image capturing device and the medical data corresponding to the input training data, and the training data being output; An ophthalmologic image processing program characterized in that, in the medical data acquisition step, a method of acquiring the medical data is changed depending on whether the evaluation information acquired in the evaluation information acquisition step satisfies a standard set based on a plurality of input training data used to train the mathematical model.

7. An ophthalmic image processing system for processing an ophthalmic image which is an image of tissue of a subject's eye, comprising: The ophthalmologic image processing system includes: The OCT apparatus includes an optical coherence tomography (OCT) device that captures an ophthalmic image, which is a tomographic image of a tissue of a test eye, by receiving interference light between a reference light and a reflected light of a measurement light irradiated onto the tissue of the test eye, The control unit of the ophthalmologic image processing system includes: An image acquisition step of acquiring an ophthalmic image captured by the OCT device; an evaluation information acquisition step of acquiring evaluation information indicating suitability for acquiring at least any one of medical data of an analysis result on a disease of the subject's eye, an analysis result on a structure of the subject's eye, and a converted image obtained by converting the acquired ophthalmic image, for the ophthalmic image acquired in the image acquisition step; a medical data acquisition step of acquiring the medical data by inputting the ophthalmic image acquired in the image acquisition step into a mathematical model trained by a machine learning algorithm; Run the mathematical model is trained with a plurality of training data sets, the training data being input ophthalmic images captured by an ophthalmic image capturing device and the medical data corresponding to the input training data, and the training data being output; The control unit further includes: When the evaluation information acquired in the evaluation information acquisition step does not satisfy a criterion set based on the plurality of input training data used in training the mathematical model, an addition step is executed to additionally capture an image of the same region as the region from which the ophthalmic image was captured, and to perform an arithmetic averaging process of the additionally captured image on the ophthalmic image; An ophthalmologic image processing system, characterized in that in the medical data acquisition step, the ophthalmologic image on which the arithmetic averaging has been performed is input into a mathematical model trained by a machine learning algorithm to acquire the medical data.

Images