Theory-Driven Domain Control for an Ophthalmic Prediction Method Based on Machine Learning

Abstract

The present invention relates to a computer-implemented method for identifying the refractive power of an intraocular lens to be inserted. The method includes the steps of providing a physical model for identifying the refractive power and training a machine learning system to form a learning model for identifying the refractive power with clinical ophthalmic training data and an associated desired outcome. A loss function for training includes two components: a first component of the loss function considers the clinical ophthalmic training data and the associated desired outcome, and a second component of the loss function considers the limitations of the physical model such that the larger the loss function component value of the second component, the more the predicted refractive power during training deviates from the result of the physical model when the same clinical ophthalmic training data is used as input. Furthermore, the method includes the steps of providing ophthalmic data of a patient and predicting the refractive power of an intraocular lens to be used with the trained machine learning system, the provided ophthalmic data being used as input data.

Description

Technical Field

[0001] The present invention relates to a method for determining the refractive power of an intraocular lens, and in particular for determining the refractive power of an intraocular lens to be inserted by a learning model based on a specific loss function, a corresponding system, and a computer program product corresponding thereto for performing this method.

Background Art

[0002] The replacement of the eye's biological lens with an artificial intraocular lens (IOL) is performed, for example, in cases of refractive anomalies (such as due to aging) or cataracts, and has become increasingly common in the field of ophthalmology in recent years. In this process, the biological lens is separated from the lens capsule by a non-invasive intervention and removed. And in the case of cataracts, the opaque lens is replaced with an artificial lens implant. In this process, this artificial lens implant, i.e., the intraocular lens, is inserted into the now-empty lens capsule. Knowledge of the exact position and the required refractive power of the intraocular lens are interdependent.

[0003] In known and currently available IOL calculation formulas, physical models with different complexities are utilized (for example, the convergence principle of the known Haigis formula). In this way, the determination of the available specific refractive power of the IOL can be carried out not only based on data but also by leveraging the available physical knowledge. Although the accuracy is slightly improved, these formulas are always only approximations and cannot reproduce the very complex reality of the biological eye. By using the ray tracing method, the accuracy of the model is more easily improved compared to many other old models that only function in paraxial approximation. However, in that case too, for example, due to the shape of the refractive interface, approximations are included in this system. Regarding the availability of data, physical models can be fine-tuned or adjusted using various parameters. However, the structure of the models and the selection of these parameters are specified by their respective developers and thus are not necessarily the best representation. The optimal adaptation of the entire system in this form is very limited, and its flexibility is restricted by the model selected.

Summary of the Invention

Problems to be Solved by the Invention

[0004] Proceeding from the drawbacks of known methods for approximately determining the correct refractive power of the IOL to be inserted, the underlying objective presented in this application is to disclose an improved method and system for IOL refractive power prediction for intraocular lenses.

Means for Solving the Problems

[0005] This objective is achieved by the method according to the independent patent claims proposed here, the corresponding system, and the related computer program product. Another embodiment is described in each of the dependent claims.

[0006] According to one aspect of the present invention, there is provided a computer-implemented method for determining the refractive power of an intraocular lens to be inserted. In this case, the method particularly includes providing a physical model for determining the refractive power of the intraocular lens, and training a machine learning system to form a learning model for determining the refractive power based on clinical ophthalmic training data and related desired results. In this case, the loss function used for training can include two components. The first component of the loss function can take into account the corresponding items in the clinical ophthalmic training data and related desired results, and the second component of the loss function can take into account the limitation of the physical model that the greater the loss function component value of this second component, the greater the deviation of the predicted value of the refractive power during training from the result of the physical model when the same clinical ophthalmic training data is used as the input value.

[0007] The method can further include providing the ophthalmic data of the patient and predicting the refractive power of the intraocular lens to be inserted by the trained machine learning system, and the provided ophthalmic data can be used as input data for the machine learning system.

[0008] According to another aspect of the present invention, there is provided a system for determining the refractive power of an intraocular lens to be inserted. The system particularly includes a providing module in which a physical model for determining the refractive power of the intraocular lens is stored, and a training module adapted to train a machine learning system to form a learning model for determining the refractive power using clinical ophthalmic training data and related desired results. In this case, the parameter values of the learning model can be stored in the learning system. The loss function used for training can include two components. The first component of the loss function can take into account the corresponding items in the clinical ophthalmic training data and related desired results, and the second component of the loss function can take into account the limitation of the physical model that the greater the loss function component value of this second component, the greater the deviation of the predicted value of the refractive power during training from the result of the physical model when the same clinical ophthalmic training data is used as the input value.

[0009] Furthermore, the system can include a memory for the patient's ophthalmic data and a prediction unit adapted to predict the refractive power of the intraocular lens to be inserted, and the provided ophthalmic data is used as input data for the machine learning system.

[0010] Furthermore, embodiments can relate to a computer program product accessible from a medium that can be used by, or is readable by, a computer including program code used by, or in conjunction with, a computer or other instruction processing system. As used herein, a medium that can be used by, or is readable by, a computer can be any device suitable for storing, communicating, transmitting, or transporting program code.

[0011] Computer-implemented instructions for identifying the refractive power of the intraocular lens to be inserted have several advantages and technical effects that are also correspondingly applicable to the associated system. First, a machine learning system that identifies the refractive power of the ophthalmic lens to be inserted based only on available clinical ophthalmic data requires a relatively long training time, and second, it cannot neatly take into account the known characteristics of the physical model. Furthermore, if only clinical training data is used, a very large number of data points, i.e., training data, will be required to ensure a large amount of anatomical variability. Furthermore, the actual variability in purely clinical, i.e., ophthalmic, training data cannot be actively controlled. The entire parameter space can be systematically sampled by the physical model.

[0012] The method presented in this application, in contrast, uses the best from both domains, namely, first, not only the domain of physical and mathematical models, but second, also the domain of clinical ophthalmological data. Additionally, the machine learning model can be pre-trained prior to training with clinical ophthalmological data for this purpose. For this purpose, anatomically generated training data can be generated by the physical model. This physical model does not necessarily have to be the same as what affects the loss function. In this way, the influence of different physical models can be taken into account during training.

[0013] In this case, the method presented has a significant impact on the robustness of training, both for the case of an untrained and a pre-trained system. In the case of an untrained neural network, the physical constraints of the loss function ensure that the system cannot learn physically inconsistent predictions during training with real data. As a result, the influence of outliers in the dataset is avoided, and the trained network as a whole can make more stable predictions. If the machine learning model has already been pre-trained with artificial data using physical constraints and thus contains physical knowledge, the control over the loss function can have the effect that "catastrophic forgetting" cannot start, i.e., the knowledge learned in the past cannot be easily overwritten by training with ophthalmological data. Due to the physical constraints of the loss function, the network has to continue to take into account physical boundary conditions and limitations.

[0014] Due to this constraint and the additional physical information that can be made available during training, the data required for training can be significantly less overall, because it is not necessary to learn the physical boundary conditions from the data. This makes the application of the method much more flexible and faster, since it is not necessary to collect large amounts of data (i.e., clinical training data) in advance. Additionally, by training with dataset per clinic, it is possible to accurately coordinate the method and the corresponding system with these datasets in this way. This is possible because relatively few items of clinical data are sufficient for training.

[0015] The physical constraints of the loss function itself can represent the coverage range of the entire parameter space. This can generate an accurate physical solution for any recognizable data point, and thus can enable systematic representation of the entire parameter range. Compared with conventional methods, this is an important advantage for the training process, because the actual data that exists in normal situations can represent only a small part of the parameter space. The latter further always has the possibility of easily generating errors. All of these can be compensated by physical constraints. Therefore, it is a significant expansion and improvement of the training process.

[0016] Since the actual data also always has to be taken into account in addition to this direct training using physical models, the correct weighting of components can constitute another important aspect of the concept of the present application. What can be achieved by the implemented weighting is that the machine learning model can, firstly, take into account physical boundary conditions and, secondly, has sufficient degrees of freedom to adapt itself to the ideal data situation. This balanced interaction provides important advantages during the training process and makes it possible to improve the final prediction of the IOL refractive power for new ophthalmic data.

[0017] In addition, it will also be possible to take into account theoretical ophthalmic data. These data can also be composed from literature value data. Intermediate values can also be generated by interpolation between literature value data, i.e., data from other sources. These additional reference data thus obtained can complement or replace other mathematical physics models in addition to or instead of the physical model of the loss function.

[0018] There is a wide tolerance regarding taking into account the mathematical physics model during the training of a machine learning system trained with clinical ophthalmic data in this way. On the other hand, a large source pool of additional data that affects the loss function and is not used in the currently used physical models can also be obtained.

[0019] The proposed concept can also be extended to the use of not only one but also physical models that have an impact on the loss function. Instead, the impact on the loss function can also take into account at least one other physical model. In such a case, the loss function is supplemented with another term, which is included with an additional weighting factor. The rest of the function, especially the supply of input data, will be executed in parallel with and in accordance with the first physical model.

[0020] Overall, the advantages in terms of speed can be realized during training, which are obtained not only because this includes training with clinical ophthalmological data during training, but also because the outliers in the measurement of clinical ophthalmological data are directly corrected by the physical model. The training phase of the machine learning system can also proceed with less or not so much annotated data. Overall, the computing power can be significantly reduced, and thus the available computer capabilities can be better utilized.

[0021] Another exemplary embodiment is described below, which can be effective both with respect to the method and to the corresponding system.

[0022] In summary, different from known machine learning systems and corresponding methods for determining the refractive power of an intraocular lens (IOL) that operate based on actual data during training, i.e., ophthalmic data, thus requiring a large amount of training data and being unable to perform reliable predictions (i.e., determination of IOL refractive index) outside the parameter space covered by the training data set, the methods and systems presented in the present application can use the physical model and the boundary conditions of clinical ophthalmic training data equally, with corresponding weighting. Therefore, the concept presented in the present application is superior to conventional methods that typically do not cover the entire physical boundary conditions with the large number of clinical training data available. Furthermore, these are also prone to measurement errors, which further complicates the situation. Briefly speaking, (i) a more robust system for determining or predicting the refractive power of an IOL can be obtained that can cover the entire expected anatomical variability with a physical model. In addition, (ii) the combination of the physical model and clinical data means that less clinical data is required for a robust model. In this way, a robust model can be created for each clinic, each doctor, or each lens.

[0023] According to one advantageous embodiment of this method, the first and second components of the loss function can be weighted in a configurable manner. Therefore, fine-tuning of the training model of the machine learning system to be trained is possible. In this regard, for example, it can be set which of the two components of the loss function should be given a greater weight, i.e., (i) clinical ophthalmic training data or (ii) the constraints obtained from the physical model. In this way, the influencing parameters can be adjusted very individually and according to the type of physical model selected. In this way, weights of different strengths or other or additional restrictions ("constraints") can be defined depending on the type of physical model selected. Therefore, the synchronization introducing the weighting is directly recognizable, i.e., a balance is struck between the potential catastrophic interference arising from the physical model and the risk of overfitting based on clinical data.

[0024] According to a further advantageous embodiment of the method, the following types of weighting functions can be applied: W L = B * [a * (Delta)-(1 - a) * Phy], and the following applies: W L = value of the loss function B = general constant of the loss function or term of another function a = weighting constant Delta = first component, i.e., the result of the error function of the value of the error during training (e.g., MSE: mean squared error) Phy = second component, i.e., the constraints obtained from the physical model.

[0025] The values for weighting can be newly set from training to training (or retraining). For this purpose, an explicit user interface can be provided, thereby enabling training under optimal conditions. Thereby, it is also possible to conveniently test various physical boundary conditions, i.e., the physical model.

[0026] According to a supplementary advantageous embodiment of this method, the ophthalmic data can include OCT image data, i.e., complete "raw" image data or explicit ophthalmological numerical values obtained from the OCT image data, or both the OCT image data and the values obtained from the OCT image data. The image data can also be completely biometric data. In this way, high flexibility is obtained in the use of the training data used.

[0027] According to another developed embodiment of the method, the predicted position of the intraocular lens to be inserted can be used as an additional input data value for the machine learning system during the manufacturing operation. It can be expected that in this way, it will be possible to further improve the determination of the refractive power of the IOL.

[0028] According to an extended form from one embodiment of the method, the learning model of the machine learning system can be trained with data artificially generated based on the laws of the provided physical model before being trained with ophthalmic data. These laws can be represented by a physical model, i.e., an equation. In this case, it is also possible to make the physical model for the pre-training described in this application different from the physical model during the main training described above. In this way, at least two types of physical models can be considered, namely, (i) one physical model during the pre-training of the two-stage training thus performed for the machine learning system learning model, and (ii) the second physical model during the subsequent main training of the learning model of the machine learning system. Depending on the physically model thus selected, the above weighting of the loss function can be easily set by a dedicatedly adapted user interface. Considering two different physical models in this way means that it is not necessary to supplement the loss function with other terms. Furthermore, the training time and / or the amount of actual training data can therefore be reduced. The available resources are better utilized.

[0029] As a result, the learning model to be trained will thereby be advantageous in terms of time from pre-training using the physical model. In theory, much more detailed physical models can be used for training or for the generation of training data.

[0030] According to an extended form from one embodiment of the method, the physical model can also include literature value data for specifying the refractive power of the intraocular lens. The literature value data can exist in the form of a table, from which a tuple of values can be provided, for example, by interpolation of the existing values, as a supplement to or instead of the physical model. In this way, the physical model can be eliminated, and moreover, it is not necessary to exclude the influence of known limit values ("constraints").

[0031] According to a highly useful embodiment of the method, the intraocular lens to be inserted can be a spherical, toric, or multifocal intraocular lens to be inserted, or other lens shapes. The concept presented in this application is therefore widely applicable. Advantageously, the training data and the physical model (or multiple physical models) can also be appropriately selected.

[0032] According to another exemplary embodiment of the method, the machine learning system can be a neural network. In this case, a convolutional neural network (CNN) can be included. CNNs have been found to be particularly beneficial when the task is the processing of image data to be classified, as is the case with raw data of, for example, ophthalmic data.

[0033] Alternatively, data that is perhaps current time-dependent data (three spatial directions and the time variation of the eye scan data) from a 4D scan of the eye can be utilized. In this case, an RNN (recurrent neural network) can be utilized instead of or in addition to the above-mentioned CNN.

[0034] According to one advantageous exemplary embodiment of the method, the ophthalmic data of an eye can include axial length, anterior chamber depth, lens thickness, posterior chamber depth, corneal thickness, corneal curvature measurements, lens equator, WTW value, and pupil diameter. It should be understood that the numerical values of each of the above-mentioned parameters are also contemplated. Currently, these eye parameter values can be conveniently and accurately identified by an eye scan.

[0035] According to an extended exemplary embodiment of the method, the second physical model can be represented as a mathematical model or a ray tracing model. As a result, the choice of using different methods to make available improved training data based on the model is also possible in the second stage of training data generation. This can expand the scope for customizing the proposed method to a specific user.

[0036] According to another extended exemplary embodiment of the method, clinical ophthalmology training data can be identified or generated manually or by a third machine learning system. In this regard, manually means measuring by an eye scanning device. In contrast, the training data generated by the third geometric learning system tends to be more artificial, however, it is also possible to use a relatively small amount of clinical ophthalmology data to provide more training data for the final learning step by a third, already trained machine learning system. In this way, the method presented in the present application can also be used with a relatively small amount of clinical ophthalmology data, which would normally be insufficient to refine from a physical model to real clinical data in a two-step training step. For example, a GAN (Generative Adversarial Network) can be used for this purpose.

[0037] It should be pointed out that the exemplary embodiments of the present invention can be described with respect to various implementation categories. In particular, one exemplary embodiment is described with respect to a method, and other exemplary embodiments can be described with respect to the corresponding apparatus. Nevertheless, those skilled in the art will be able to identify the features of the method and their possible combinations, as well as the possible combinations of the features of the corresponding system, from the above and following descriptions, even if they belong to different categories of claims, and can combine them, especially if there is no explicit disclaimer to the contrary.

[0038] The aspects of the present invention already described above and additional aspects will become apparent, inter alia, from the exemplary embodiments described and from additional specific embodiments described with respect to the drawings.

[0039] Preferred exemplary embodiments of the present invention will be described by way of example and with reference to the following drawings.

Brief Description of the Drawings

[0040]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

DETAILED DESCRIPTION OF THE INVENTION

[0041] For the purposes of explanation, the conventions, terms, and / or expressions should be understood as follows.

[0042] The term "intraocular lens" refers to an artificial lens that can be surgically inserted into a patient's eye to replace the natural biological lens.

[0043] The term "loss function" refers to an error function that outputs a set of values or error values during the training of a machine learning system. Generally, the larger it is, the greater the deviation between the predicted value and the expected value of the machine learning system for the associated set of input values. There are many ways to identify and use this difference (e.g., MSE = mean squared error or cross-entropy). The output value of the loss function is fed into the neural network or the nodes of the weighting function (backpropagation). In this way, the actually predicted output value of the machine learning system converges in the direction of the annotated, i.e., desired result value.

[0044] The term "machine learning system" represents a system that is typically also assigned to a certain method, and the said system learns from examples. For this purpose, annotated training data (i.e., including metadata) is supplied to the machine learning system, and the pre-set output value - in the case of a classification system, the output class - is predicted. When the output class is output accurately with sufficient accuracy, i.e., at a pre-specified error rate, the machine learning system is called trained. Various machine learning systems are known. These include neural networks, convolutional neural networks (CNNs), or recurrent neural networks (RNNs).

[0045] In principle, the term "machine learning" is a basic term or basic function in the field of artificial intelligence, and it imparts the ability to "learn" to a computer system, for example, using statistical methods. For example, in this case, specific behavior patterns within a specific task range are optimized. The method used imparts the ability to analyze data to a trained machine learning system without requiring explicit procedural programming for this purpose. Typically, for example, an NN (neural network) or a CNN (convolutional neural network) is an example of a machine learning system that consists of a network of nodes that function as artificial neurons and artificial connections (so-called links) between the artificial neurons, in which case parameters (e.g., weighting parameters for the links) can be assigned to the artificial links. During the training of a neural network, the link weighting parameter values are automatically adjusted based on the input signal to generate the desired result. In the case of supervised learning, the images supplied as input values (training data) - generally data (between inputs) - are supplemented with the desired output data (annotations) to generate the desired output value (desired class). Generally speaking, the mapping of the input data to the output data is learned.

[0046] The term "neural network" refers to a network consisting of electronically implemented nodes having one or more inputs and one or more outputs for performing computational operations (activation functions). Here, the selected nodes are interconnected by connections - so-called links or edges. The connections can have specific attributes, such as weighted parameter values, thereby being able to affect the output values of the preceding nodes.

[0047] Neural networks typically consist of multiple layers. There are at least an input layer, a hidden layer, and an output layer. In a simple example, image data is supplied to the input layer, and the output layer can have a classification result regarding that image data. However, typical neural networks have a large number of hidden layers. The way the nodes are connected by the links depends on the type of each neural network. In this example, the predicted value of the neural learning system can be the required refractive power of the intraocular lens.

[0048] The term "recurrent neural network" refers to a neural network that is distinguished from a feedforward network by links from neurons (i.e., nodes) in one layer to the same or preceding layer neurons. This is a preferred way of interconnecting neural networks in the brain, particularly in the neocortex. In artificial neural networks, recurrent interconnections of model neurons are frequently used to discover time-coded - i.e., dynamic - information in the data. Examples of such recurrent neural networks include Elman networks, Jordan networks, Hopfield networks, and fully connected neural networks. These are also suitable for examining the dynamic behavior in eye recordings, particularly for considering the accommodation behavior of the eye.

[0049] The term "Convolutional Neural Network" (CNN) represents a class of artificial neural networks based on the feed-forward method, as an example of a classifier / classification system. These are often used in image analysis that typically uses images or their pixels as input data. The main component of a convolutional neural network in this case is the convolutional layer (hence the name), which enables efficient evaluation through parameter sharing. Unlike CNNs, each pixel of a recorded image is typically associated with an artificial neuron of a neural network as an input value of the conventional neural network.

[0050] The term "parameter value" represents a geometric or biometric value of a patient's eye, or ophthalmic data. Examples of eye parameter values are explained in more detail based on Figure 2.

[0051] The term "scan result" represents digital data based on, for example, a digital image / recording, which represents the result of an OCT (optical coherence tomography) examination of a patient's eye.

[0052] The term "optical coherence tomography" (abbreviated as OCT) represents a known ophthalmic imaging method for obtaining 2- and 3-dimensional recordings (2D or 3D) of scattering substances (e.g., biological tissues) with a resolution in micrometers. In this process, basic light sources, beam splitters, and sensors, such as sensors in the form of digital image sensors, are utilized. OCT is used to detect local differences in the reflection behavior of individual retinal layers and can represent morphological structures with high resolution.

[0053] An "A-scan" (also called an axial depth scan) represents a 1-dimensional result of a scan of a patient's eye, which provides information about the geometric dimensions and positions of structures within the eye.

[0054] The term "B-scan" refers to a plurality of the above-described A-scans stacked horizontally to obtain a cross-section of the eye. A volume view can also be generated by combining the plurality of layers of the eye thus generated.

[0055] The term "en face OCT" in this case refers to a method for generating a cross-sectional image of the eye in the horizontal direction, which is in contrast to the longitudinal cross-sectional image using the above-described A- or B-scan.

[0056] The term "dynamic eye data" refers to a sequence of 2D cross-sectional images of the eye - usually at the same position - for recognizing dynamic changes, i.e., changes over time, such as the accommodation ability of the eye.

[0057] The term "digital image" - for example, by scanning - in this case refers to the result of generating an image representation of a physically existing object, in this case the retina of the eye, or a certain amount of data in the form of pixel data therefrom. More generally, a "digital image" can be understood as a two-dimensional signal matrix. The individual vectors of the matrix can be combined with each other to generate an input vector for a certain layer of a CNN. A digital image can also be an individual frame of a video sequence.

[0058] The term "clinical ophthalmology training data" refers to data regarding patients' eyes and intraocular lenses inserted into these patients in the past. The clinical ophthalmology training data may include specified ophthalmological parameter values, which are also, for example, the refractive index and position of the inserted lens. These data are used for the purpose of training a machine learning system that has been trained up to that point based on data from a physical model. Basically, the clinical ophthalmology training data are annotated.

[0059] The term "training data" refers to data that can be used to train a machine learning system. These training data for a machine learning system are ophthalmological data and related refractive power values from past successful cases of cataract surgery.

[0060] The term "physical model" relates to mathematical formulas that correlate various parameters of the eye to determine refractive power. Known formulas include the Haigis formula and the Universal Barrett II formula. Additionally, ray tracing methods can also be used.

[0061] The term "refractive power of the intraocular lens" represents the refractive index of the IOL.

[0062] The figures will be described in detail below. In this case, it should be understood that all details and information in the figures are shown schematically. First, a block diagram of one exemplary embodiment of a computer-implemented method according to the present invention for determining the refractive power of an intraocular lens to be inserted is shown. Another exemplary embodiment, or an exemplary embodiment of the corresponding system, will be described later.

[0063] FIG. 1 shows a flowchart-like representation of an exemplary embodiment of a computer-implemented method 100 according to the present invention. The method 100 includes a step 102 of providing a physical model for determining the refractive power of an intraocular lens. This can include formulas for determining refractive power based on, for example, a set of input parameters stored in tabular form, data from any other trained machine learning system, or literature value data.

[0064] Furthermore, the method 100 includes a step 104 of training a machine learning system with clinical ophthalmology training data and related desired results to form a learning model for determining refractive power, where the training loss function includes two components. The desired result is the result of the machine learning system to be predicted when specific input parameter values exist. The combination of input data and expected result data is also referred to as "ground truth" in relation to machine learning. This particularly applies to so-called "supervised learning," which is carried out here.

[0065] In the first component of the loss function, corresponding items among the clinical ophthalmology training data and the associated desired results are considered. For this component of the loss function, a known mean squared error method can be used. In this case, the larger the component of the loss function (quadratically), the more the predicted value deviates from the associated result (prediction) value with annotation. The use of the square ensures that both numerically positive and numerically negative error values are equally considered.

[0066] In the second component of the loss function, it is considered that the larger the loss function component value of this second component, the more the predicted value of the refractive index by the machine learning system during training deviates from the result of the physical model when the same clinical ophthalmology training data is used as the input value for the physical model.

[0067] Method 100 further includes step 106 of providing the identified ophthalmic data of the patient and step 108 of predicting the refractive power of the intraocular lens to be inserted by a trained machine learning system, and the provided ophthalmic data is used as input data for the machine learning system.

[0068] Optionally (therefore shown using a dashed line), the position of the intraocular lens to be inserted can also be used as an additional input value for the machine learning system (see 110).

[0069] FIG. 2 shows the eye 200 together with various biometric or ophthalmic parameters of the eye. In particular, the following parameters are shown: axial length 202 (AL), anterior chamber depth 204 (ACD), corneal curvature measurement 206 (K, radius), refractive power (power) of the lens, lens thickness 208 (LT), central corneal thickness 210 (CCT), WTW value 212 (WTW), pupil diameter 214 (PS), posterior chamber depth 216 (PCD), corneal thickness 218 (RT). At least one of these parameters is included in both the ophthalmic training data and the patient's ophthalmic data, and each of them is included in the subject matter of the concepts presented in this application.

[0070] Figure 3 shows a schematic structure 300 of basic functional blocks that are beneficial for the implementation of the proposed method. First, a suitable physical model 302 of the eye for identifying refractive power is selected and provided. Second, training data 304 is made available by a machine learning system 310. These are first the so-called correct data, that is, refractive power values 308 and result values for the measured ophthalmic data 306 (with annotations). Alternatively, instead of the measured ophthalmic data, complete image data of the corresponding eye can also be used additionally or alternatively (e.g., A-scan, B-scan, etc.).

[0071] At the same time, the input values of the training data (measured ophthalmic data 306) are provided to a calculation module for result values for the physical model. This identifies the deviation from the physically correct solution of the output of the machine learning system 310 (explained in more detail in the next paragraph) in parallel with the desired or annotated IOL refractive power value, and the greater this is, the more the output from the machine learning system 310 returns a value that deviates more from this solution. Other data sources such as literature values can also be used instead of the calculated or otherwise identified output of the physical model 302.

[0072] The machine learning system 310 during training is shown as a deep neural network (DNN). This has an input layer of nodes (left) and an output layer of nodes (right). Only four and two nodes respectively are shown, but the number of input and output nodes will typically be much larger in the case of a neural network that can be used in practice. There are a plurality of further layers having nodes between the input layer and the output layer (typically more than two layers inside the DNN shown as an example), and these are selectively interconnected via their respective weight functions.

[0073] Training a machine learning system or its learning model involves repeatedly identifying parameters for nodes or corresponding weight functions for connections between nodes. The loss function 312 identifies which values are adapted by the weight function or the parameter values of the nodes during training. Put simply, training continues until the difference between the desired IOL refractive power and the IOL refractive power predicted by the machine learning system is lower than a predetermined minimum value.

[0074] However, a specific feature of the method proposed in the present application is that here, the value of the loss function 312 is not based only on the above-mentioned difference, but is determined by the result of a calculation module for the underlying physical model 302, having a second - typically additional, for example further linear - component. By weighting the components of the loss function 312, it is possible to perform fine - tuning skillfully and advantageously during the training of the machine learning system 310.

[0075] To ensure that the two components of the loss function are reliably available simultaneously, advantageously, a synchronization unit can be used, which controls the supply of further training data so that both components of the loss function are already available for the backpropagation cycle step and thus new training data is available only when the training step is completely finished.

[0076] When the training of the machine learning system 310 is finished, this system can be used productively. The machine learning system 314 is then trained and can receive the ophthalmic data 316 of a patient, and the refractive power 318 for the intraocular lens to be inserted can be predicted by the prediction unit 320 using its trained machine learning model. In this case, in addition to the desired position of the intraocular lens to be inserted, additional input parameter values for the trained machine learning system 314 can be used. Furthermore, instead of or in addition to the ophthalmic data 316, image data of the eye of a specific patient can be used as input values for the trained machine learning system 314.

[0077] Figure 4 shows a preferred exemplary embodiment of a component that aids in training a machine learning system of a proposed method 100 for determining refractive power, and that can also be used in the computational phase of the method, for a system 400 for determining refractive power - to be complete.

[0078] System 400 includes a processor 402, which can execute program modules or program code stored in memory 404. As a result, the processor affects the method elements to be able to execute the functions of the following components. In particular, for this purpose, system 400 includes a physical model providing module 406 for storage. In this case, for example, literature values for combinations of measured ophthalmic data and associated IOL refractive power values can also be stored, or the model can be stored in the form of physical equations with corresponding parameters. There can also be a calculation unit 408 for the physical model, which utilizes the memory of the physical model providing module 406.

[0079] Supplementary, there can also be a calculation unit 418 for a loss function that takes into account the aforementioned two components.

[0080] A training module 410, which is adapted to train a machine learning system to form a learning model for determining the refractive power of an IOL using clinical ophthalmic training data and associated desired results, uses the results of the loss function during training. In this case, the loss function includes: (i) a first component that takes into account corresponding items of the clinical ophthalmic training data and the associated desired results, and (ii) a second component that takes into account the limits of the physical model - or any other boundary conditions ("constraints") - such that the greater the value of the associated loss function component of this second component, the greater the deviation of the predicted refractive power value during training from the result of the physical model when the same clinical ophthalmic training data is used as the input value. For this purpose, in addition to a linear approach, for example, polynomial or exponential functions can also be used.

[0081] Via the memory 414, the patient's ophthalmic data is ultimately provided to the machine learning system 412 (which corresponds to the machine learning system 310 in FIG. 3). The prediction unit 416 (see 320 in FIG. 3) outputs prediction data identified by the machine learning system 412 regarding the refractive power of the intraocular lens to be inserted, and the provided ophthalmic data is used as input data for the machine learning system. The memory 414 may also be used for ophthalmic training data.

[0082] It should be clearly pointed out that the modules and units - in particular the processor 402, the memory 404, the physical model providing module 406 for storage, the physical model calculation unit 408, the loss function calculation unit 418, the training module 410, the machine learning system 412, the ophthalmic data memory 416, and the prediction unit 416 - can be connected for signal or data exchange via electrical signal lines or via the internal system bus system 420. In addition, the display unit can also be connected to the internal system bus system 420 or the prediction unit 416, thereby outputting, displaying, or otherwise further processing or transferring the refractive power.

[0083] When a classification system is used as the machine learning system, the predicted refractive power is obtained according to the predicted class predicted with the highest probability. Alternatively, the final refractive power of the IOL can also be implemented by a regression system as a machine learning system using a numerical output variable value.

[0084] FIG. 5 shows a block diagram of a computer system that can include at least a portion of a system for identifying refractive power. Embodiments of the concepts proposed in this application can in principle be used with virtually any type of computer, regardless of the platform used therein for storing and / or executing program code. FIG. 5 shows, by way of example, a computer system 500 that is suitable for executing program code according to the methods presented in this application and that can further include all or a portion of the prediction system.

[0085] Computer system 500 has a plurality of multi-purpose functional units. In this case, the computer system can be a tablet computer, a laptop / notebook computer, other portable or mobile electronic device, a microprocessor system, a microprocessor-based system, a smartphone, a computer system with specially configured special functional units, or a component of a microscope system. Computer system 500 can be configured to execute computer system executable instructions - such as program modules - executable to implement the functions of the concepts proposed in this application. For this purpose, the program modules include routines, programs, objects, components, logic, data structures, etc., and can implement specific tasks or specific abstract data types.

[0086] The components of the computer system can include: one or more processors or processing units 502, a storage system 504, and a bus system 506 that connects various system components including the storage system 504 to the processor 502. Computer system 500 typically includes a plurality of volatile or non-volatile storage media accessible by computer system 500. Storage system 504 can store data and / or instructions (commands) of a volatile form of storage media, such as RAM (random access memory) 508, so as to be executed by processor 502. These data and instructions realize one or more functions and / or steps of the concepts presented in this application. Another component of storage system 504 can be a permanent memory (ROM) 510 and a long-term memory 512, in which program modules and data (reference numeral 516) and workflows can also be stored.

[0087] The computer system includes a plurality of dedicated devices for communication (keyboard 518, mouse / pointing device (not shown), screen 520, etc.). These dedicated devices can also be combined within a touch-sensitive display. The separately provided I / O controller 514 ensures frictionless data exchange with external devices. A network adapter 522 is available for communication via a local or global network (LAN, WAN via the Internet). The network adapter can be accessed by other components of the computer system 500 via the bus system 506. In this case, although not shown, it should be understood that other devices can also be connected to the computer system 500.

[0088] Furthermore, at least a part of the system 400 (see FIG. 4) for specifying the refractive power of the IOL can be connected to the bus system 506.

[0089] The description of various exemplary embodiments of the present invention is provided for better understanding and does not directly limit the concept of the present invention to these exemplary embodiments. A person skilled in the art will be able to develop further improvements and modifications. The terms used in this application are selected to best explain the basic principles of the exemplary embodiments and to enable a person skilled in the art to easily reach them.

[0090] The principles presented in this application can be embodied as a system, as a method, as a combination thereof, and / or as a computer program product. The computer program product can in this case include one (or more) computer-readable storage media containing computer-readable program instructions for causing a processor or control system to execute various aspects of the present invention.

[0091] As a medium, electronics, magnetics, optics, electromagnetics, or infrared or infrared media or semiconductor systems are used as a transmission medium, which is, for example, SSDs (solid state devices / drives as solid state memory), RAM (random access memory) and / or ROM (read-only memory), EEPROM (electrically erasable ROM), or any combination thereof. Suitable transmission media also include electromagnetic waves that propagate, electromagnetic waves in waveguides, or other transmission media (for example, optical pulses in optical cables) or electrical signals transmitted within wires.

[0092] A computer-readable storage medium can be an embodiment device that holds or stores instructions used by an instruction execution device. The computer-readable program instructions described herein can also be downloaded to a corresponding computer system, for example, via a cable connection or a mobile wireless network from a service provider.

[0093] The computer-readable program instructions for performing the operations of the present invention described herein can be machine-dependent or machine-independent instructions, microcode, firmware, situation-specifying data, or any source code or object code, which can be written in, for example, C++, Java or others, or a conventional procedural programming language, for example, the programming language "C" or a similar programming language. The computer-readable program instructions can be executed entirely by a computer system. In some exemplary embodiments, it may be an electronic circuit such as a field programmable gate array (FPGAs) of a programmable logic circuit, or a programmable logic array (PLAs), etc., which uses the situation information of the computer-readable program instructions to execute the computer-readable program instructions to configure or individualize an electronic circuit according to an aspect of the present invention.

[0094] The invention presented in this application is further illustrated with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to exemplary embodiments of the invention. It should be noted that any block in a flowchart and / or block diagram can be implemented as computer-readable program instructions.

[0095] The computer-readable program instructions can be made available to a general-purpose computer, a special-purpose computer, or a data processing system that can be programmed in any other way, such that the instructions executed by a processor or other programmable data processing apparatus generate means for implementing the functions or processes shown in the flowchart and / or block diagram. These computer-readable program instructions can thus also be stored in a computer-readable storage medium.

[0096] In this sense, any block shown in a flowchart or block diagram can represent a module, segment, or portion of instructions that represent a plurality of executable instructions for performing a particular logical function. In some exemplary embodiments, the functions shown within individual blocks can be executed in a different order, optionally in parallel.

[0097] All of the illustrated structures, materials, sequences, and equivalents of the means and / or steps having related functions in the following claims are to be considered as applicable to all of the structures, materials, or sequences represented by the claims.

Description of the Reference Numerals

[0098] 100 Method for specifying refractive power 102 Method step of 100 104 Method step of 100 106 Method step of 100 108 Method step of 100 110 Optional method step of 100 Parameters of 200 eyes 202 Axial length 204 Anterior chamber depth 206 Corneal curvature measurement value 208 Thickness of the lens 210 Central corneal thickness 212 WTW value 214 Pupil diameter 216 Posterior chamber depth 218 Retinal thickness 300 Functional blocks for method execution 302 Physical model 304 Training data 306 Ophthalmic training input data 308 Annotated training result data 310 Machine learning system 312 Loss function 314 Trained machine learning system 316 Patient's ophthalmic data 318 Predicted refractive power of the IOL to be inserted 320 Prediction unit 400 System for predicting refractive power 402 Processor 404 Memory 406 Memory for physical model 408 Calculation unit for physical model 410 Training unit 412 Machine learning system 414 Memory for ophthalmic data 416 Prediction unit 418 Calculation unit for loss function 420 Bus system 500 Prediction system 500 Computer system 502 Processor 504 Memory system 506 Bus system 508 RAM 510 ROM 512 Long-term memory 514 I / O controller 516 Program module, potential data 518 Keyboard 520 Screen 522 Network adapter

Claims

1. In a computer-implemented method (100) for determining the refractive power of an intraocular lens to be inserted, - providing a physical model (102) for determining the refractive power of the intraocular lens; - training a machine learning system (310, 412) to form a learning model for determining the refractive power using measured clinical ophthalmic training data (304) and associated desired results (306), wherein the loss function (312) for the training includes two components, a first component of the loss function takes into account corresponding items among the measured clinical ophthalmic training data (304) and the associated desired results (306), a second component of the loss function (312) takes into account the limitation of the physical model that the greater the loss function component value of the second component, the greater the deviation of the predicted value of the refractive power during the training from the result of the physical model when the same clinical ophthalmic training data (304) is used as the input value (step 104); - providing measured ophthalmic data of a patient (106); - predicting (108) the refractive power of the intraocular lens to be inserted by the trained machine learning system, wherein the provided measured ophthalmic data is used as input data for the machine learning system; comprising, wherein the first and second components of the loss function (312) are weighted in a configurable manner, method (100).

2. The following type of weighting function: W Ｌ = B ＊ [a ＊ (Delta) - (1 - a) ＊ Phy], provided that W Ｌ = the value of the loss function B = a general constant of the loss function (312) or a term of another function a = a weighting constant Delta = the first component, Phy = the second component is applied, the method (100) according to claim 1.

3. - The measured ophthalmic data is OCT image data, or - The measured ophthalmic data is an explicit numerical value obtained from OCT image data, or - The measured ophthalmic data includes both OCT image data and numerical values obtained from OCT image data, The method (100) according to any one of claims 1 to 2.

4. The predicted position of the intraocular lens to be inserted is used as additional input data for the machine learning system (310, 412), the method (100) according to any one of claims 1 to 3.

5. The learning model of the machine learning system (310, 412) has already been trained by training data artificially generated based on the laws of the provided physical model before the training on the measured ophthalmic data, the method (100) according to any one of claims 1 to 4.

6. The physical model also includes literature value data for specifying the refractive power of the intraocular lens, the method (100) according to any one of claims 1 to 5.

7. The intraocular lens to be inserted is a spherical, toric, or multifocal intraocular lens to be inserted, the method (100) according to any one of claims 1 to 6.

8. In a system (400) for specifying the refractive power of an intraocular lens to be inserted, - A providing module (406) in which a physical model for specifying the refractive power of the intraocular lens is stored therein, - A training module (410) adapted to train a machine learning system (310, 412) to form a learning model for identifying the refractive power using the measured clinical ophthalmic training data (304) and associated desired results (306), wherein parameter values of the learning model are stored in the learning system (310, 412), and the loss function for the training includes two components, A first component of the loss function (312) takes into account corresponding items of the measured clinical ophthalmic training data (304) and associated desired results (306), A second component of the loss function (312) takes into account the limitation of the physical model that the predicted value of the refractive power during the training deviates more from the result of the physical model as the loss function component value of this second component increases when the same measured clinical ophthalmic training data is used as an input value, the training module (410); - A memory (414) for the measured ophthalmic data (316) of a patient, - A prediction unit (320, 416) adapted to predict the refractive power of the intraocular lens to be inserted by the trained machine learning system (314), wherein the stored measured ophthalmic data is used as input data for the trained machine learning system (314, 412), the prediction unit (320, 416); comprising The system (400), wherein the first and second components of the loss function (312) are weighted in a configurable manner. **Claim 9** A computer program product for identifying the refractive power of an intraocular lens to be inserted, comprising a computer-readable storage medium having program instructions stored thereon, the program instructions being executable by one or more computers (500) or a control unit, and causing the one or more computers (500) or the control unit to execute the method according to one of claims 1 to 7.

Images