COMPUTER IMPLEMENTED METHOD, PROCESSING PIPELINE SYSTEM AND COMPUTER PROGRAM PRODUCT FOR A MACHINE LEARNING ASSISTANT PIPELINE FOR SIZING INTRAOcular LENSES - Patent application
The integration of machine learning with physical models in a processing pipeline for IOL determination addresses anatomical inaccuracies in current methods, providing efficient and accurate prediction of IOL position and power.
Patent Information
- Application Number
- JP2022544699
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2020-01-24
- Filing Date
- 2021-01-21
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2041-01-21
AI Technical Summary
Current IOL calculation formulas lack anatomical accuracy and rely on predefined models, limiting their effectiveness in predicting refractive power and position, and machine learning approaches without physical concepts are data-dependent and inefficient.
A computer-implemented method using a machine learning-assisted processing pipeline that integrates ocular biometric data and physical models to determine the final position and refractive power of intraocular lenses, employing trained machine learning systems to optimize predictions based on clinical data and physical models.
This approach allows for anatomically accurate and efficient prediction of IOL position and power, reducing training time and computational resources while ensuring universal applicability and accuracy.
Smart Images

Figure 0007579864000001 
Figure 0007579864000002 
Figure 0007579864000003
Abstract
Description
[Technical field]
[0001] The present invention relates to determining the refractive power of an intraocular lens, and in particular to a computer-implemented method for a machine learning assisted processing pipeline for determining parameter values for an inserted intraocular lens, a corresponding system, and a corresponding computer program product for performing the method. [Background technology]
[0002] Replacing the biological lens of the eye with an artificial intraocular lens (IOL), for example in the case of (age-related) refractive errors or in the case of cataracts, has become increasingly common in recent years in the field of ophthalmology. In the process, the biological lens is detached and removed from the lens capsule by a minimally invasive intervention. In the case of cataracts, the clouded lens is then replaced by an artificial lens implant. In the process, this artificial lens implant or intraocular lens is inserted into the lens capsule in its time and space. Knowing the exact position of the intraocular lens and the required refractive power are mutually dependent.
[0003] There are several problems with currently available IOL calculation formulas. First, the position of the intraocular lens is calculated in many formulas as the effective lens position (ELP). This variable is not a real anatomical variable and therefore cannot be directly considered in the physical models used to calculate the complex ophthalmic optics of the patient. The ELP is calculated and optimized for each formula, so there is no direct comparability between the ELPs of different formulas and the models do not use anatomically accurate optical systems.
[0004] The second aspect consists of the fact that current IOL formulas use models in their predictions, which try to fine-tune to the availability of data by means of several parameters. These are predefined manually by the developer, which is not necessarily the best representation in each case. New formulas, such as the Hill RBF formula, avoid this limitation by using machine learning approaches that can perform optimization on their own, based on the availability of data. However, in this case, the predictions are based only on a large amount of data, i.e. the system does not use physical concepts and is therefore limited in terms of its validity.
[0005] In general, current approaches do not represent an optimal combination of all available information and models that exist. Summary of the Invention [Problem to be solved by the invention]
[0006] Proceeding from the shortcomings of known methods for approximately determining the correct refractive power for an implanted IOL, the objective underlying the concepts presented herein is to identify methods and systems for improved, integrated and fast IOL refractive power prediction for intraocular lenses that are sophisticated and scalable. [Means for solving the problem]
[0007] This object is achieved by the proposed method, a corresponding system and an associated computer program product according to the independent claims. Further embodiments are described by the respective dependent claims.
[0008] According to one aspect of the present invention, a computer-implemented method for a machine learning assisted processing pipeline for determining parameter values for an intraocular lens to be inserted is presented. The method may include providing an eye scan. The scan may represent an image of the ocular anatomy.
[0009] The method may further include determining ocular biometric data from the eye scan and using a first trained machine learning system to determine a final position of the intraocular lens to be inserted, where the ophthalmic data may serve as input data for the first machine learning system. Finally, the method may include determining a first refractive power of the intraocular lens to be inserted, where the determining is based on a physics model in which the determined final position of the intraocular lens and the determined biometric data are used as input variables of the physics model.
[0010] According to another aspect of the present invention, a processing pipeline system for a machine learning assisted processing pipeline for determining parameter values for an intraocular lens to be inserted is presented.
[0011] The processing pipeline system may include a receiving module configured to provide an eye scan, where the scan may represent an image of the eye anatomy.
[0012] The processing pipeline system may further comprise a determining unit configured to determine ocular biometric data from the eye scan and a first trained machine learning system for determining a final position of the inserted intraocular lens. The ophthalmic data may serve as input data for the first machine learning system.
[0013] Finally, the processing pipeline system may comprise a determination unit configured to determine a first refractive power of the intraocular lens to be inserted, the determination being based on a physics model in which the determined final position of the intraocular lens and the determined biometric data are used as input variables of the physics model.
[0014] Furthermore, embodiments may relate to a computer program product accessible from a computer usable or computer readable medium comprising program code for use by or in connection with a computer or other instruction processing system. In the context of this description, a computer usable or computer readable medium may be any device suitable for storing, communicating, transferring or transporting program code.
[0015] The computer-implemented method for determining the refractive power for an intraocular lens to be inserted has several advantages and technical effects that may also be applied to related systems accordingly, and the method presented here precisely addresses the known negative characteristics of the drawbacks already described above. In particular, the "ZAI" algorithm on which the method is based facilitates the optimization calculation of the required refractive power of an intraocular lens to be inserted during cataract surgery. The presented algorithm allows the integration of anatomically correct prediction of the IOL position optimized by machine learning with complex physical models, allowing the refinement of the IOL calculation by machine learning. Thus, both the IOL position and the IOL refractive power determination can be determined in one process, or as otherwise expressed, in the pipeline, without media destruction.
[0016] In this case, it is possible to link both physical computational models and machine learning concepts based on clinical ophthalmic data in a pipeline for integrated determination of intraocular lens position and refractive power determination.
[0017] Machine learning systems for determining the refractive power for an implanted intraocular lens, based solely on available clinical ophthalmic data, firstly, require relatively long training times and, secondly, cannot take into account known properties of the physical model.
[0018] In this case, the speed advantages that arise when an already trained machine learning model is retrained with better or further training data are exploited in both cases, which can significantly reduce the overall training time and allows for significant savings in computing power, thus allowing for better use of available computer capacity.
[0019] Furthermore, the use of the actual physical position of the IOL allows the use of models with any desired accuracy, and ultimately the use of accurate physical models. Thus, the presented method is not limited to a model of a certain size, and the final determined value is ultimately universal. This is in contrast to the formula previously used for the effective lens position (ELP), since this variable is not a real anatomical variable. Therefore, it also cannot be directly taken into account in the physical model used to calculate the complex ophthalmic optics of the patient.
[0020] Intraocular lenses are calculated with many formulas as Effective Lens Position (ELP). This variable is not a real anatomical variable and therefore cannot be directly taken into account in the physical models used to calculate the complex ophthalmic optics of the patient. The ELP is calculated and optimized for each formula, so there is no direct comparability between the ELPs of the different formulas and the models do not use anatomically accurate optical systems.
[0021] Further exemplary embodiments are presented below, which may have utility both in connection with the methods and in connection with corresponding systems.
[0022] According to an advantageous exemplary embodiment, the method may additionally comprise a determination of the final refractive power of the intraocular lens by a second machine learning system, which may use as input variables at least one variable from the biometric data and the first refractive power. By way of example, the at least one variable may be the axial length of the eye. It is thus possible to actually carry out a transfer learning step, using knowledge present in the physical model as a basis to facilitate a more accurate determination of the refractive power. To this end, the second machine learning system must be trained with clinical ophthalmic data, i.e. data from previous real patients. Such clinical ophthalmic data is usually annotated. In this way, no information is lost in the pipeline, but both theoretical data of the physical model and real empirical data from clinical routine can be taken into account.
[0023] In this way it is also possible to include in the pipeline characteristic features of a particular clinic or its operating method. Generally, the use of physical models does not allow this, or allows this only with the drawback of deviating from known standards.
[0024] According to a further exemplary embodiment of the method, the ocular biometric data may comprise at least one selected from the group consisting of pre-operative axial length, pre-operative lens thickness, pre-operative anterior chamber depth and intra-operative anterior chamber depth. These may be derived from the method step "Determining ocular biometric data from an eye scan". This may be performed in the traditional sense, but machine learning systems may also already be used to achieve this purpose, said machine learning systems determining the ocular biometric data in a scan-direct manner that does not require manual steps. The recorded image data of the scan may be used directly to determine the biological parameters.
[0025] According to an advantageous exemplary embodiment of the method, a convolutional neural network, a graph attention network, or a combination of the two said networks can be used in the first machine learning system. As an example, a convolutional neural network can be used to identify characteristic features in the recorded scan results and compress the generated image data. As a result of the graph attention network, it is possible to place the known annotated images or their compressed representations in a graph. With the newly recorded current image of the patient's eye, it is possible to determine the required biometric data, for example the postoperative final position of an intraocular lens, by distance measurement with respect to the images already present in the graph. This can then be used directly in the ZAI pipeline.
[0026] According to an advanced exemplary embodiment of the method, the second machine learning system can be trained in two stages, the first training step can include the generation, particularly by a computer, of first training data for the machine learning system based on a first physical model for refractive power for intraocular lenses. The second machine learning system can then be trained by the generated first training data for the purpose of forming a corresponding learning model for determining refractive power. In this case, the hyperparameters of the machine learning system are defined by the design and selection of the machine learning system, while the internal parameters of the machine learning system are piecewise adapted by training.
[0027] In a second training step, the machine learning system trained by the first training data can be trained with clinical ophthalmology training data for the purpose of forming a second learning model for determining refractive power.In this case, the transfer learning principle is used, i.e. the knowledge already learned from the physical model is now further specified by using actual clinical ophthalmology training data.In this way, the training process can be significantly accelerated, and less clinical ophthalmology training data is required, since the basic structure is already pre-adjusted by training with data from the physical model.
[0028] According to an extended exemplary embodiment of the method, one variable from the biometric data can be the preoperative axial length, which can be accurately determined using known measurement methods (e.g., by OCT measurement, e.g., A-scan, B-scan, or en face OCT measurement).
[0029] According to an extended exemplary embodiment of the method, the eye biometric data can be determined manually from the image or by a machine learning system from a provided scan of the eye. At this point, the proposed method leaves open which partial method is used to determine the eye biometric data. However, the machine learning based determination of the biometric data is done within the meaning of the pipeline concept.
[0030] According to further extended exemplary embodiments of the method, further parameters of the eye can be determined when determining the final position of the inserted intraocular lens. These may relate to the following: The IOL position, in particular the expected final position of the IOL after the growth process, can be specified as a general further parameter. Moreover, it is also possible to use a value of the IOL shift, which indicates a deviation perpendicular to the optical axis. The beam path in the respectively selected model changes depending on the shift value.
[0031] Additionally or complementary, it is also possible to use the IOL tilt value (i.e. the tilt angle of the IOL with respect to the optical axis), in which case the beam path should also be adjusted as a result of the change. The type of IOL, especially the haptics used, shape, etc. are also considered, which may determine the position of the lens by the haptics / shape and thus affect the final quality of the surgery (correct IOL insertion).
[0032] In addition, the forces exerted on the IOL by the capsular bag etc. should be specified as additional parameters, allowing for possible changes in expected position over time to be taken into account.
[0033] It should be pointed out that the exemplary embodiments of the present invention may be described with reference to different implementation categories. In particular, some exemplary embodiments may be described with reference to a method, whereas other exemplary embodiments may be described in the context of a corresponding device. Regardless, a person skilled in the art can identify and combine possible combinations of features of the method and also possible combinations of features with the corresponding system from the above and below description, even if they belong to different claim categories, unless otherwise stated.
[0034] The above-mentioned aspects and further aspects of the invention will become apparent, inter alia, from the exemplary embodiments described and from further particular embodiments described with reference to the drawings.
[0035] Preferred exemplary embodiments of the invention will now be described, by way of example only, with reference to the following drawings, in which: [Brief description of the drawings]
[0036] [Figure 1] 1 illustrates a flow diagrammatic representation of an exemplary embodiment of a computer-implemented method for a machine learning assisted processing pipeline for determining parameter values for an inserted intraocular lens. [Diagram 2] 1 shows a cross section of a portion of the eye. [Diagram 3] 1 shows an eye with different biometric parameters of the eye. [Figure 4] 1 depicts a schematic structure of essential functional blocks of a machine learning assisted pipeline for sizing an intraocular lens according to a specified method. [Diagram 5] FIG. 1 shows a diagram of a processing pipeline system according to the present invention for a machine learning assisted processing pipeline for determining parameter values for an intraocular lens to be inserted. [Figure 6] 6 shows a diagram of a computer system which may additionally be equipped, completely or partly, with a processing pipeline system according to FIG. 5; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0037] In the context of this description, the conventions, terms and / or expressions should be understood as follows:
[0038] The term "machine learning assisted processing pipeline" describes the overall concept of the method and system provided herein. Proceeding from the recorded digital images, it is possible to determine the final refractive power of the intraocular lens to be inserted without destroying the medium and without manual determination of the intervening parameters. In this case, the final postoperative IOL position is used as an intermediate result, even without manual determination of the necessary parameters. At different points, the processing pipeline uses machine learning systems trained with real patient data. In addition, physical models can be used. In this way, both the know-how of theoretical models and the practical experience are included in the final determination of the refractive power.
[0039] The term "intraocular lens" describes an artificial lens that can be surgically inserted into a patient's eye to replace the natural lens.
[0040] The term "machine learning system" describes a system, which is also usually assigned to a method, that learns from examples. To this end, annotated training data (i.e. training data also including metadata) is fed to the machine learning system in order to predict a pre-defined output value, an output class in the case of a classification system. If the output class is correctly output with sufficient accuracy, i.e. a pre-determined error rate, the machine learning system is said to be trained. Various machine learning systems are known. These include neural networks, convolutional neural networks (CNN) or else recurrent neural networks (RNN).
[0041] In principle, the term "machine learning" is a basic term or basic function from the field of artificial intelligence, where for example statistical methods are used to give computer systems the ability to "learn". As an example, certain behavioral patterns within a certain task range are optimized in this case. The methods used give the trained machine learning system the ability to analyze data without the need for explicit procedural programming for this purpose. In general, NN (Neural Network) or CNN (Convolutional Neural Network) is an example of a system for machine learning, for example a network of nodes acting as artificial neurons and for forming artificial connections (so-called links) between the artificial neurons, where parameters (for example weighting parameters for the links) can be assigned to the artificial links. When training a neural network, the weighting parameter values are automatically adjusted to the links based on the input signals so as to generate the desired results. In the case of supervised learning, images provided as input values (training data), generally (input) data, are supplemented by desired output data (annotations) in order to generate the desired output values (desired classes). Considered quite generally, a mapping of input data to output data is learned.
[0042] The term "neural network" describes a network made up of electronically implemented nodes with one or more inputs and one or more outputs for performing computational operations, where selected nodes are interconnected by connections, so-called links or edges, which may have certain attributes, e.g. weighting parameter values, that can affect the output values of preceding nodes.
[0043] Neural networks are usually composed of several layers. There are at least an input layer, a hidden layer, and an output layer. In a simple embodiment, image data can be fed to the input layer and the output layer can have a classification result on the image data. However, a typical neural network has many hidden layers. The way in which the nodes are connected by links depends on the type of the respective neural network. In this embodiment, the predicted value of the neural learning system can be the desired refractive power of the intraocular lens.
[0044] The term "convolutional neural network" (CNN) describes a class of artificial neural networks based on feed-forward techniques, as an example of a classifier / classifier system. They are often used in image analysis using images or their pixels as input data. The main component of a convolutional neural network is the convolution layer (hence the name), which in this case allows efficient evaluation through parameter sharing. In contrast to a CNN, each pixel of a recorded image is generally associated with an artificial neuron of the neural network as an input value in a conventional neural network.
[0045] The term "Graph Attention Network" (GAT) describes a neural network that operates on graph-structured data. It behaves better than the older "Graphical Convolutional Network" (GCN). In the process, it uses a masked self-referencing layer of nodes that improves on the known approximations in GCN without being based on computationally intensive matrix operations. "GCN" (Graphical Convolutional Network) can be considered as an alternative to GAT, which is a certain architecture of neural networks that can also operate directly on graphs and use the structural information present there. As an alternative, the "GraphSage" framework could be used, which is well suited for inductive representation learning in the context of large graphs. In this case, GraphSage can be used to generate low-dimensional vector representations for the nodes, which is particularly useful for diagrams with comprehensive node attribute information.
[0046] In the context of this text, the term "transfer learning" (or curriculum learning) describes that a learning model once developed, developed by training a machine learning system with the training data of a physical model, is trained again. This second time it is trained with relevant data, which come from a different source than in the first training. These may consist either of clinical ophthalmological data or of a second physical model known to provide a higher accuracy of the results obtained. As a result, a second learning model is generated that unifies in itself both the physical model parameters and the actual clinical data. The "knowledge" of the respective first learning model is thus used as a basis or starting point for training to generate the second learning model. The learning effects of the first training can thus be transferred to the learning effects of the second training. A substantial advantage lies in the fact that the second training can be performed relatively more effectively, which in turn can save computer resources, which in turn can be performed in a faster and more targeted manner.
[0047] The term "parameter values" describes geometric or biological values or ophthalmological data of a patient's eye. Examples of ocular parameter values are explained in more detail with reference to FIG.
[0048] The term "scan result" describes digital data based on, for example, a digital image / recording representing the results of an OCT (Optical Coherence Tomography) examination of a patient's eye.
[0049] The term "optical coherence tomography" (abbreviated OCT) describes a known imaging method in ophthalmology for obtaining two-dimensional and three-dimensional recordings (2-D or 3-D) of scattering materials (e.g. biological tissues) with micrometer resolution. In the process, a light source, a beam splitter and a sensor, for example in the form of a digital image sensor, are basically used. In ophthalmology, OCT is used to detect spatial differences in the reflective behavior of individual retinal layers and can reveal morphological structures with high resolution.
[0050] The term "A-scan" (also referred to as axial depth scan) describes the one-dimensional result of a scan of a patient's eye that reveals information about the size, shape and location of structures within the eye.
[0051] The term "B-scan" describes the lateral overlay of multiple aforementioned A-scans to obtain a cross-section through the eye. Volumetric views can also be generated by combining multiple layers of the eye thus generated.
[0052] The term "en face OCT" in this case describes a method for generating transverse cross-sectional images of the eye, as opposed to longitudinal cross-sectional images using the A-scans or B-scans discussed above.
[0053] The term "image" or "digital image", e.g. from a scan, describes the result of generating an image representation of a physically existing object, in this case the retina of the eye, or a quantity of data in the form of pixel data therefrom. More generally, a "digital image" can be understood to be a two-dimensional signal matrix. The individual vectors of the matrix can then be adjacent to each other to generate input vectors for a layer of a CNN. The digital image can be an individual frame of a video sequence. Image and digital image can be understood to be synonymous in this case.
[0054] The term "clinical ophthalmic training data" describes data relating to patients' eyes and intraocular lenses already previously inserted in these patients. The clinical ophthalmic training data may also include determined ophthalmic parameter values such as the refractive power and the position of the inserted lenses. These data are used for the purpose of training a machine learning system that has already been previously trained on data from a physical model. In principle, the clinical ophthalmic training data is annotated.
[0055] The term "physical model" relates to a mathematical formula that relates various parameters of the eye to each other in order to make a determination of refractive power. A well-known formula is the Haigis formula.
[0056] The term "intraocular lens power" describes the refractive index of the IOL.
[0057] A detailed description of the drawings is given below, where it is understood that all the details and information in the drawings are shown in schematic form. First, a block diagram of an exemplary embodiment of a computer-implemented method according to the present invention for a machine learning assisted processing pipeline for determining parameter values of an intraocular lens to be inserted is shown. Further exemplary embodiments, or exemplary embodiments for a corresponding system, are described below.
[0058] 1 shows a flow diagrammatic representation of an exemplary embodiment of a computer-implemented method 100 according to the present invention for a machine learning assisted processing pipeline for determining parameter values for an intraocular lens to be inserted, in particular an intraocular lens to be inserted into a patient's eye. In this case, the method 100 includes providing 102 an eye scan, the scan representing an image of the ocular anatomy. This can be performed by OCT. An alternative, albeit less accurate, method is based on ultrasound.
[0059] The method 100 further comprises determining 104 eye biometric data from the eye scan, either conventionally or already with the aid of a machine learning system, and using the first trained machine learning system 106 to determine the final position of the intraocular lens to be inserted in the eye. In this case, the long-term post-operative position of the IOL is understood to mean the final position. The determination based on the trained machine learning system may determine the long-term post-operative position directly from one (or more) recorded images of the patient's eye, and manual intermediate steps can be omitted in the process. Alternatively, ophthalmic data, in particular from the preceding steps or determined by "scan direct", can serve as input data for the first trained machine learning system.
[0060] Finally, the method 100 includes determining 108 a first refractive power of the intraocular lens to be inserted, the determining being based on a physics model in which the determined final position of the intraocular lens and the determined biometric data are used as input variables of the physics model, in this case a mathematical deterministic model.
[0061] Optionally, the final refractive power determination 110 can be refined or improved by a second machine learning system, where the first refractive power and at least one variable of the biometric data, e.g., the axial length, are used as input data for the second trained machine learning system.
[0062] FIG. 2 shows a symbolic representation of a cross section of an eye 200. It is possible to identify an inserted intraocular lens 202, which has been surgically inserted into a capsular bag 204 after removal of the crystalline lens. Lateral structures 206 on the intraocular lens 202 should ensure that the intraocular lens 202 is truly and stably fixed in the capsular bag 204. However, the exact position of the intraocular lens 202, which is set after a relatively long growth phase, for example several weeks, could not be practically predicted until now. This is due in particular to the fact that the capsular bag 204, which previously surrounded the entire natural crystalline lens but has now been removed, is substantially larger than the inserted intraocular lens 202. These ligaments and muscle tissues 208 that fix the capsular bag 204 in the eye or on the skull, change after such surgery, as a consequence of which the size, shape and position of the capsular bag 204 and therefore the position of the inserted intraocular lens 202 also change. Thus, the distance between the inserted intraocular lens 202 and the retina, which is located further back in the eye, also changes. However, optimal post-operative results can only be achieved by optimal matching of the refractive power (index of refraction) and distance to the retina of the inserted intraocular lens 202. Since the refractive power of the inserted intraocular lens 202 is typically not subsequently alterable, prediction of the position of the inserted intraocular lens 202 is highly desirable.
[0063] 3 depicts an eye 300 with different biometric parameters of the eye. In particular, the following parameters are represented: axial length 302 (AL), anterior chamber depth 304 (ACD), keratometric curvature 306 (K, radius), lens power, lens thickness 308 (LT), central corneal thickness 310 (CCT), corneal width 312 (WTW), pupil diameter 314 (PS), posterior chamber depth 316 (PCD), retinal thickness 318 (RT). At least one of these parameters is included in both ophthalmic training data and patient ophthalmic data, each of which is within the subject of the concepts presented herein.
[0064] In other words, a machine learning system model incorporating known physical prior knowledge is first created using a physical model. This can be implemented, for example, by a machine learning system that is pre-trained using simulated data, or by the training itself possibly including physical constraints (constraint-based training). The learning model is then adapted to true anatomical variations using real clinical ophthalmology data. In this case, the selected approach facilitates self-learning optimization of the entire machine learning system for any availability of data (e.g., after LASIK surgery). In this case, the adaptation can be done explicitly for each doctor or each clinic. The real biometric data is then used as input values for the machine learning system in the application phase of said machine learning system to determine or predict the optimized intraocular lens power accordingly.
[0065] The formulation of the physical model is converted into a purely parameterized form of the neural network. The latter can then, independently and to the best possible extent, adapt itself to the actual data structure in a second training phase. Any amount of training data can thus be generated with the optical physical model. These include the parameters of the eye model and the associated IOL refractive power as the so-called ground truth. Using the "transfer learning" concept, the model thus trained can be transferred to a more complex physical model that generates training data according to the same concept. The neural network thus already has pre-trained artificial neurons and can therefore adapt itself more quickly and more easily to a stronger or better physical model. This curriculum learning can be performed for models of any strength (e.g. ray tracing models).
[0066] In the last step, the learning model is "fine-tuned" by the real biometric data of the patient's eye, using the actually used IOL power as ground truth. The trained model can then implement the prediction of the final required IOL power in the prediction phase. What has been found in practice is that the more real data (clinical ophthalmic data) is available, the better the machine learning system can be optimized on said data. Thus, the learning model can be continuously developed according to the availability of data, so that it can be adapted to different real data records.
[0067] In principle, the pipeline uses a machine learning model to use input data from OCT measurements of the patient's eye for an optimized prediction of the anatomically correct position of the intraocular lens. This position is then used in a physical model, which can be any realistic model (e.g., a regular mathematical physical model or ray tracing) for the known position of the intraocular lens. The physical model calculates the IOL power required for the eye, and the results are then further refined using machine learning to correct relatively small model errors in the physical model. To optimize the information usage, both IOL power ground truth data and IOL position ground truth data are used for training.
[0068] In this regard, Fig. 4 shows a schematic structure of the essential functional blocks 400 of a machine learning assisted pipeline for sizing an intraocular lens by the aforementioned method, which includes scan results / images 402 of an eye scan. These results, in particular in the form of at least one digital image, can be used for the conventional extraction of biometric data 404. At least a part of these biometric data and the scan results themselves are fed as input data to a graph-based neural network 406 in order to directly determine therefrom the final post-operative IOL position 408.
[0069] Then there is a manual formula-based determination 410 of the refractive power based on a mathematical physical model. Both the extracted biometric data (or part thereof) and the final IOL position 408 serve as input values for this determination 410 of the refractive power. In addition, a further machine learning system 412 can be used for an optimized determination of the refractive power, which uses both the initially determined refractive power of the intraocular lens (as a result of the refractive power determination 410) and the previously determined biometric data 404 (or part thereof) as input data. The trained machine learning system 412 then provides a final final refractive power 414 based on a suitable machine learning model.
[0070] Since both the final post-operative IOL position 408 and the final optimized IOL power can be determined by an integrated process with meshed partial processes, the term pipeline is directly and clearly disclosed by the representation of the individual steps or functional units identified in FIG. 4.
[0071] For the sake of completeness, Figure 5 illustrates a preferred exemplary embodiment of components of a processing pipeline system 500 for a machine learning assisted processing pipeline for determining parameter values for an implanted intraocular lens. The processing pipeline system 500 comprises a receiving module 502 configured to provide an eye scan, the scan representing at least one image of an ocular anatomical structure.
[0072] Further, the pipeline processing system 500 comprises a determination unit 504 configured to determine eye biometric data from the eye scan results and a first trained machine learning system 506 (see also the graph-based neural network 406 in FIG. 4 ) for determining a final position of the intraocular lens to be inserted, the ophthalmic data serving as input data for the first machine learning system.
[0073] Furthermore, the processing pipeline system 500 comprises a determination unit 508 (see also functional block 410 in FIG. 4 ) configured to determine a first refractive power of the intraocular lens to be inserted, the determination being based on a physics model in which the determined final position of the intraocular lens and the determined biometric data are used as input variables for the physics model.
[0074] Additionally, a further machine learning system 510 can also be used for improved prediction of IOL power (see function block 412 in FIG. 4).
[0075] Explicit reference is made to the fact that the modules and units, in particular the receiving module 502, the determining unit 504, the first trained machine learning system 506, and the determining unit for determining the first refractive power 508, may be connected by electrical signal lines or via a system internal bus system 512 in order to transfer appropriate signals and / or data from one module (one unit) to another. Furthermore, additional modules or functional units may be optionally connected to the system internal bus system 512.
[0076] If the classification system is used as a machine learning system, the predicted power occurs according to the predicted class that is predicted with the greatest probability. Alternatively, the final power of the IOL can be implemented by a regression system as a machine learning system with a numerical output variable.
[0077] Furthermore, the system 500 may comprise an output unit (not shown here) suitable for outputting or displaying the predicted final IOL power and, optionally, also the predicted IOL position.
[0078] Fig. 6 shows a block diagram of a computer system that may have at least a part of a system for determining refractive power. The embodiments of the proposed concepts may in principle be used with virtually any type of computer, regardless of the platform used therein to store and / or execute the program code. Fig. 6 shows as an example a computer system 600 suitable for executing program code according to the proposed method, which may include a prediction system completely or partially.
[0079] The computer system 600 has multiple general-purpose functions. The computer system may be a tablet computer, a laptop / notebook computer, another portable or mobile electronic device, a microprocessor system, a microprocessor-based system, a smart phone, a computer system with specially configured special functions, or a component part of another microscope system. The computer system 600 may be configured to execute computer system executable instructions, such as program modules, which may be executed to implement the functionality of the concepts proposed herein. To this end, program modules may include routines, programs, objects, components, logic, data structures, etc., for implementing particular tasks or particular abstract data types.
[0080] Components of the computer system may include the following: one or more processors or processing units 602, a storage system 604, and a bus system 606 that couples various system components including the storage system 604 to the processor 602. The computer system 600 typically has multiple volatile or non-volatile storage media accessible by the computer system 600. The storage system 604 may store data and / or instructions (commands) of the storage media in a volatile form, such as a RAM (random access memory) 608, for execution by the processor 602. These data and instructions implement one or more functions and / or steps of the concepts presented herein. Further components of the storage system 604 may be a permanent memory (ROM) 610 and a long-term memory 612, in which program modules and data (reference numeral 616) as well as workflows may be stored.
[0081] The computer system comprises a number of dedicated devices for communication purposes, such as a keyboard 618, a mouse / pointing device (not shown), a visual display unit 620, etc. These dedicated devices may be combined with a touch-sensitive display. A separately provided I / O controller 614 ensures frictionless data exchange with external devices. A network adapter 622 can be used for communication over a local or global network (LAN, WAN, e.g. via the Internet). The network adapter may be accessed by other components of the computer system 600 via the bus system 606. It is understood that other devices may also be connected to the computer system 600, although not shown in this case.
[0082] At least a portion of the system 500 for determining the optical power of an IOL (see FIG. 5) may also be connected to a bus system 606 .
[0083] The description of various exemplary embodiments of the present invention has been given for the purpose of improved understanding, but does not serve to directly limit the concept of the present invention to these exemplary embodiments. Further modifications and variations will be developed by those skilled in the art. The terms used herein are selected to best explain the basic principles of the exemplary embodiments and to make them easily understandable to those skilled in the art.
[0084] The principles presented herein may be embodied as a system, a method, a combination thereof, and / or as a computer program product, which may comprise one or more computer-readable storage media having computer-readable program instructions for causing a processor or control system to implement various aspects of the invention.
[0085] As a medium, electronic, magnetic, optical, electromagnetic, or infrared media, or semiconductor systems may be used as a transmission medium, such as SSD (Solid State Device / Drive as solid state memory), RAM (Random Access Memory), and / or ROM (Read Only Memory), EEPROM (Electrically Erasable Read Only Memory), or any combination thereof. Suitable transmission media also include propagating electromagnetic waves, electromagnetic waves in a wave guide or other transmission medium (e.g., light pulses in an optical cable), or electrical signals transmitted in wires.
[0086] A computer readable storage medium may be an embodied device that holds or stores instructions for use by an instruction execution device. The computer readable program instructions described herein may be downloaded, for example as a (smartphone) app, to a corresponding computer system from a service provider via a cable-based connection or a mobile wireless network.
[0087] The computer readable program instructions for carrying out the operations of the present invention described herein may be machine dependent or independent instructions, microcode, firmware, status definition data, or any source or object code written in a conventional procedural programming language, such as C++, Java, or the like, or the programming language "C" or a similar programming language. The computer readable program instructions may be executed entirely by a computer system. In some exemplary embodiments, there may also be electronic circuitry, such as, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), that executes the computer readable program instructions by using status information of the computer readable program instructions to configure or individualize an electronic circuit according to aspects of the present invention.
[0088] The invention presented herein is further described with reference to flow diagrams and / or block diagrams of methods, devices (systems) and computer program products according to exemplary embodiments of the invention. It should be noted that in practice any block of the flow diagrams and / or block diagrams may be embodied as computer readable program instructions.
[0089] The computer readable program instructions may be made available to a general purpose computer, a special purpose computer, or some other way programmable data processing system to generate machines such that the instructions, executed by a processor or computer or other programmable data processing device, generate means for implementing the functions or processes shown in the flow diagrams and / or block diagrams. These computer readable program instructions may thus be stored on a computer readable storage medium.
[0090] In this sense, any block in the illustrated flow diagrams or block diagrams may represent a module, segment, or portion of instructions that represents multiple executable instructions for implementing a particular logical function(s). In some exemplary embodiments, the functions represented in the individual blocks may be implemented in different orders, optionally even in parallel.
[0091] The illustrated structures, materials, sequences, and equivalents of all means and / or steps having the relevant functionality in the following claims are intended to cover all such structures, materials, or sequences as are expressed by the claims. [Explanation of symbols]
[0092] 100 Ways to Adjust White Balance for Different Situations 102 100 Method Steps 104 100 Method Steps 106 100 Method Steps 108 100 Method Steps 110 100 Method Steps 200 Eye with intraocular lens 202 Intraocular Lens 204 Capsule 206 Lateral structure 208 Ligaments and Muscles 300 eyes 302 Axial length 304 Anterior chamber thickness 306 Corneal curvature measurements 308 Lens thickness 310 Central Corneal Thickness 312 Corneal lateral diameter distance 314 Pupil diameter 316 Posterior chamber depth 318 Retinal Thickness 400 Function blocks for implementing the method 402 scan result 404 Biometric Data 406 Neural Networks 408 IOL position 410 Formula-based determination of refractive power 412 Machine Learning Systems 414 Final power 500 A system for predicting refractive power 502 Product Module 504 First Training Module 506 Second Training Module 508 Receiver Module 510 Further Machine Learning Systems 512 Bus System 600 Prediction System 600 Computer Systems 602 Processor 604 Storage System 606 Bus System 608 RAM 610 ROM 612 Long-term Memory 614 I / O Controller 616 Program Modules, Latent Data 618 Keyboard 620 screens 622 Network Adapter
Claims
1. 1. A computer-implemented method (100) for a machine learning assisted processing pipeline for determining parameter values for an intraocular lens (202) to be inserted, comprising: - providing (102) a scan (402) of an eye (300), said scan (402) representing an image of the anatomical structure of said eye (300); - determining (104) biometric data (404) of said eye (300) from said eye scan (402); - using (106) a first trained machine learning system (406) comprising a first trained machine learning model to determine a final position (408) of an inserted intraocular lens (202), said biometric data (404) serving as input data for said first trained machine learning system (406); - determining (108) a first refractive power of the intraocular lens (202) to be inserted, said determining being based on a physical model (410) in which the determined final position (408) of the intraocular lens (202) and the determined biometric data (404) are used as input variables for said physical model (410); determining the final refractive power (414) of said intraocular lens by a second trained machine learning system (412), in which said biometric data (404) and said first refractive power are used as input variables, The second trained machine learning system (412) is trained in two stages: The first training step is generating first training data for said second machine learning system (412) based on a first physical model for refractive power for an intraocular lens; training said second machine learning system (412) based on said first physical model for the purpose of forming a first learning model for determining refractive power (414), said first training data being used to train said second machine learning system (412); The second training step is training the already-trained second machine learning system (412) based on the first physical model for the purpose of forming a second learning model for the second machine learning system (412) determining refractive power (414), wherein clinical ophthalmology training data is used for training the already-trained second machine learning system (412), Method (100).
2. 2. The method (100) of claim 1, wherein the biometric data of the eye includes at least one selected from the group consisting of pre-operative axial length (302), pre-operative lens thickness, pre-operative anterior chamber depth (304), and intra-operative anterior chamber depth.
3. 3. The method of claim 1 or 2, wherein the first machine learning system is a convolutional neural network, a graph attention network, or a combination of two of said networks.
4. The method (100) of claim 2 or claim 3, wherein the one variable from the biometric data (404) is the pre-operative axial length.
5. The method (100) according to any one of the preceding claims, wherein the eye biometric data (404) is determined manually from the image or by a machine learning system from the provided scan result (402) of the eye.
6. The method (100) according to any one of the preceding claims, wherein further parameters of the eye (300) are determined when determining the final position of the inserted intraocular lens.
7. A processing pipeline system (500) for a machine learning assisted processing pipeline for determining parameter values for an intraocular lens (202) to be inserted, comprising: a receiving module (502) configured to provide a scan (402) of an eye (300), said scan (402) representing an image of the anatomical structure of said eye (300); a determination unit (504) adapted to determine biometric data (404) of the eye (300) from said scan (402) of said eye (300); a first trained machine learning system (406), comprising a first trained machine learning model for determining a final position (408) of an inserted intraocular lens (202), said biometric data (404) serving as input data for said first trained machine learning system (406); a determination unit (508) configured to determine a first refractive power of the intraocular lens to be inserted, said determination being based on a physical model for which the determined final position of the intraocular lens and the determined biometric data are used as input variables for the physical model; a determination unit configured to determine a final refractive power (414) of said intraocular lens by a second trained machine learning system (412), said biometric data (404) and said first refractive power being used as input variables, The second trained machine learning system (412) is trained in two stages: The first training step is generating first training data for said second machine learning system (412) based on a first physical model for refractive power for an intraocular lens; training said second machine learning system (412) based on said first physical model for the purpose of forming a first learning model for determining refractive power (414), said first training data being used to train said second machine learning system (412); The second training step is training the already-trained second machine learning system (412) based on the first physical model for the purpose of forming a second learning model for the second machine learning system (412) determining refractive power (414), wherein clinical ophthalmology training data is used for training the already-trained second machine learning system (412), A processing pipeline system (500).
8. 10. A computer program product for a machine learning assisted processing pipeline for determining parameter values for an intraocular lens to be inserted, the computer program product having a computer readable storage medium having program instructions stored thereon, the program instructions being executable by one or more computers or control units to cause the one or more computers or control units to perform the method of any one of claims 1 to 6.
Citation Information
Patent Citations
Automated lens design and manufacturing system
JP2004534964A
Ophthalmologic apparatus and IOL diopter determination program
JP2018051223A
Systems and methods for intraocular lens selection
JP2021531071A
Methods and apparatuses for designing optical systems
US20190094532A1
Systems, apparatuses, and methods for intraocular lens selection using artifical intelligence
US20190099262A1
