Augmentation of ocular optical coherence tomography images based on learning models

Abstract

A system and method for enhancing an original OCT (optical coherence tomography) image includes a controller having a processor and a tangible, non-transitory memory having instructions recorded thereon. The system includes one or more learning modules selectively executable by the controller. The learning modules are trained by a training network using a training dataset having a plurality of training ultrasound biomicroscopy images and corresponding training OCT images. The processor executes the instructions to cause the controller to acquire an original OCT image captured via an OCT device. The controller is configured to execute the (trained) learning module to generate an enhanced OCT image based in part on the original OCT image. The enhanced OCT image at least partially enhances a peripheral portion of the original OCT image.

Description

[Technical Field]

[0001] The present disclosure generally relates to enhancing optical coherence tomography images of the eye based on one or more learning models. Optical coherence tomography ("OCT") is a non-invasive imaging technique that uses low-coherence interferometry to generate high-resolution images of ocular structures. OCT imaging works in part by measuring the time delay of echoes and the magnitude of backscattered light. Images generated by OCT are useful for many purposes, including identifying and evaluating ocular diseases. OCT images are often taken before cataract surgery, in which an intraocular lens is implanted in a patient's eye. An inherent limitation of OCT imaging is the inability of the illumination beam to penetrate the iris. As a result, posterior regions of the eye, such as the lens structure behind the iris, may not be accurately visualized. Summary of the Invention [Means for solving the problem]

[0002] Disclosed herein are systems and methods for enhancing original optical coherence tomography (OCT) images of an eye. The system includes a controller having a processor and a tangible, non-transitory memory having instructions recorded thereon. The system includes one or more learning modules (hereafter omitted as "one or more") selectively executable by the controller. The learning modules are trained by a training network using a training dataset having a plurality of training ultrasound biomicroscopy images and corresponding training OCT images. The processor executes the instructions to cause the controller to acquire original OCT images captured via an OCT device. The controller is configured to execute the (trained) learning module to generate an enhanced OCT image based in part on the original OCT image. The enhanced OCT image at least partially enhances peripheral portions of the original OCT image. In other words, the system enables reconstruction of information missing from the original OCT image.

[0003] The peripheral portion may be posterior to the iris of the eye, and the enhanced OCT image may enable visualization of one or more structures posterior to the iris. The controller may be configured to obtain at least one lens parameter based on the enhanced OCT image. The lens parameter may include a lens diameter and / or a lens capsule profile. The controller may be configured to select an intraocular lens for the eye based in part on the lens parameter. The OCT device may comprise an array of laser beams for illuminating the eye.

[0004] In some embodiments, the corresponding training OCT image is associated with a plurality of training ultrasound biomicroscopy images, and the corresponding training OCT image and the plurality of training ultrasound biomicroscopy images form a paired set (i.e., images of the same eye). The learning module may include a generator trained to generate corresponding synthetic OCT images based in part on the corresponding training OCT images. The training network may be a generative adversarial network having a discriminator. The discriminator is adapted to distinguish between the plurality of training ultrasound biomicroscopy images and the corresponding synthetic OCT images.

[0005] In some embodiments, the corresponding training OCT image is not associated with the plurality of training ultrasound biomicroscopy images (i.e., is an image of a different eye), and the corresponding training OCT image and the plurality of training ultrasound biomicroscopy images form an unpaired set. The training network may be a generative adversarial network having a first discriminator and a second discriminator. The learning module may include a first generator and a second generator. The augmented OCT image is generated by sequentially executing the first generator and the second generator, where the first generator is adapted to convert the original OCT image of the eye into a corresponding synthetic UBM image, and the second generator is adapted to convert the corresponding synthetic UBM image into the augmented OCT image.

[0006] The training network may be configured to perform a positive training cycle using a first generator, a second generator, and a first discriminator. Here, a first training OCT image is input to the first generator, and the first training OCT image is selected from the corresponding training OCT images. The first generator is adapted to convert the first training OCT image into a first synthetic ultrasound biomicroscopy image. The second generator is adapted to convert the first synthetic ultrasound biomicroscopy image into a second synthetic OCT image. The first discriminator is adapted to distinguish the first synthetic ultrasound biomicroscopy image from multiple training ultrasound biomicroscopy images in the positive training cycle. The training network incorporates a first loss function that minimizes the difference between the first training OCT image and the second synthetic OCT image.

[0007] The training network may be further configured to perform a reverse training cycle using a first generator, a second generator, and a second discriminator. Here, a second training ultrasound biomicroscopy image is input to the second generator, and the second training ultrasound biomicroscopy image is selected from the plurality of training ultrasound biomicroscopy images. The second generator is configured to convert the second training ultrasound biomicroscopy image into a third synthetic OCT image. The first generator is configured to convert the third synthetic OCT image into a fourth synthetic ultrasound biomicroscopy image. The second discriminator is adapted to distinguish between the third synthetic OCT image and the corresponding training OCT image in the reverse training cycle. The training network may incorporate a second loss function that minimizes the difference between the second training ultrasound biomicroscopy image and the fourth synthetic ultrasound biomicroscopy image.

[0008] Disclosed herein is a method for enhancing an original optical coherence tomography ("OCT") image of an eye by a system including a controller having at least one processor and at least one non-transitory tangible memory. The method includes configuring the controller to selectively execute one or more learning modules. The learning modules are trained via a training network using a training dataset having a plurality of training ultrasound biomicroscopy images and corresponding training OCT images. The method includes capturing an original OCT image of the eye via an OCT device. An enhanced OCT image is generated based in part on the original OCT image by executing the one or more learning modules. The enhanced OCT image at least partially enhances a peripheral portion of the original OCT image.

[0009] In some embodiments, the peripheral portion is positioned posterior to the iris of the eye such that the enhanced OCT image allows visualization of one or more structures posterior to the iris. The method includes obtaining at least one lens parameter based on the enhanced OCT image, where the lens parameter may include a lens diameter and / or a lens capsule profile. An intraocular lens may be selected based in part on the lens parameter.

[0010] Capturing the original OCT image of the eye may include illuminating the eye with an array of laser beams via an OCT device. The method may include constructing a training data set using paired sets of training ultrasound biomicroscopy images and corresponding training OCT images (i.e., images of the same eye). Alternatively, the method may include constructing a training data set using unpaired sets of training ultrasound biomicroscopy images and corresponding training OCT images (i.e., images of different eyes).

[0011] The above and other features and advantages of the present disclosure will become readily apparent from the following detailed description of the best mode for carrying out the disclosure, when read in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a schematic diagram of a system for enhancing original optical coherence tomography ("OCT") images of an eye, the system including a controller and one or more learning modules. [Figure 2] FIG. 2 is a schematic flow chart of a method that can be performed by the controller of FIG. [Figure 3] FIG. 3 is a schematic diagram of an exemplary original OCT image of an eye. [Figure 4] FIG. 4 is a schematic diagram of an ultrasound biomicroscopy (UBM) image of the eye. [Figure 5] Figure 5 is a schematic illustration of a dilated OCT image of the eye. [Figure 6] FIG. 6 is a schematic flowchart of an exemplary training method for the learning module of FIG. 1 according to the first embodiment. [Figures 7A-7B] 7A is a schematic flowchart of an exemplary forward training cycle for the learning module of FIG. 1 according to the second embodiment, and FIG. 7B is a schematic flowchart of an exemplary reverse training cycle for the learning module of FIG. 1 according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Referring to the drawings, wherein like reference numbers refer to like components throughout, FIG. 1 schematically illustrates a system 10 for enhancing an original optical coherence tomography (hereinafter "OCT") image of an eye 12 captured via an optical coherence tomography (OCT) device 14. The OCT device 14 may employ an array of laser beams 16 to illuminate the eye 12. The array of laser beams 16 may cover an area or width of the eye 12. In one example, the OCT device 14 is an anterior segment high-resolution OCT imaging device. It should be understood that the OCT device 14 may take many different forms and may include multiple and / or alternative components.

[0014] 1, system 10 includes a controller C having at least one processor P and at least one memory M (or non-transitory tangible computer-readable storage medium) having instructions recorded thereon for performing a method 100 for enhancing an original OCT image 200 of an eye 12. Method 100 is shown in and described below with reference to FIG.

[0015] An example of an original OCT image 200 is shown schematically in FIG. 3 and described below. In the illustrated example, the original OCT image 200 shows an anterior segment image. Referring to FIG. 3, the original OCT image 200 shows an iris 202, a lens 204, and a pupil 205. OCT imaging does not capture a peripheral portion 206 of the lens 204 behind the iris 202. This is because the illumination laser used in OCT imaging cannot penetrate the iris 202. However, OCT imaging techniques offer high resolution and a non-contact scanning method that is convenient in terms of patient compliance and comfort in everyday clinical settings. For example, OCT imaging is performed in a seated position, takes a relatively short time, and does not involve the use of an eyecup or coupling medium. As described below, the system 10 acquires one or more learning modules 18 (hereinafter, "one or more" omitted) that are trained to extrapolate the original OCT image 200 captured by the OCT device 14 and reconstruct an enhanced OCT image showing the peripheral portion 206. The system 10 allows for the reconstruction of a complete image of the crystalline lens 204 based on the original OCT image 200 .

[0016] Referring to FIG. 1 , the controller C is specifically programmed to selectively execute a learning module 18, which may be embedded in the controller C or stored elsewhere and accessible to the controller C. Referring to FIG. 1 , the learning module 18 is trained by a training network 20 using a training dataset including a plurality of training ultrasound biomicroscopy images (and corresponding training OCT images, described below). An example of a training ultrasound biomicroscopy image 300 is shown schematically in FIG. 4 . The training ultrasound biomicroscopy image 300 shows an iris 302 and a lens 304. The training ultrasound biomicroscopy image 300 also shows a peripheral portion 306 of the lens 304. While ultrasound biomicroscopy images can capture the entire lens structure, they do so at a lower resolution than images acquired by OCT imaging. However, ultrasound biomicroscopy is inconvenient for patients. For example, ultrasound biomicroscopy requires longer image acquisition times, a skilled operator, and a plastic or silicone eyecup to hold the coupling medium.

[0017] The controller C is configured to generate an enhanced OCT image based in part on the original OCT image 200 by executing one or more learning modules 18. An example of an enhanced OCT image 400 is shown schematically in FIG. 5 and described below. The enhanced OCT image 400 shows an iris 402 and a lens 404. The enhanced OCT image 400 at least partially enlarges a peripheral portion 406 of the original OCT image. The peripheral portion 406 is posterior to the iris 402, allowing visualization of one or more structures posterior to the iris 402.

[0018] 1 utilizes convolutional neural network (CNN)-based deep learning techniques to enhance the original OCT image 200. The training network 20 may incorporate a generative adversarial network (GAN). In one embodiment, the training network 20 incorporates a cycle generative adversarial network (cycleGAN).

[0019] The controller C is configured to acquire at least one lens parameter based on the enhanced OCT image 400. Referring to FIG. 4 , the lens parameters may include a lens capsule profile 408, a lens diameter 410, and a thickness 412 of the lens 404 along the lens diameter 410. The lens parameters may be output to the lens selection module 22 for selecting an intraocular lens 24 for insertion into the eye 12. A complete image of the preoperative lens structure is important for selecting the appropriate power of the intraocular lens 24 during preoperative evaluation of cataract surgery. This information is particularly useful for accommodative intraocular lenses 24 because the functional performance of the accommodative intraocular lenses 24 has been found to be correlated with the lens diameter 410. Additionally, the lens capsule profile 408 may be employed to estimate the possible postoperative position of the intraocular lens 24.

[0020] The various components of the system 10 of FIG. 1 may communicate via a short-range network 26 and / or a long-range network 28. The short-range network 26 may be a bus implemented in various ways, such as a serial communication bus in the form of a local area network. The local area network may include, but is not limited to, a Controller Area Network (CAN), a Controller Area Network with Flexible Data Rate (CAN-FD), Ethernet, Bluetooth, Wi-Fi, and other data connection topologies. Referring to FIG. 1, the long-range network 28 may be a wireless local area network (LAN) that links multiple devices in a wirelessly distributed manner, a wireless metropolitan area network (MAN) that connects several wireless LANs, or a wireless wide area network (WAN) that covers a large geographic area such as a nearby city or town. Other types of connections may also be used.

[0021] Referring now to Figure 2, there is shown a flowchart of a method 100 executable by the controller C of Figure 1. The method 100 does not have to be applied in the particular order described herein, and some blocks may be omitted. The memory M may store a set of controller-executable instructions, and the processor P may execute the set of controller-executable instructions stored in the memory M.

[0022] In block 102 of Figure 2, controller C is configured to collect one or more training data sets from one or more facilities or clinical sites around the world. Controller C may communicate with the facilities via short-range network 26 and / or long-range network 28. Referring to Figure 1, system 10 may include a data management module 30 having a computerized data management system that can store information from the facilities' corresponding electronic medical records. Data management module 30 may be configured to collect the training data sets from the facilities and provide them to controller C.

[0023] The training dataset may include images taken from multiple patients. In some embodiments, the training dataset further includes paired datasets, i.e., corresponding training OCT images that are associated with the multiple training ultrasound biomicroscopy images because they are from the same eye. In other embodiments, the training dataset further includes unpaired datasets, i.e., corresponding training OCT images that are not associated with the multiple training ultrasound biomicroscopy images (taken from different eyes). The training dataset may be stratified based on demographic data, patients with similar eye dimensions, or other health factors.

[0024] 2, the method 100 includes training a learning module via the training network 20 using the training data set from block 102. Two embodiments of the training process are described below. It should be understood that the system 10 is not limited to any particular deep neural network methodology. Reconstruction of missing information from the original OCT image 200 can be aided by other deep neural network methodologies available to those skilled in the art.

[0025] In the first embodiment, the training network 20 is trained using a discriminator D * Combined with the generator G for image synthesis * Incorporating a deep learning architecture (such as a generative adversarial network (GAN)) to train the discriminator D. An example of a first embodiment is described below in connection with FIG. 6. * is trained directly on real and generated images and is responsible for classifying images as real or fake. * is not trained directly, but instead is trained on the discriminator D * are trained through

[0026] Referring to FIG. 6, a training method 500 is shown that may be executed by a controller C. In block 502, training OCT images are acquired. In this embodiment, the training data set includes paired data sets of OCT images and ultrasound biomicroscopy images taken of the same patient. In block 504 of FIG. 6, the training method 500 starts with a generator G * In block 506 of FIG. * generates a synthetic ultrasound biomicroscopy image based in part on a corresponding training OCT image that extrapolates the data acquired in block 502. In block 508, a training ultrasound biomicroscopy image is acquired that pairs with the training OCT image (acquired in block 502).

[0027] In block 510 of FIG. 6, the training method 500 begins by training a discriminator D *The discriminator D* is a generator G * The output of generator G is used to "evaluate" the output of generator G to determine whether the output (the synthetic ultrasound biomicroscopy images of block 506) is sufficiently close to the "real" training data (the training ultrasound biomicroscopy images of block 508). Comparisons are made between images. For example, the loss function may minimize the difference between the intensities of individual pixels between the two images. * attempts to create synthetic ultrasound biomicroscopy images that are as close as possible to the "real" training data (the training ultrasound biomicroscopy images of block 508). Thus, the discriminator D * is the generator G * It is trained to provide a loss function for

[0028] The training method 500 then proceeds to block 512 to determine whether a predefined threshold has been met. In one example, the predefined threshold is met if the difference in corresponding intensities of pixels (registered to the same physical location) between the two images is within a predetermined value, such as 10%. In another example, the predefined threshold is met if the difference in lens diameter between the two images is within a predetermined value. In addition, the predefined threshold may be met if the difference in other lens-related parameters, such as endocapsule height, between the two images is within a predetermined value. The predetermined value may be within 5% or within 5 mm. If the predefined threshold is met, the training method 500 ends. If the predefined threshold is not met, the training method 500 proceeds to block 512, where the learning module 18 is updated, and the training method 500 loops back to block 504. The training process occurs in a closed-loop or iterative manner, where the learning module 18 is trained until certain criteria are met. In other words, the training process continues until the discrepancy between the network's results and the ground truth reaches a point below a certain threshold. The learning module 18 reaches convergence when the loss function associated with the training data set is minimized. Convergence signals the completion of training.

[0029] System 10 may be configured to be "adaptive" and periodically updated after collection of additional data for the training dataset. In other words, learning module 18 may be configured to be an "adaptive machine learning" algorithm that is not static but improves after additional training datasets are collected. In some embodiments, training network 20 may employ a separate image bank of lens structure from multiple training ultrasound biomicroscopy images. For example, training ultrasound biomicroscopy image 300 may include only structural details of lens 304.

[0030] In a second embodiment, the training network 20 incorporates a cycle generative adversarial network (cycleGAN), an example of which is illustrated in Figures 7A and 7B. In this embodiment, the training dataset includes an unpaired set of training ultrasound biomicroscopy images and corresponding training OCT images. In other words, the learning module 18 is adapted to take properties of one image domain and determine how these properties can be transferred to another image domain, all without paired training examples.

[0031] 7A and 7B, the training network 20 includes a first discriminator D1 and a second discriminator D2. Each of the first generator G1, the second generator G2, the first discriminator D1, and the second discriminator D2 may incorporate a separate neural network with a different goal.

[0032] The training network 20 is configured to perform a positive training cycle 600 using a first generator G1, a second generator G2, and a first discriminator D1, as shown in FIG. 7A. As indicated by arrow 602, a first training OCT image T1 (selected from the corresponding training OCT images) is input to the first generator G1. The first generator G1 converts the first training OCT image T1 into a first synthetic ultrasound biomicroscopy (UBM) image S1, as indicated by arrow 604. At arrow 606, the first synthetic ultrasound biomicroscopy image S1 is input to a second generator G2. The second generator G2 then converts the first synthetic ultrasound biomicroscopy image S1 into a second synthetic OCT image S2, as indicated by arrow 608.

[0033] 7A, a first synthetic ultrasound biomicroscopy image S1 is input to a first discriminator D1 at arrow 610. A plurality of training ultrasound biomicroscopy images (including a plurality of images) are also input to the first discriminator D1 at arrow 612. The first discriminator D1 is adapted to distinguish between the first synthetic ultrasound biomicroscopy image S1 and the plurality of training ultrasound biomicroscopy images in a positive training cycle 600.

[0034] Referring to FIG. 7A, the training network 20 incorporates a first loss function L1 that minimizes the difference between a first training OCT image T1 and a second synthetic OCT image S2. The first loss function L1 attempts to capture the difference between the distribution of generated data and the "ground truth." The first loss function L1 can incorporate both an adversarial loss and a cycle consistency loss, and can include, but is not limited to, a minimax function, a least-squares function, a Wasserstein loss function, or other suitable functions. A first discriminator D1 attempts to minimize the first loss function L1, and a first generator G1 attempts to maximize the first loss function L1 by synthesizing indistinguishable images from multiple training ultrasound biomicroscopy images. Simultaneously, a second generator G2 attempts to maximize this loss by synthesizing images that are indistinguishable from the corresponding OCT training image. Additional loss functions may be added as appropriate based on the application. For example, generator G1 and generator G2 may be adapted to minimize the difference in lens diameter 410 between the first training OCT image T1 and the second composite OCT image S2.

[0035] The training network 20 (see FIG. 1) is configured to perform a reverse training cycle 650 using a first generator G1, a second generator G2, and a second discriminator D2, as shown in FIG. 7B. In composition, the reverse training cycle 650 uses the same first generator G1 and second generator G2 as the forward training cycle 600.

[0036] As shown by arrow 652, a second training ultrasound biomicroscopy image T2 (taken from multiple training ultrasound biomicroscopy images) is input to a second generator G2. The second generator G2 converts the second training ultrasound biomicroscopy image T2 into a third synthetic OCT image S3 at arrow 654. The third synthetic OCT image S3 is input to a first generator G1 at arrow 656. The first generator G1 converts the third synthetic OCT image S3 into a fourth synthetic ultrasound biomicroscopy image S4 at arrow 658. Referring to FIG. 7B, the training network 20 incorporates a second loss function L2 that minimizes the difference between the second training ultrasound biomicroscopy image T2 and the fourth synthetic ultrasound biomicroscopy image S4. The second loss function L2 is similar to the first loss function L1 and may incorporate both an adversarial loss and a cycle consistency loss.

[0037] The third composite OCT image S3 is input to a second discriminator D2 according to arrow 656. Referring to FIG. 7A , the second discriminator D2 is adapted to distinguish between the third composite OCT image S3 (input at arrow 660) and the corresponding training OCT image (input at arrow 662) in a reverse training cycle 650. Additional loss functions may be added. For example, generators G1 and G2 may be adapted to minimize the difference in lens diameter 410 between the second training ultrasound biomicroscopy image T2 and the fourth composite biomicroscopy image S4. As data becomes more abundant, the newly trained learning module 18 may continually improve to produce a more refined augmented OCT image 400 with the entire lens structure of the lens 404.

[0038] Referring now to block 106 of FIG. 2 , the controller C is configured to acquire subject data, i.e., original OCT images 200 of the eye 12, via the OCT device 14. The controller C may be configured to receive and transmit data via a user interface 32. The user interface 32 may be installed on a smartphone, laptop, tablet, desktop, or other electronic device and may include a touchscreen interface or I / O devices such as a keyboard or mouse. The user interface 32 may be a mobile application. Circuits and components of mobile applications (“apps”) available to those skilled in the art may be employed. The user interface 32 may include an integrated processor and integrated memory.

[0039] 2, the controller C is configured to execute the (trained) learning module 18 to acquire the enhanced OCT image 400. In some embodiments, the learning module 18 is configured to execute the (trained) learning module 18 to acquire the enhanced OCT image 400. * The enhanced OCT image 400 includes a generator G * (See FIG. 6 ). In some embodiments, the learning module 18 includes a first generator G1 and a second generator G2, and the enhanced OCT image 400 is generated by sequentially executing the first generator G1 and the second generator G2. The first generator G1 is adapted to convert the original OCT image 200 of the eye 12 into a corresponding composite ultrasound biomicroscopy image, and the second generator G2 is adapted to convert the corresponding composite ultrasound biomicroscopy image into the enhanced OCT image 400.

[0040] 2, the controller C is configured to determine at least one lens parameter based on the enhanced OCT image 400. As described above and with reference to FIG. 4, the lens parameters may include a lens capsule profile 408, a lens diameter 410, and a thickness 412 of the lens 404 along the lens diameter 410. The lens parameters may be output to the lens selection module 22. Additionally, in block 112 of FIG. 2, the controller C may be configured to select an intraocular lens 24 based at least in part on the lens parameters determined in block 112.

[0041] In summary, the system 10 demonstrates a robust manner for reconstructing information not available from the original OCT image 200 of the eye 12 by utilizing one or more learning modules 18. The system 10 is adapted to estimate the peripheral portion 206 of the original OCT image 200. Technical advantages include improved power calculations for the intraocular lens 24 and appropriate selection of the accommodating intraocular lens 24.

[0042] The controller C of FIG. 1 includes computer-readable media (also referred to as processor-readable media) that include non-transitory (e.g., tangible) media that participate in providing data (e.g., instructions) that can be read by a computer (e.g., by a processor of a computer). Such media can take many forms, including, but not limited to, non-volatile and volatile media. Non-volatile media include, for example, optical or magnetic disks and other persistent memory. Volatile media include, for example, dynamic random access memory (DRAM), which may constitute primary storage. Such instructions may be transmitted over one or more transmission media, including coaxial cables, copper wire, and optical fiber, including the wires that comprise a system bus coupled to the computer's processor. Some forms of computer-readable media include, for example, floppy disks, flexible disks, hard disks, magnetic tape, other magnetic media, CD-ROMs, DVDs, other optical media, punch cards, paper tape, other physical media with patterns of holes, RAM, PROMs, EPROMs, Flash EEPROMs, other memory chips or cartridges, or other computer-readable media.

[0043] The lookup tables, databases, data repositories, or other data stores described herein may include various types of mechanisms for storing, accessing, and retrieving various types of data, including a hierarchical database, a set of files in a file system, a proprietary application database, a relational database management system (RDBMS), etc. Each such data store may be contained within a computing device employing a computer operating system such as one of those described above, or may be accessed over a network in one or more of a variety of ways. The file system may be accessible from the computer operating system and may include files stored in various formats. The RDBMS may employ a Structured Query Language (SQL) in addition to a language for creating, storing, editing, and executing stored procedures, such as the PL / SQL language described above.

[0044] While the detailed description and drawings or figures support and explain the present disclosure, the scope of the present disclosure is defined solely by the claims. While the best mode and some other embodiments for carrying out the claimed disclosure have been described in detail, various alternative designs and embodiments exist for carrying out the disclosure defined in the appended claims. Furthermore, the features of the various embodiments shown in the drawings or described herein should not necessarily be understood as independent embodiments of one another. Rather, each of the characteristics described in one of the example embodiments can be combined with one or more other desirable characteristics from other embodiments, resulting in other embodiments not described in words or with reference to the drawings. Accordingly, such other embodiments are encompassed within the scope of the appended claims. According to aspect (1), a controller having at least one processor and at least one non-transitory tangible memory having recorded thereon instructions for a method for enhancing an original optical coherence tomography ("OCT") image of an eye; one or more learning modules selectively executable by said controller; In a system comprising: the one or more learning modules are trained by a training network using a training dataset having a plurality of training ultrasound biomicroscopy images and corresponding training OCT images; Execution of the instructions by the processor causes the controller to acquire the original OCT image of the eye, the original OCT image being captured via an OCT device; the controller is configured to execute the one or more learning modules to generate an augmented OCT image based in part on the original OCT image; The system wherein the enhanced OCT image at least partially expands a peripheral portion of the original OCT image. According to aspect (2), the peripheral portion is behind the iris of the eye, and the enhanced OCT image allows visualization of one or more structures behind the iris. According to aspect (3), the controller is configured to acquire at least one lens parameter based on the extended OCT image. According to aspect (4), the controller is configured to select an intraocular lens based in part on the at least one lens parameter; The at least one lens parameter includes a lens diameter and / or a lens capsule profile. According to aspect (5), the OCT device comprises an array of laser beams for illuminating the eye. According to aspect (6), the corresponding training OCT image is associated with the plurality of training ultrasound biomicroscopy images, and the corresponding training OCT image and the plurality of training ultrasound biomicroscopy images form a paired set. According to aspect (7), the one or more learning modules include a generator trained to generate corresponding synthetic OCT images based in part on the corresponding training OCT images; the training network is a generative adversarial network with a discriminator; The discriminator is adapted to distinguish between the plurality of training ultrasound biomicroscopy images and the corresponding synthetic OCT images. According to aspect (8), the corresponding training OCT image is not associated with the plurality of training ultrasound biomicroscopy images, and the corresponding training OCT image and the plurality of training ultrasound biomicroscopy images form an unpaired set; The training network is a generative adversarial network having a first discriminator and a second discriminator. According to aspect (9), the one or more learning modules include a first generator and a second generator; The enhanced OCT image is generated by sequentially executing the first generator and the second generator, the first generator adapted to convert the original OCT image of the eye into a corresponding composite UBM image, and the second generator adapted to convert the corresponding composite UBM image into the enhanced OCT image. According to aspect (10), the training network is configured to perform a positive training cycle using the first generator, the second generator, and the first discriminator; a first training OCT image is input to the first generator, and the first training OCT image is selected from the corresponding training OCT images; the first generator is adapted to convert the first training OCT image into a first synthetic ultrasound biomicroscopy image; The second generator is adapted to convert the first composite ultrasound biomicroscopy image into a second composite OCT image. According to aspect (11), the first discriminator is adapted to distinguish between the first synthesized ultrasound biomicroscopy image and the plurality of training ultrasound biomicroscopy images in the positive training cycle; The training network incorporates a first loss function that minimizes the difference between the first training OCT image and the second synthetic OCT image. According to aspect (12), the training network is configured to perform a reverse training cycle using the first generator, the second generator, and the second discriminator; a second training ultrasound biomicroscopy image is input to the second generator, and the second training ultrasound biomicroscopy image is selected from the plurality of training ultrasound biomicroscopy images; the second generator is configured to convert the second training ultrasound biomicroscopy image into a third composite OCT image; The first generator is configured to convert the third composite OCT image into a fourth composite ultrasound biomicroscopy image. According to aspect (13), the second discriminator is adapted to distinguish between the third synthetic OCT image and the corresponding training OCT image in the reverse training cycle; The training network incorporates a second loss function that minimizes the difference between the second training ultrasound biomicroscopy image and the fourth synthesized ultrasound biomicroscopy image. According to aspect (14), there is provided a method for enhancing an original optical coherence tomography ("OCT") image of an eye by a system comprising a controller having at least one processor and at least one non-transitory tangible memory, the method comprising: configuring the controller to selectively execute one or more learning modules; training one or more learning modules via a training network using a training dataset having a plurality of training ultrasound biomicroscopy images and corresponding training OCT images; capturing the original OCT image of the eye via an OCT device; generating an enhanced OCT image based in part on the original OCT image by executing the one or more training modules, wherein the enhanced OCT image at least partially expands a peripheral portion of the original OCT image; The method includes: According to aspect (15), the peripheral portion is positioned behind the iris of the eye such that the enhanced OCT image allows visualization of one or more structures behind the iris. Further includes:

Claims

1. a controller having at least one processor and at least one non-transitory tangible memory having instructions recorded thereon for a method for enhancing an original optical coherence tomography ("OCT") image of an eye; one or more learning modules selectively executable by said controller; In a system comprising: the one or more learning modules are trained by a training network using a training dataset having a plurality of training ultrasound biomicroscopy images and corresponding training OCT images; Execution of the instructions by the processor causes the controller to acquire the original OCT image of the eye, the original OCT image being captured via an OCT device; the controller is configured to execute the one or more learning modules to generate an augmented OCT image based in part on the original OCT image; The system wherein the extended OCT image including a peripheral portion of the crystalline lens is constructed by extrapolating the original OCT image.

2. The system of claim 1 , wherein the peripheral portion is behind the iris of the eye and the extended OCT image allows visualization of one or more structures behind the iris.

3. The system of claim 1 , wherein the controller is configured to obtain at least one lens parameter based on the enhanced OCT image.

4. the controller is configured to select an intraocular lens based in part on the at least one lens parameter; The system of claim 3 , wherein the at least one lens parameter comprises a lens diameter and / or a lens capsule profile.

5. The system of claim 1 , wherein the OCT device comprises an array of laser beams for illuminating the eye.

6. The system of claim 1 , wherein the corresponding training OCT image is associated with the plurality of training ultrasound biomicroscopy images, and the corresponding training OCT image and the plurality of training ultrasound biomicroscopy images form a paired set.

7. the one or more learning modules include a generator trained to generate corresponding synthetic OCT images based in part on the corresponding training OCT images; the training network is a generative adversarial network with a discriminator; The system of claim 6 , wherein the discriminator is adapted to distinguish between the plurality of training ultrasound biomicroscopy images and the corresponding synthetic OCT images.

8. the corresponding training OCT image is not associated with the plurality of training ultrasound biomicroscopy images, and the corresponding training OCT image and the plurality of training ultrasound biomicroscopy images form an unpaired set; The system of claim 1 , wherein the training network is a generative adversarial network having a first discriminator and a second discriminator.

9. the one or more learning modules include a first generator and a second generator; 9. The system of claim 8, wherein the extended OCT image is generated by sequentially executing the first generator and the second generator, the first generator adapted to convert the original OCT image of the eye into a corresponding composite UBM image, and the second generator adapted to convert the corresponding composite UBM image into the extended OCT image.

10. the training network is configured to perform a positive training cycle using the first generator, the second generator, and the first discriminator; a first training OCT image is input to the first generator, and the first training OCT image is selected from the corresponding training OCT images; the first generator is adapted to convert the first training OCT image into a first synthetic ultrasound biomicroscopy image; The system of claim 9 , wherein the second generator is adapted to convert the first synthetic ultrasound biomicroscopy image into a second synthetic OCT image.

11. the first discriminator is adapted to distinguish between the first synthesized ultrasound biomicroscopy image and the plurality of training ultrasound biomicroscopy images during the positive training cycle; The system of claim 10 , wherein the training network incorporates a first loss function that minimizes a difference between the first training OCT image and the second synthetic OCT image.

12. the training network is configured to perform a reverse training cycle using the first generator, the second generator, and the second discriminator; a second training ultrasound biomicroscopy image is input to the second generator, the second training ultrasound biomicroscopy image being selected from the plurality of training ultrasound biomicroscopy images; the second generator is configured to convert the second training ultrasound biomicroscopy image into a third composite OCT image; The system of claim 9 , wherein the first generator is configured to convert the third composite OCT image into a fourth composite ultrasound biomicroscopy image.

13. the second discriminator is adapted to distinguish between the third composite OCT image and the corresponding training OCT image in the reverse training cycle; 13. The system of claim 12, wherein the training network incorporates a second loss function that minimizes the difference between the second training ultrasound biomicroscopy image and the fourth synthetic ultrasound biomicroscopy image.

14. 1. A method for enhancing an original optical coherence tomography ("OCT") image of an eye by a system comprising a controller having at least one processor and at least one non-transitory tangible memory, the method comprising: configuring the controller to selectively execute one or more learning modules; training one or more learning modules via a training network using a training dataset having a plurality of training ultrasound biomicroscopy images and corresponding training OCT images; capturing the original OCT image of the eye via an OCT device; generating an enhanced OCT image based in part on the original OCT image by executing the one or more training modules, wherein the enhanced OCT image includes a peripheral portion of a lens by extrapolating the original OCT image; A method comprising:

15. positioning the peripheral portion posterior to the iris of the eye such that the enhanced OCT image allows visualization of one or more structures posterior to the iris. The method of claim 14 further comprising:

Images