Noise reduction in ophthalmic images

Abstract

To provide a computer-implemented method of processing at least one image of a retina of an eye acquired by an ophthalmic imaging device, where the at least one image shows a texture of the retina.SOLUTION: The method comprises processing a first image of the at least one image using a noise reduction algorithm based on machine learning to generate a de-noised image of the retina, where the texture of the retina shown in the first image is at least partially removed by the noise reduction algorithm to generate the de-noised image. The method further comprises combining a second image of the at least one image with the de-noised image to generate at least one hybrid image of the retina which shows more of the texture of the retina than the de-noised image.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] Exemplary aspects relate generally to the field of noise reduction in ophthalmic images, and in particular to techniques for reducing noise in ophthalmic images based on machine learning. [Background technology]

[0002] Ophthalmic imaging devices use a variety of imaging technologies to image different parts of the eye, such as the retina, and are used by clinicians to diagnose and manage a variety of eye conditions. Ophthalmic imaging devices include (but are not limited to) autofluorescence (AF) ophthalmic imaging devices, scanning laser ophthalmoscopy (SLO), optical coherence tomography (OCT) imaging devices, fundus cameras, and microperimetry devices (among others), or a combination of two or more such devices.

[0003] Images acquired by such ophthalmic imaging devices are subject to noise sources (e.g., Gaussian, quantum or speckle noise) that reduce the signal-to-noise ratio (SNR) of the acquired image. This can reduce the clinical value of the acquired image, as important clinical information that may be useful in diagnosing various ocular pathologies may be obscured. Therefore, noise reduction algorithms are often used to improve the SNR of acquired ophthalmic images. Some modern noise reduction algorithms use machine learning algorithms, such as convolutional neural networks (CNNs), which have been demonstrated to efficiently perform noise reduction tasks. Summary of the Invention

[0004] According to a first exemplary aspect, a computer-implemented method is provided for processing at least one image of a retina of an eye acquired by an ophthalmic imaging device, the at least one image exhibiting retinal texture. The method includes processing a first image of the at least one image using a machine learning based noise reduction algorithm to generate a de-noised image of the retina, where the retinal texture exhibited in the first image is at least partially removed by the noise reduction algorithm to generate the de-noised image. The method further includes combining a second image of the at least one image with the de-noised image to generate at least one hybrid image of the retina exhibiting more retinal texture than the de-noised image.

[0005] At least one hybrid image of the retina may be generated by combining the second image with the denoised image using respective weightings for the second image and the denoised image. Additionally, at least one hybrid image of the retina may be generated by calculating one of a weighted sum or a weighted average of the second image and the denoised image using the weightings.

[0006] The computer-implemented method of the first exemplary aspect or any of its exemplary implementations described above may further include receiving a setting instruction from a user to set the weightings and setting the weightings using the setting instruction. Additionally or alternatively, the method may include generating a control signal for a display device to display the at least one hybrid image.

[0007] In some example implementations, a plurality of hybrid images are generated by combining the second image with the denoised image using different respective weightings, and a plurality of control signals are generated for a display device to display the plurality of hybrid images.

[0008] The computer-implemented method of the first exemplary aspect or any of its exemplary implementations described above may further include receiving an update instruction from a user to update the weightings, and updating the weightings using the update instruction.

[0009] In any of the foregoing, the noise reduction algorithm may be based on a convolutional neural network, and the at least one image of the retina of the eye may be at least one fundus autofluorescence image of the retina of the eye.

[0010] According to a second exemplary aspect, there is also provided a computer program comprising computer readable instructions which, when executed by a processor, cause the processor to perform a method according to the first exemplary aspect or any of its exemplary implementations described above. The computer program may be stored on a non-transitory computer readable storage medium (such as, for example, a computer hard disk or a CD) or may be conveyed by a computer readable signal.

[0011] According to a third exemplary aspect of the present specification, there is also provided a data processing device configured to perform a method according to the first exemplary aspect or any of its exemplary implementations described above. The data processing device may comprise at least one processor and at least one memory storing computer readable instructions that, when executed by the at least one processor, cause the at least one processor to perform a method according to the first exemplary aspect or any of its exemplary implementations described above.

[0012] Exemplary embodiments will now be described in detail, by way of non-limiting example only, with reference to the accompanying drawings, in which: like reference numbers appearing in different drawings indicate identical or functionally similar elements, unless otherwise indicated, and in which: [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a schematic diagram of a data processing apparatus 100 according to an exemplary embodiment. [Diagram 2] FIG. 2 is a schematic diagram of an exemplary implementation of the data processing device 100 of an exemplary embodiment in programmable signal processing hardware 200. [Diagram 3] FIG. 3 is a flow diagram illustrating a process by which the data processing device 100 processes at least one image, according to an exemplary embodiment. [Figure 4A] FIG. 4A is an enlarged version of image 10-1 shown in FIG. [Figure 4B] FIG. 4B is an enlarged version of the denoised image 110-1 shown in FIG. [Figure 5A] FIG. 5A is a hybrid image 50-1 generated using image 10-1, denoised image 110-1, and Equation 1 described herein with an alpha value of 0.1. [Figure 5B] FIG. 5B is a hybrid image 50-2 generated using image 10-1, denoised image 110-1, and Equation 1 described herein with an alpha value of 0.2. [Figure 5C] FIG. 5C is a hybrid image 50-3 generated using image 10-1, denoised image 110-1, and Equation 1 described herein using an alpha value of 0.3. [Figure 5D] FIG. 5D is a hybrid image 50-4 generated using image 10-1, denoised image 110-1, and Equation 1 described herein with an alpha value of 0.4. [Figure 5E] FIG. 5E is a hybrid image 50-5 generated using image 10-1, denoised image 110-1, and Equation 1 described herein using an alpha value of 0.5. [Figure 5F] FIG. 5F is a hybrid image 50-6 generated using image 10-1, denoised image 110-1, and Equation 1 described herein with an alpha value of 0.6. [Figure 5G] FIG. 5G is a hybrid image 50-7 generated using image 10-1, denoised image 110-1, and Equation 1 described herein using an alpha value of 0.7. [Figure 5H] FIG. 5H is a hybrid image 50-8 generated using image 10-1, denoised image 110-1, and Equation 1 described herein with an alpha value of 0.8. [Figure 5I] FIG. 5I is a hybrid image 50-9 generated using image 10-1, denoised image 110-1, and Equation 1 described herein with an alpha value of 0.9. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] The inventors have recognized that the use of conventional noise reduction algorithms such as those described above to reduce noise in retinal images tends to result in the loss of some or all of the retinal texture shown in the image. This appears to be caused by the noise reduction algorithm having difficulty distinguishing between noise and texture in the acquired image, which often have a similar appearance. This loss of texture can result in the loss of important clinical information from the acquired image, which may otherwise be useful in diagnosing various ocular pathologies, and can result in images that appear unrealistic to the clinician (e.g., they may appear "flat" to the clinician).

[0015] To address the above problems identified by the inventors, the inventors have devised a computer-implemented method of processing at least one image of a retina of an eye acquired by an ophthalmic imaging device, the at least one image exhibiting retinal texture. The method includes processing a first image of the at least one image using a machine learning based noise reduction algorithm to generate a denoised image of the retina, where the retinal texture exhibited in the first image is at least partially removed by the noise reduction algorithm to generate the denoised image. The method further includes combining a second image of the at least one image with the denoised image to generate at least one hybrid image of the retina exhibiting more retinal texture than the denoised image. This improvement in the amount of texture exhibited in the hybrid image relative to the amount of texture exhibited in the denoised image may be clinically significant, since information regarding retinal texture may be important clinical information useful in diagnosing various ocular pathologies. Furthermore, the hybrid image may appear more realistic to clinicians due to the increased amount of retinal texture exhibited in the hybrid image (e.g., the hybrid image may appear less "flat"). The drawbacks of the conventional noise reduction algorithms described above arise not only in processing images of the retina, but also in processing images of other parts of the eye, particularly the anterior segment, and the computer-implemented methods described herein are applicable to processing such images as well.

[0016] FIG. 1 is a schematic diagram of a data processing apparatus 100 configured to process at least one image 10 of a retina of an eye 20 acquired by an ophthalmic imaging device 30 .

[0017] The at least one image 10 may be at least one fundus autofluorescence (FAF) image (as shown, see also FIG. 4A ) acquired by a FAF ophthalmic imaging device as the ophthalmic imaging device 30, as in this exemplary embodiment. For example, the at least one FAF image may be at least one green (i.e., green laser) FAF image acquired by an Optos™ California or Silverstone imaging device, or at least one blue (i.e., blue laser) FAF image. However, the at least one image 10 is not limited thereto and may take other forms indicative of the retinal texture T of the eye 20. For example, the at least one image 10 may alternatively be at least one optical coherence tomography (OCT) image (e.g., B-scan) acquired by an OCT imaging device as the ophthalmic imaging device 30. As a further example, the at least one image 10 may be at least one reflectance image (e.g., a red, blue, or green reflectance image) of the retina acquired by a fundus camera, SLO, or other ophthalmic imaging device for acquiring reflectance images. The portion of the eye 20 imaged in the image 10 need not include the retina, but may instead include the anterior segment (AS) of the eye 20.

[0018] At least one image 10 of the retina of the eye 20 exhibits a retinal texture T. In this context, texture is an anatomical texture associated with the retina, which is indicative of an anatomical structure of the retina of the eye 20. For example, if the image acquired by the ophthalmic imaging device 30 is an FAF image, as in the present exemplary embodiment, the structure may be defined by a spatial distribution of fluorophores across the retina. As another example, if the image acquired by the ophthalmic imaging device 30 is an OCT image, the structure may include a physical structure of one or more layers of the retina. As a further example, if the image acquired by the ophthalmic imaging device 30 is a reflectance image of the retina, the structure may include the top surface of the retina, such that a texture is a physical texture of a surface that reflects the topography of the surface. The at least one image 10 processed by the data processing device 100 may be an image 10-1 acquired by the ophthalmic imaging device 30, as in the present exemplary embodiment, but the data processing device 100 may alternatively process multiple images 10-1, 10-2, ..., 10-n of the retina, as described below.

[0019] The data processing device 100 may be configured to process the image 10-1 using a machine learning based noise reduction algorithm 110 to generate a denoised image 110-1 of the retina, as in this exemplary embodiment and as described in more detail below. The data processing device 100 may be configured to receive the image 10-1 from the ophthalmic imaging device 30, but the data processing device 100 may alternatively be configured to acquire the image 10-1 by controlling the ophthalmic imaging device 30 to capture the image 10-1. For example, this may be the case when the functions of the data processing device 100 are performed by a processor of the ophthalmic imaging device 30 that controls the acquisition of images by the ophthalmic imaging device 30, or when the data processing device 100 is configured to control the functions of a processor of the ophthalmic imaging device 30 that controls the acquisition of images by the ophthalmic imaging device 30.

[0020] The data processing device 100 may further be configured, as in this exemplary embodiment, to combine the (single) image 10-1 with a denoised image 110-1 derived therefrom to generate a first hybrid image 40-1 (shown as a FAF image in FIG. 1 for this exemplary embodiment, see also FIG. 4B) that exhibits more texture T than the denoised image 110-1. However, the data processing device 100 may more generally generate at least one hybrid image 40, and thus a plurality of hybrid images 40-1, 40-2, ..., 40-n, as will be explained below. The data processing device 100 may use weights w1 and w2, as indicated by reference 120 in FIG. 1, when combining the image 10-1 with the denoised image 110-1 to generate a first hybrid image 40-1 that exhibits more texture T than the denoised image 110-1, as will be explained below.

[0021] The data processing device 100, as in this exemplary embodiment, generates a control signal S for the display device 50 to display at least one hybrid image 40. C1 , ..., S Cn The display device 50 may be, for example, part of the ophthalmic imaging device 30 or may be a screen of an external computer.

[0022] The data processing device 100, as in this exemplary embodiment, receives an instruction I from a user of the data processing device 100 to set and / or update the weightings 120, as described below. u , I s The signal may be further configured to receive at least one of:

[0023] The data processing apparatus 100 may be provided in any suitable form, for example as a processor 220 of programmable signal processing hardware 200 of the type shown diagrammatically in Figure 2. The components of the programmable signal processing hardware 200 may be included in the data processing apparatus 100. The programmable signal processing apparatus 200 receives (in some exemplary embodiments) at least one image 10, if provided, in accordance with the instructions I mentioned above. u , I s and outputs at least one hybrid image 40, if provided, in accordance with the above-mentioned control signal S C1 , ..., S Cn The signal processing hardware 200 further comprises a processor 220 (e.g., a CPU, which is a central processing unit, and / or a GPU, which is a graphics processing unit), a working memory 230 (e.g., a random access memory), and an instruction store 240 that stores a computer program 245 including computer readable instructions that, when executed by the processor 220, cause the processor 220 to perform various functions of the data processing device 100 described herein.

[0024] The working memory 230 stores information used by the processor 220 during the execution of the computer program 245, including the noise reduction algorithm 110 and the weights w1 and w2 optionally used. The instruction store 240 may comprise a ROM (e.g. in the form of an Electrically Erasable Programmable Read Only Memory (EEPROM) or Flash memory) preloaded with computer readable instructions. Alternatively, the instruction store 240 may comprise a RAM or similar type of memory, and the computer readable instructions of the computer program 245 may be input from a computer program product, such as a non-transitory computer readable storage medium 250 in the form of a CD-ROM, DVD-ROM, etc., or a computer readable signal 260 carrying computer readable instructions. In any case, the computer program 245, when executed by the processor 220, causes the processor 220 to perform the functions of the data processing apparatus 100 described herein. In other words, the data processing device 100 of this exemplary embodiment may include a computer processor 220 and a memory 240 that stores computer-readable instructions that, when executed by the computer processor 220, cause the computer processor 220 to perform the functions of the data processing device 100 described in this specification.

[0025] It should be noted, however, that the data processing apparatus 100 may alternatively be implemented with non-programmable hardware, such as an ASIC, FPGA, or other integrated circuit dedicated to performing the functions of the data processing apparatus 100 described herein, or a combination of such non-programmable hardware and programmable hardware as described above with reference to Figure 2. Furthermore, in some exemplary embodiments, the programmable signal processing hardware 200 may further perform the functions of the ophthalmic imaging device 30 and may control the display device 50.

[0026] Figure 3 is a flow diagram illustrating the process by which data processing device 100 processes image 10-1. Although the processes in Figure 3 are shown in a specified order, one skilled in the art will appreciate that the processes in Figure 3 are not limited to this order and one or more of the processes may be performed in parallel (e.g., processes S30 and S40 may be performed before or in parallel with process S10). Processes in the flow diagram represented by dashed boxes are optional and may be omitted as described below.

[0027] In process S10 of Fig. 3, the data processing device 100 processes the image 10-1 using a noise reduction algorithm 110 based on machine learning to generate a denoised image 110-1. The noise in the image 10-1 is at least partially removed by the noise reduction algorithm 110 to generate the denoised image 110-1. However, the retinal texture T shown in the image 10-1 is also at least partially removed by the noise reduction algorithm 110 in this process. This loss of the retinal texture T in the denoised image 110-1 is undesirable as previously mentioned.

[0028] The noise reduction algorithm 110 may be based on a convolutional neural network (CNN), as in this exemplary embodiment. For example, the noise reduction algorithm 110 may be a denoising autoencoder, a generative adversarial network (GAN), or any of the denoising CNNs described in Ilesanmi, AE, Ilesanmi, TO, "Methods for image denoising using convolutional neural network: a review.", Complex Intell. Syst. 7, p. 2179-2198 (2021), which is incorporated herein by reference in its entirety. The noise reduction algorithm 110 may be the U-NET CNN, as described in Ronneberger, O., Fischer, P., and Brox, T., 2015, “U-net: Convolutional networks for biomedical image segmentation,” In Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18 (pp. 234-241), Springer International Publishing, the contents of which are incorporated herein by reference in their entirety, or Lehtinen, J., Munkberg, J., Hasselgren, J., Laine, S., Karras, T., Aittala, M., and Aila, T., 2018, “Noise2Noise: Learning image restoration without clean data,” arXiv preprint It may be the U-NET CNN described in arXiv:1803.04189. However, the form of the noise reduction algorithm 110 is not so limited and may alternatively be based on other machine learning algorithms.For example, the noise reduction algorithm 110 may alternatively be a Bayesian image denoising algorithm such as that described in Kataoka, S., Yasuda, M., "Bayesian Image Denoising with Multiple Noisy Images.", Rev Socionetwork Strat 13, p. 267-280 (2019), the contents of which are incorporated herein by reference in their entirety. The noise reduction algorithm 110 may be pre-trained before being stored in the memory of the data processing device 100, or may be trained by the data processing device 100 upon receiving a training dataset for training the noise reduction algorithm 110.

[0029] Figures 4A and 4B are zoomed-in versions of image 10-1 and denoised image 110-1, respectively, shown in Figure 1. As can be seen, the noise visible in image 10-1 has been reduced in denoised image 110-1 by noise reduction algorithm 110 (in this case based on the U-NET CNN similar to that described above).

[0030] 3, in process S20, the data processing device 100 combines the image 10-1 with the denoised image 110-1 to generate a hybrid image 40-1 that exhibits more texture T than the denoised image 110-1. That is, by combining the image 10-1 with the denoised image 110-1 to generate the hybrid image 40-1 (which thus exhibits more retinal texture T than the denoised image 110-1), the retinal texture T shown in the image 10-1 is added to the denoised image 110-1. The hybrid image 40-1 can be generated by combining the image 10-1 with the denoised image 110-1 using the weighting 120 as in the first exemplary embodiment. For example, the combination of the image 10-1 and the denoised image 110-1 using the weighting 120 may be a weighted sum or weighted average of the image 10-1 and the denoised image 110-1. However, the hybrid image 40-1 may alternatively be synthesized without weighting 120 by simply averaging or summing the image 10-1 and the denoised image 110-1 (i.e. by performing an unweighted average or sum). That is, the data processing device 100 may calculate the average or sum of the image 10-1 and the denoised image 110-1 such that the respective pixel value of each pixel of the resulting averaged image (or the sum image as the case may be) is the average (or the sum as the case may be) of the pixel values ​​of the pixels at corresponding positions in the image 10-1 and the denoised image 110-1 that correspond to respective common positions on the retina.

[0031] The amount of texture in the hybrid image 40-1 and the denoised image 110-1 can be quantified using algorithms such as those described in the paper by R. Bergman et al., "Detection of Textured Areas in Images Using a Disorganization Indicator Based on Component Counts," J. Electronic Imaging. 17.043003 (2008), the contents of which are incorporated herein by reference in their entirety. The texture detector presented in this paper is based on the intuition that texture in natural images is "messy." The measure used to detect texture examines the structure of local regions of the image. This structural approach allows for the detection of both structured and unstructured textures at many scales. It also distinguishes between edges and texture, and between texture and noise. The automated detection results are shown in the paper to match human classifications of the corresponding image regions. The amount of texture in the hybrid image 40-1 and the denoised image 110-1 can be compared by comparing the areas of these images designated by the algorithm as "texture." Because noise and texture may have a similar appearance, an indication of the amount of texture in the hybrid image 40-1 and the denoised image 110-1 may be obtained using a measure of SNR or a structural similarity index measure (SSIM).

[0032] If the combination of image 10-1 and denoised image 110-1 using weighting 120 is a weighted average, then the weighted average can be expressed as:

[0033]

number

[0034] where X' is hybrid image 40-1, X is image 10-1, N(X) is denoised image 110-1 generated by noise reduction algorithm 110, and α is a predetermined constant between 0 and 1. Note that when α is 1, hybrid image 40-1 is the same as denoised image 110-1, and when α is 0, hybrid image 40-1 is the same as image 10-1. The weights in this case are (1-α) and α, and the sum of the weights is equal to 1.

[0035] 5A-5I show image 10-1 of FIG. 4A, denoised image 110-1 of FIG. 4B, U-NET CNN based noise reduction algorithm 110, and hybrid images 50-1, 50-2, ..., 50-9 generated using Equation 1 above with alpha values ​​of 0.1, 0.2, ..., 0.9, respectively. As shown, as the value of alpha increases, more retinal texture T is gradually added to denoised image 110-1, although some noise may also be added.

[0036] The weightings 120 may be set at the time of manufacture, entered manually, downloaded from an external server, or determined by a processor of the ophthalmic imaging device 30 to provide the best compromise for clinical purposes between noise and texture in the hybrid image 40-1, taking into account the capabilities of the ophthalmic imaging device 30. However, the weightings 120 may alternatively be set by a user (e.g., a clinician) of the data processing apparatus 100 via optional processes S30 and S40 preceding process S20 of Figure 3, as described below.

[0037] In optional process S30 of FIG. 3, the data processing device 100 receives setting instructions I s For example, a setting instruction I s may include user-selected weights 120 or may be values ​​used to set weights 120 (eg, the value of α in Equation 1 above).

[0038] In optional process S40 of FIG. 3, the data processing device 100 receives a setting instruction I s For example, the setting instruction I s If the setting instruction I includes user-selected weights 120, the data processing device 100 sets the weights 120 to these user-selected weights. s is the value used to set the weighting 120, then the data processing device 100 uses that value to set the weighting 120 (e.g., the data processing device 100 uses the received value of α to set the weighting in Equation 1).

[0039] Once the hybrid image 40-1 is generated by the data processing device 100 in process S20 of Figure 3 as described above, it may be stored in a memory of the data processing device 100 or may be transmitted to an external storage device. In optional process S50 of Figure 3, the data processing device 100 generates a control signal S for the display device 50 to display the hybrid image 40-1. C1 For example, the control signal S C1 may include hybrid image 40-1 and an instruction that hybrid image 40-1 should be displayed by display device 50, or may include an instruction that display device 50 should retrieve hybrid image 40-1 from a memory that stores hybrid image 40-1 and display hybrid image 40-1.

[0040] In optional process S60 of FIG. 3, the data processing apparatus 100 generates update instructions I for updating the weights 120 in a similar manner as described above in process S30 of FIG. ufrom a user. For example, the user may view hybrid image 40-1 displayed by display device 50 and determine that the ratio between weightings 120 should be increased or decreased (or may determine that the ratio between weightings 120 does not need to be changed, in which case optional process S60 is not performed).

[0041] Update instructions I u In optional process S70 of FIG. 3, where update instruction I is received by data processing device 100, data processing device 100 generates an update instruction I in a manner similar to that described above in process S40. u Update the weights 120 using

[0042] After the optional processes S60 and S70 of FIG. 3, the data processing apparatus 100 may again execute the process S20 in the same manner as described above, except for the weights 120 updated by the above-mentioned process S70, and may then execute the above-mentioned process S50 to generate the control signal S for the display device 50 to display the hybrid image so generated. C2 These processes may be repeated as often as necessary to allow a user of the data processing device 100 to adjust the weightings 120, thus allowing the user to adjust the weightings 120 for the image at hand 10-1 to provide the best compromise for clinical purposes between noise and texture.

[0043] In an alternative exemplary embodiment, in process S20 of FIG. 3, the plurality of hybrid images 40-1, 40-2, ..., 40-n may instead be generated by the data processing device 100 by combining the image 10-1 and the denoised image 110-1 using different respective weightings for each of the hybrid images 40-1, 40-2, ..., 40-n. For example, the plurality of hybrid images 40-1, 40-2, ..., 40-n may be the hybrid images 50-1, 50-2, ..., 50-9 of FIG. 5A-FIG. 5I, which are generated using different pairs of weightings resulting from different respective values ​​of α in Equation 1. A plurality of control signals S are then generated for the display device 50 to display the plurality of hybrid images 40-1, 40-2, ..., 40-n in a manner similar to that described above in connection with process S50 of FIG. 3. C1 , ..., S Cn can be generated by the data processing device 100. Thus, continuing with the previous example, a number of control signals S C1 , ..., S Cn can cause display device 50 to display each of hybrid images 50-1, 50-2, . . . , 50-9 of FIGS.

[0044] The user can compare the displayed hybrid images 40-1, 40-2, ..., 40-n and select the hybrid image of the plurality of hybrid images 40-1, 40-2, ..., 40-n that is determined to offer the best compromise for clinical purposes between noise and texture. This selection can be used by the data processing device 100 to update the weightings 120 (to the weightings used to generate the selected hybrid image) in the same manner as described above in process S70 of Figure 3, or by sending an update instruction I as described above in process S60. u3 in the same manner as described above, except for the now updated weightings, and can perform process S50 as described above to generate further control signals for display device 50 to display the new hybrid image so generated.

[0045] In summary, a computer-implemented method has been described above for processing an image of a portion of an eye 20 (e.g., the retina or anterior segment) acquired by an ophthalmic imaging device 30 (such as an ophthalmic FAF or OCT imaging device), where the image exhibits a texture T of the portion, the method including processing the image using a machine learning based noise reduction algorithm 110 to generate a de-noised image 110-1 of the portion, where the texture T of the portion exhibited in the image is at least partially removed by the noise reduction algorithm 110 to generate the de-noised image 110-1, and combining the image with the de-noised image 110-1 to generate at least one hybrid image 40 of the portion that exhibits more of the texture T of the portion than the de-noised image 110-1.

[0046] The data processing device 100 combines the image 10-1 with a denoised image 110-1 derived from the image 10-1 in process S20 of Fig. 3 to generate a hybrid image 40-1, which shows more retinal texture T than the denoised image 110-1, although this is not necessarily the case. In alternative exemplary embodiments, the denoised image may be combined with an image of the retina other than the image from which the denoised image is derived. In particular, the data processing device 100 may process a plurality of images 10-1, 10-2, ..., 10-n of the retina of the eye 20 acquired by the ophthalmic imaging device 30, including a first image 10-1 of the retina of the eye 20 and a second image 10-2 of the retina of the eye 20. These images 10-1, 10-2, ..., 10-n may be, for example, a sequence of repeated images of the retina of the eye 20, which may be acquired at different (e.g. closely spaced, consecutive) respective times. Since both the first image 10-1 and the second image 10-2 show the texture T of the eye 20, after processing the first image 10-1 to generate the denoised image 110-1, the data processing device 100 can instead combine the second image 10-2 (which may be next to the first image 10-1 in the sequence of images or may be another image in a different sequence from the first image 10-1) with the denoised image 110-1 to generate a hybrid image 40-1 showing more of the retinal texture T than the denoised image 110-1. That is, by combining the second image 10-2 with the denoised image 110-1 to generate the hybrid image 40-1, the retinal texture T shown in the second image 10-2 is added to the denoised image 110-1 (e.g. due to the second image 10-2 showing more of the retinal texture T than the denoised image 110-1 and the weighted averaging performed, as described above).

[0047] Accordingly, an alternative exemplary embodiment provides a computer-implemented method of processing two or more images 10-1, 10-2 of a portion of an eye 20 (e.g., the retina or anterior segment) acquired by an ophthalmic imaging device 30 (such as an ophthalmic FAF or OCT imaging device), each of the images exhibiting a texture T of the portion, the method including processing a first image 10-1 of the two or more images using a machine learning based noise reduction algorithm 110 to generate a de-noised image 110-1 of the portion, wherein the texture T of the portion exhibited in the first image 10-1 is at least partially removed by the noise reduction algorithm 110 to generate the de-noised image 110-1, and combining a second image 10-2 of the two or more images (the second image different from the first image 10-1) with the de-noised image 110-1 to generate at least one hybrid image 40 of the portion exhibiting more of the texture T of the portion than the de-noised image 110-1.

[0048] More generally, according to an exemplary embodiment, there is provided a computer-implemented method of processing at least one image 10 of a portion of an eye 20 (e.g., retina or anterior segment) acquired by an ophthalmic imaging device 30 (such as an ophthalmic FAF or OCT imaging device), the at least one image 10 exhibiting a texture T of the portion, the method including processing a first image 10-1 of the at least one image 10 using a machine learning based noise reduction algorithm 110 to generate a de-noised image 110-1 of the portion, wherein the texture T of the portion exhibited in the first image 10-1 is at least partially removed by the noise reduction algorithm 110 to generate the de-noised image 110-1, and combining a second image of the at least one image 10 with the de-noised image 110-1 to generate at least one hybrid image 40 of the portion exhibiting more of the texture T of the portion than the de-noised image 110-1.

[0049] In the preceding description, exemplary aspects have been described with reference to several exemplary embodiments. This specification should therefore be considered illustrative rather than restrictive. Similarly, the diagrams shown in the drawings that highlight the functionality and advantages of the exemplary embodiments are presented for illustrative purposes only. The architecture of the exemplary embodiments is sufficiently flexible and configurable so that it can be utilized in ways other than those shown in the accompanying figures.

[0050] Some aspects of the examples presented herein, such as the functionality of the data processing apparatus 100, may be provided as computer programs, or software, e.g., one or more programs having instructions or sequences of instructions contained or stored in an article of manufacture, such as a machine-accessible or machine-readable medium, instruction store, or computer-readable storage device, each of which may be non-transitory in one exemplary embodiment. The programs or instructions of the non-transitory machine-accessible medium, machine-readable medium, instruction store, or computer-readable storage device may be used to program a computer system or other electronic device. Machine or computer-readable media, instruction stores, and storage devices may include, but are not limited to, floppy diskettes, optical disks, and magneto-optical disks, or other types of media / machine-readable media / instruction stores / storage devices suitable for storing or transmitting electronic instructions. The techniques described herein are not limited to any particular software configuration. They may find applicability in any computing or processing environment. As used herein, the terms "computer-readable," "machine-accessible medium," "machine-readable medium," "instruction store," and "computer-readable storage device" are intended to include any medium that can store, encode, or transmit instructions or sequences of instructions for execution by a machine, computer, or computer processor, causing the machine / computer / computer processor to perform any one of the methods described herein. Furthermore, it is common in the art to speak of software in some form or another (e.g., program, procedure, process, application, module, unit, logic, etc.) as taking an action or causing a result. Such expressions are merely a shorthand way of stating that execution of the software by a processing system causes the processor to perform operations to produce a result.

[0051] Some or all of the functionality of data processing apparatus 100 may also be implemented by the preparation of application specific integrated circuits, field programmable gate arrays, or by interconnecting an appropriate network of conventional component circuits.

[0052] The computer program product may be provided in the form of one or more storage media, instruction stores, or storage devices having stored thereon instructions that can be used to control a computer or computer processor or to cause a computer or computer processor to perform any of the procedures of the exemplary embodiments described herein. The storage media / instruction stores / storage devices may include, by way of example and not limitation, optical disks, ROM, RAM, EPROM, EEPROM, DRAM, VRAM, flash memory, flash cards, magnetic cards, optical cards, nanosystems, molecular memory integrated circuits, RAID, remote data storage / archive / warehousing, and / or any other type of device suitable for storing instructions and / or data.

[0053] When stored on any one of one or more computer readable media, instruction stores, or storage devices, some implementations include software for controlling both the hardware and the system and for enabling the system or microprocessor to utilize the results of the exemplary embodiments described herein to interact with a human user or other mechanisms. Such software may include, but is not limited to, device drivers, operating systems, and user applications. Finally, such computer readable media or storage devices further include software for performing the exemplary aspects of the present invention, as described above.

[0054] The programming and / or software of the system includes software modules for performing the procedures described herein, in some exemplary embodiments, the modules include software, while in other exemplary embodiments, the modules include hardware or a combination of hardware and software.

[0055] While various exemplary embodiments of the present invention have been described above, it should be understood that they are presented by way of example and not limitation. Various changes in form and details can be made by those skilled in the art. Thus, the present invention should not be limited by any of the exemplary embodiments described above, but should be defined only in accordance with the following claims and their equivalents.

[0056] Moreover, the purpose of the Abstract is to enable the Patent Office and the public, particularly scientists, engineers and practitioners in the art who are not familiar with patent or legal terms or phrases, to quickly determine the nature and substance of the technical disclosure of the present application from a cursory review. The Abstract is not intended to be limiting in any way with respect to the scope of the exemplary embodiments presented herein. It should also be understood that the steps recited in the claims need not be performed in the order presented.

[0057] Although this specification contains many specific embodiment details, these should not be construed as limitations on the scope of any invention or what may be claimed, but rather as descriptions of features specific to the specific embodiments described herein. Certain features described herein in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may be described above as acting in a particular combination and may initially be claimed as such, one or more features from a claimed combination may, in some cases, be deleted from the combination, and the claimed combination may be directed to a subcombination or a variation of the subcombination.

[0058] In certain circumstances, multitasking and parallel processing may be advantageous. Furthermore, the separation of various components in the above-described embodiments should not be understood to require such separation in all embodiments, and it should be understood that the program components and systems described may generally be integrated together in a single software product or packaged into multiple software products.

[0059] Having now described several exemplary embodiments and implementations, it is clear that the above are presented by way of example and not by way of limitation. In particular, many of the examples presented herein include specific combinations of device or software elements, but those elements may be combined in other ways to achieve the same purpose. Operations, elements, and features discussed only in relation to one embodiment are not intended to be excluded from a similar role in other embodiments or implementations.

Claims

1. 1. A computer-implemented method for processing at least one image (10) of a retina of an eye (20) acquired by an ophthalmic imaging device (30), said at least one image (10) exhibiting a texture (T) of said retina, said computer-implemented method comprising: processing (S10) a first image (10-1) of the at least one image (10) using a machine learning based noise reduction algorithm (110) to generate a de-noised image (110-1) of the retina, wherein the texture (T) of the retina shown in the first image (10-1) is at least partially removed by the noise reduction algorithm (110) to generate the de-noised image (110-1); combining (S20) a second image (10-1, 10-2) of the at least one image (10) with the denoised image (110-1) to generate at least one hybrid image (40) of the retina that shows more of the texture (T) of the retina than the denoised image (110-1); 23. A computer-implemented method comprising:

2. 2. The computer-implemented method of claim 1, wherein the at least one hybrid image (40) of the retina is generated by combining the second image (10-1, 10-2) with the denoised image (110-1) using respective weights (120) for the second image (10-1, 10-2) and the denoised image (110-1).

3. 3. The computer-implemented method of claim 2, wherein the at least one hybrid image (40) of the retina is generated by calculating one of a weighted sum or a weighted average of the second image (10-1, 10-2) and the denoised image (110-1) using the weighting (120).

4. A setting instruction (I) for setting the weighting (120) ｓ ) from the user (S30); and ｓ 4. The computer-implemented method of claim 2 or 3, further comprising: setting (S40) the weightings (120) using a weighting factor (120).

5. A control signal (S) for a display device (50) to display said at least one hybrid image (40). Ｃ１ The computer-implemented method of any one of claims 2 to 4, further comprising generating (S50) a

6. A plurality of hybrid images (40-1, 40-2, . . . , 40-n) are generated by combining the second image (10-1, 10-2) with the denoised image (110-1) using different respective weightings (120), and a plurality of control signals (S Ｃ１ , . . . , S Ｃｎ 6. The computer-implemented method of claim 5, wherein a plurality of hybrid images (40-1, 40-2, . . . , 40-n) are generated for displaying said plurality of hybrid images (40-1, 40-2, . . . , 40-n) on said display device (50).

7. In order to update the weighting (120), an update instruction (I) is given from the user. ｕ ) (S60), and the update instruction (I ｕ 7. The computer-implemented method of claim 5 or 6, further comprising updating (S70) the weightings (120) using the weights.

8. The computer-implemented method of any one of claims 1 to 7, wherein the noise reduction algorithm (110) is based on a convolutional neural network.

9. The computer-implemented method of any one of claims 1 to 8, wherein the at least one image (10) of the retina of the eye (20) is at least one fundus autofluorescence image of the retina of the eye (20).

10. The computer-implemented method of any one of claims 1 to 9, wherein the second image (10-1, 10-2) is the same as the first image (10-1).

11. When executed by a processor (220), the processor (220) A computer program (245) comprising computer readable instructions for causing the computer to carry out the method according to any one of claims 1 to 10.

12. A data processing apparatus (100) configured to perform the method according to any one of the preceding claims.

Images