Noise reduction in ophthalmic images

Abstract

To provide a computer-implemented method of processing at least one image of a retina of an eye acquired by an ophthalmic imaging device, where the at least one image shows a texture of the retina.SOLUTION: The method comprises processing a first image of the at least one image using a noise reduction algorithm based on machine learning to generate a de-noised image of the retina, where the texture of the retina shown in the first image is at least partially removed by the noise reduction algorithm to generate the de-noised image. The method further comprises combining a second image of the at least one image with the de-noised image to generate at least one hybrid image of the retina which shows more of the texture of the retina than the de-noised image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] Exemplary embodiments generally relate to the field of noise reduction in ophthalmic images, and more particularly to techniques for reducing noise in ophthalmic images based on machine learning. [Background technology]

[0002] Ophthalmic imaging devices employ various imaging techniques to image different parts of the eye, such as the retina, and are used by clinicians to diagnose and manage various eye conditions. Examples of ophthalmic imaging devices include, but are not limited to, autofluorescence (AF) ophthalmic imaging devices, scanning laser ophthalmoscopy (SLO), optical coherence tomography (OCT) imaging devices, fundus cameras and microfield analyzers (among others), or combinations of two or more such devices.

[0003] Images acquired by such ophthalmic imaging devices are affected by noise sources (e.g., Gaussian, quantum, or speckle noise) that reduce the signal-to-noise ratio (SNR) of the acquired image. This can reduce the clinical value of the acquired image, as important clinical information that may be useful in diagnosing various eye conditions may be obscured. Therefore, noise reduction algorithms are often used to improve the SNR of acquired ophthalmic images. Some modern noise reduction algorithms use machine learning algorithms, such as convolutional neural networks (CNNs), which have been demonstrated to perform denoising tasks efficiently. [Overview of the project]

[0004] According to a first exemplary embodiment, a computer implementation method is provided for processing at least one image of the retina of an eye acquired by an ophthalmic imaging device, wherein at least one image shows the texture of the retina. The method comprises processing a first image of at least one image using a machine learning-based noise reduction algorithm to generate a denoised image of the retina, wherein the retinal texture shown in the first image is at least partially removed by the noise reduction algorithm to generate the denoised image. The method further comprises combining a second image of at least one image with the denoised image to generate at least one hybrid image of the retina showing more retinal texture than the denoised image.

[0005] At least one hybrid image of the retina may be generated by compositing the second image with the denoising image using the respective weights for the second image and the denoising image. Furthermore, at least one hybrid image of the retina may be generated by calculating either a weighted sum or a weighted average of the second image and the denoising image using the weights.

[0006] A computer implementation of the first exemplary embodiment or any of the exemplary implementations described above may further include receiving a setting instruction from a user for setting weights, and setting weights using the setting instruction. Additionally or alternatively, the method may include generating a control signal for a display device to display at least one hybrid image.

[0007] In some exemplary implementations, multiple hybrid images are generated by compositing a second image with a denoising image using different weights, and multiple control signals are generated for the display device to display the multiple hybrid images.

[0008] A computer implementation of the first exemplary embodiment or any of the above exemplary implementations may further include receiving an update instruction from a user to update the weights, and updating the weights using the update instruction.

[0009] In any of the above, the noise reduction algorithm may be based on a convolutional neural network, and at least one image of the retina of the eye may be at least one fundus autofluorescence image of the retina of the eye.

[0010] According to a second exemplary embodiment, a computer program is also provided which, when executed by a processor, includes computer-readable instructions causing the processor to perform a method according to the first exemplary embodiment or the exemplary implementation described above. The computer program may be stored on a non-temporary computer-readable storage medium (e.g., a computer hard disk or CD) or carried by computer-readable signals.

[0011] A third exemplary aspect of this specification also provides a data processing device configured to perform the method according to the first exemplary aspect or the exemplary implementation described above. The data processing device may include at least one processor and at least one memory for storing computer-readable instructions that, when executed by the at least one processor, cause the at least one processor to perform the method according to the first exemplary aspect or the exemplary implementation described above.

[0012] Herein, illustrative embodiments will be described in detail, only as non-limiting examples, with reference to the accompanying drawings described below. Similar reference numerals appearing in different figures indicate identical or functionally similar elements unless otherwise indicated. [Brief explanation of the drawing]

[0013] [Figure 1] Figure 1 is a schematic diagram of a data processing device 100 according to an exemplary embodiment. [Figure 2] Figure 2 is a schematic diagram of an exemplary implementation form of a data processing device 100 in an exemplary embodiment in programmable signal processing hardware 200. [Figure 3] Figure 3 is a flowchart showing a process in which a data processing device 100 processes at least one image according to an exemplary embodiment. [Figure 4A] Figure 4A is an enlarged version of the image 10-1 shown in FIG. 1. [Figure 4B] Figure 4B is an enlarged version of the noise removal image 110-1 shown in FIG. 1. [Figure 5A] Figure 5A is a hybrid image 50-1 generated using Equation 1 described herein with image 10-1, noise removal image 110-1, and an alpha value of 0.1. [Figure 5B] Figure 5B is a hybrid image �0-2 generated using Equation 1 described herein with image 10-1, noise removal image 110-1, and an alpha value of 0.2. [Figure 5C] Figure 5C is a hybrid image 50-3 generated using Equation 1 described herein with image 10-1, noise removal image 110-1, and an alpha value of 0.3. [Figure 5D] Figure 5D is a hybrid image 50-4 generated using Equation 1 described herein with image 10-1, noise removal image 110-1, and an alpha value of 0.4. [Figure 5E] Figure 5E is a hybrid image 50-5 generated using Equation 1 described herein with image 10-1, noise removal image 110-1, and an alpha value of 0.5. [Figure 5F] Figure 5F is a hybrid image 50-6 generated using Equation 1 described herein with image 10-1, noise removal image 110-1, and an alpha value of 0.6. [Figure 5G] Figure 5G is a hybrid image 50-7 generated using Equation 1 described herein with image 10-1, noise removal image 110-1, and an alpha value of 0.7. [Figure 5H] FIG. 5H is a hybrid image 50-8 generated using Equation 1 described herein with image 10-1, noise-reduced image 110-1, and an alpha value of 0.8. [Figure 5I] FIG. 5I is a hybrid image 50-9 generated using Equation 1 described herein with image 10-1, noise-reduced image 110-1, and an alpha value of 0.9.

BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The inventors recognized that using a conventional noise reduction algorithm as described above to reduce noise in a retinal image tends to lose some or all of the texture of the retina shown in the image. This appears to be caused by noise reduction algorithms that make it difficult to distinguish between the noise and texture of the acquired image, which often have a similar appearance. This loss of texture can result in the loss of important clinical information from the acquired image, which could otherwise be useful for diagnosing various eye conditions and could result in an image that appears unrealistic to a clinician (e.g., they may appear "flat" to a clinician).

[0015] To address the aforementioned problems recognized by the inventors, the inventors have devised a computer implementation method for processing at least one image of the retina of an eye acquired by an ophthalmic imaging device, wherein at least one image shows the texture of the retina. The method comprises processing a first image of at least one image using a machine learning-based noise reduction algorithm to generate a denoised image of the retina, wherein the retinal texture shown in the first image is at least partially removed by the noise reduction algorithm to generate the denoised image. The method further comprises combining a second image of at least one image with the denoised image to generate at least one hybrid image of the retina showing more retinal texture than the denoised image. Since information about retinal texture can be important clinical information useful for diagnosing various lesions of the eye, this improvement in the amount of texture shown in the hybrid image compared to the amount of texture shown in the denoised image may be clinically significant. Furthermore, the hybrid image may appear more realistic to clinicians because of the increased amount of retinal texture shown in the hybrid image (for example, the hybrid image may not appear "flat"). The drawbacks of the conventional noise reduction algorithms described above occur not only in the processing of retinal images but also in the processing of images of other parts of the eye, particularly the anterior segment. The computer implementation methods described herein are applicable to the processing of such images as well.

[0016] Figure 1 is a schematic diagram of a data processing device 100 configured to process at least one image 10 of the retina of an eye 20 acquired by an ophthalmic imaging device 30.

[0017] At least one image 10 may be at least one fundus autofluorescence (FAF) image acquired by an FAF ophthalmic imaging device as an ophthalmic imaging device 30 (see also Figure 4A, as shown), as in this exemplary embodiment. For example, at least one FAF image may be at least one green (i.e., green laser) FAF image acquired by an Optos® California or Silverstone imaging device, or at least one blue (i.e., blue laser) FAF image. However, at least one image 10 is not limited to this and may take other forms showing the texture T of the retina of the eye 20. For example, at least one image 10 may alternatively be at least one optical coherence tomography (OCT) image (e.g., B-scan) acquired by an OCT imaging device as an ophthalmic imaging device 30. As a further example, at least one image 10 may be at least one reflectance image of the retina (e.g., a red, blue, or green reflectance image) acquired by a fundus camera, SLO, or other ophthalmic imaging device for acquiring reflectance images. The portion of the eye 20 captured in image 10 does not need to include the retina, but may instead include the anterior segment (AS) of the eye 20.

[0018] At least one image 10 of the retina of eye 20 shows the texture T of the retina. In this context, the texture is an anatomical texture related to the retina that shows the anatomical structure of the retina of eye 20. For example, if the image acquired by the ophthalmic imaging device 30 is an FAF image, the structure may be defined by the spatial distribution of phosphors across the retina, as in this exemplary embodiment. As another example, if the image acquired by the ophthalmic imaging device 30 is an OCT image, the structure may include the physical structure of one or more layers of the retina. As a further example, if the image acquired by the ophthalmic imaging device 30 is a reflectance image of the retina, the structure may include the upper surface of the retina, such that the texture is the physical texture of the surface that reflects the topography of the surface. The at least one image 10 processed by the data processing device 100 may be image 10-1 acquired by the ophthalmic imaging device 30, as in this exemplary embodiment, but the data processing device 100 may alternatively process a plurality of images 10-1, 10-2, ..., 10-n of the retina, as described below.

[0019] The data processing device 100 may be configured to process image 10-1 using a machine learning-based noise reduction algorithm 110 to generate a denoised retinal image 110-1, as described in more detail below, as in this exemplary embodiment. The data processing device 100 may be configured to receive image 10-1 from the ophthalmic imaging device 30, but alternatively, the data processing device 100 may be configured to acquire image 10-1 by controlling the ophthalmic imaging device 30 to capture image 10-1. For example, this may be done by the processor of the ophthalmic imaging device 30 controlling the acquisition of images by the ophthalmic imaging device 30, or the data processing device 100 may be configured to control the functions of the processor of the ophthalmic imaging device 30 controlling the acquisition of images by the ophthalmic imaging device 30.

[0020] The data processing device 100 may be further configured, as in this exemplary embodiment, to synthesize a (single) image 10-1 with a denoising image 110-1 derived therefrom to generate a first hybrid image 40-1 exhibiting more texture T than the denoising image 110-1 (see also Figure 4B, as shown in Figure 1 as an FAF image for this exemplary embodiment). However, more generally, the data processing device 100 can generate at least one hybrid image 40, and thus can generate a plurality of hybrid images 40-1, 40-2, ..., 40-n, as described below. When the data processing device 100 synthesizes image 10-1 with a denoising image 110-1 to generate a first hybrid image 40-1 exhibiting more texture T than the denoising image 110-1, as will be described later, it may use weights w1 and w2 indicated by reference numeral 120 in Figure 1.

[0021] The data processing device 100 controls the control signals S for the display device 50 to display at least one hybrid image 40, as in this exemplary embodiment. C1 ,..., S Cn It may be further configured to generate at least one of the following. The display device 50 may be, for example, part of the ophthalmic imaging device 30 or the screen of an external computer.

[0022] As in this exemplary embodiment, the data processing device 100 receives instructions I from the user of the data processing device 100 for at least one of setting or updating the weighting 120, as described later. u , I s It may be further configured to receive at least one of the following.

[0023] The data processing device 100 may be provided in any suitable form, for example, as a processor 220 of programmable signal processing hardware 200 of the type schematically shown in Figure 2. Components of the programmable signal processing hardware 200 may be included in the data processing device 100. The programmable signal processing device 200 receives and provides (in some exemplary embodiments) at least one image 10, if provided, according to the above instruction I u , I s It receives at least one of the above and outputs at least one hybrid image 40, and provides the above control signal S if available. C1 ,..., S Cn The signal processing hardware 200 further comprises a processor 220 (e.g., a CPU which is a central processing unit, and / or a GPU which is a graphics processing unit), working memory 230 (e.g., random access memory), and an instruction store 240 which stores a computer program 245 which, when executed by the processor 220, causes the processor 220 to perform various functions of the data processing device 100 described herein.

[0024] The working memory 230 stores information used by the processor 220 during the execution of the computer program 245, including the noise reduction algorithm 110 and weights w1 and w2 used as desired. The instruction store 240 may comprise a ROM (for example, in the form of electrically erasable programmable read-only memory (EEPROM) or flash memory) preloaded with computer-readable instructions. Alternatively, the instruction store 240 may comprise RAM or a similar type of memory, and the computer-readable instructions of the computer program 245 may be input from a computer program product such as a non-temporary computer-readable storage medium 250 in the form of a CD-ROM, DVD-ROM, or computer-readable signals 260 that carry computer-readable instructions. In either case, when the computer program 245 is executed by the processor 220, it causes the processor 220 to perform the functions of the data processing device 100 described herein. In other words, the data processing device 100 of this exemplary embodiment may include a computer processor 220 and a memory 240 that stores computer-readable instructions that, when executed by the computer processor 220, cause the computer processor 220 to perform the functions of the data processing device 100 described herein.

[0025] However, it should be noted that the data processing device 100 may be implemented as non-programmable hardware, such as an ASIC, FPGA, or other dedicated integrated circuit that performs the functions of the data processing device 100 described herein, or as a combination of such non-programmable and programmable hardware as described above with reference to Figure 2. Furthermore, in some exemplary embodiments, the programmable signal processing hardware 200 may further perform the functions of the ophthalmic imaging device 30 and control the display device 50.

[0026] Figure 3 is a flowchart showing the process by which the data processing device 100 processes image 10-1. Although the processes in Figure 3 are shown in a specified order, it will be understood by those skilled in the art that the processes in Figure 3 are not limited to this order, and one or more processes may be executed in parallel (for example, processes S30 and S40 may be executed before or in parallel with process S10). Processes in the flowchart represented by dashed boxes may be used as desired and may be omitted as described below.

[0027] In process S10 of Figure 3, the data processing device 100 processes image 10-1 using a noise reduction algorithm 110 based on machine learning to generate a denoised image 110-1. Noise in image 10-1 is at least partially removed by the noise reduction algorithm 110 to generate the denoised image 110-1. However, the retinal texture T shown in image 10-1 is also at least partially removed by the noise reduction algorithm 110 in this process. This loss of the retinal texture T in the denoised image 110-1 is undesirable, as mentioned above.

[0028] The noise reduction algorithm 110 may be based on a convolutional neural network (CNN), as in the exemplary embodiment described herein. For example, the noise reduction algorithm 110 may be a denoising autoencoder, a generative adversarial network (GAN), or any of the denoising CNNs described in Ilesanmi, AE, Ilesanmi, TO, "Methods for image denoising using convolutional neural network: a review," Complex Intell.Syst.7, pp.2179-2198 (2021), which are incorporated herein by reference in their entirety. Noise reduction algorithm 110 is incorporated herein by reference in its entirety as U-NET CNN described in Ronneberger, O., Fischer, P. and Brox, T., 2015, "U-net: Convolutional networks for biomedical image segmentation," In Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015:18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18 (p.234-241), Springer International Publishing, or as described in Lehtinen, J., Munkberg, J., Hasselgren, J., Laine, S., Karras, T., Aittala, M. and Aila, T., 2018, "Noise2Noise: Learning image restoration without clean data," arXiv preprint. This could be the U-NET CNN described in arXiv:1803.04189. However, the form of the noise reduction algorithm 110 is not limited in this way and may be based on other machine learning algorithms instead.For example, the noise reduction algorithm 110 may alternatively be a Bayesian image denoising algorithm, such as that described in Kataoka, S., Yasuda, M., "Bayesian Image Denoising with Multiple Noisy Images," Rev Socionetwork Strat 13, pp. 267-280 (2019), the contents of which are incorporated herein by reference in their entirety. The noise reduction algorithm 110 may be pre-trained before being stored in the memory of the data processing unit 100, or it may be trained by the data processing unit 100 upon receiving a training dataset for training the noise reduction algorithm 110.

[0029] Figures 4A and 4B are enlarged versions of image 10-1 and denoised image 110-1, respectively, shown in Figure 1. As can be seen from the figures, the noise visible in image 10-1 has been reduced in denoised image 110-1 by the noise reduction algorithm 110 (in this case, based on the same U-NET CNN as described above).

[0030] Referring again to Figure 3, in process S20, the data processing device 100 combines image 10-1 with denoising image 110-1 to generate a hybrid image 40-1 that shows more texture T than denoising image 110-1. That is, by combining image 10-1 with denoising image 110-1 to generate hybrid image 40-1 (which thus shows more retinal texture T than denoising image 110-1), the retinal texture T shown in image 10-1 is added to denoising image 110-1. Hybrid image 40-1 can be generated by combining image 10-1 with denoising image 110-1 using weighting 120, as in the first exemplary embodiment. For example, the combination of image 10-1 and denoising image 110-1 using weighting 120 may be a weighted sum or weighted average of image 10-1 and denoising image 110-1. However, the hybrid image 40-1 may alternatively be synthesized without weighting 120 by simply averaging or summing image 10-1 and denoising image 110-1 (i.e., by performing an unweighted average or sum). That is, the data processing device 100 can calculate the average or sum of image 10-1 and denoising image 110-1 such that each pixel value of each pixel in the resulting averaged image (or possibly summed image) is the average (or possibly sum) of the pixel values ​​of the pixels at the corresponding positions in image 10-1 and denoising image 110-1 corresponding to each common position on the retina.

[0031] The amount of texture in hybrid image 40-1 and denoised image 110-1 can be quantified using an algorithm such as that described in the paper "Detection of Textured Areas in Images Using a Disorganization Indicator Based on Component Counts" by R. Bergman et al., J.Electronic Imaging.17.043003 (2008), the contents of which are incorporated in their entirety herein by reference. The texture detector presented in this paper is based on the intuition that texture in natural images is "disorganized". The measure used to detect texture examines the structure of local regions of the image. This structural approach allows for the detection of both structured and unstructured textures at many scales. Furthermore, it distinguishes between edges and textures, and between textures and noise. The paper shows that the automated detection results match human classifications of the corresponding image regions. The amount of texture in hybrid image 40-1 and denoised image 110-1 can be compared by comparing the regions of these images designated as "texture" by the algorithm. Since noise and texture can have similar appearances, the amount of texture in the hybrid image 40-1 and the denoised image 110-1 can be represented using SNR measurements or structural similarity index (SSIM) measurements.

[0032] If the composite of image 10-1 with a weight of 120 and denoised image 110-1 is a weighted average, this weighted average can be expressed as follows:

[0033]

number

[0034] Here, X’ is the hybrid image 40-1, X is the image 10-1, N(X) is the noise removal image 110-1 generated by the noise reduction algorithm 110, and α is a predetermined constant with a value between 0 and 1. When α is 1, the hybrid image 40-1 is the same as the noise removal image 110-1. When α is 0, the hybrid image 40-1 is the same as the image 10-1. The weighting in this case is (1-α) and α, and the sum of the weightings is equal to 1.

[0035] Figures 5A to 5I are the image 10-1 of Figure 4A, the noise removal image 110-1 of Figure 4B, the noise reduction algorithm 110 based on U-NET CNN, and the hybrid images 50-1, 50-2,..., 50-9 generated using the above formula 1 with alpha values of 0.1, 0.2,..., 0.9 respectively. As shown in the figure, as the value of alpha increases, some noise will be added, but more retinal texture T will be gradually added to the noise removal image 110-1.

[0036] The weighting 120 may be set during manufacturing, manually input, downloaded from an external server, or determined by the processor of the ophthalmic imaging device 30 in order to give the best compromise between noise and texture in the hybrid image 40-1 considering the capabilities of the ophthalmic imaging device 30. However, as an alternative, the weighting 120 may be set by the user (e.g., a clinician) of the data processing device 100 via processes S30 and S40 used as desired prior to process S20 of Figure 3 as described below.

[0037] In the optionally used process S30 of Figure 3, the data processing device 100 receives a setting instruction I s from the user for setting the weighting 120. For example, the setting instruction I s from the user may include the user-selected weighting 120 or may be a value used to set the weighting 120 (e.g., the value of α in the above formula 1).

[0038] In process S40 used as desired in Figure 3, the data processing device 100 receives setting instruction I s Use setting instruction I to set a weight of 120. s If the user selection includes a weight 120, the data processing device 100 sets the weight 120 to the weight of these user selections. Setting instruction I s If is the value used to set the weight 120, the data processing device 100 uses that value to set the weight 120 (for example, the data processing device 100 uses the received value of α to set the weight of equation 1).

[0039] Once the hybrid image 40-1 is generated by the data processing device 100 in process S20 of Figure 3 as described above, it may be stored in the memory of the data processing device 100 or transmitted to an external storage device. In process S50 as desired in Figure 3, the data processing device 100 sends a control signal S to the display device 50 to display the hybrid image 40-1. C1 It generates a control signal S. C1 This may include a hybrid image 40-1 and an instruction that the hybrid image 40-1 should be displayed by the display device 50, or it may include an instruction that the display device 50 should retrieve the hybrid image 40-1 from memory storing the hybrid image 40-1 and display the hybrid image 40-1.

[0040] In process S60 as desired in Figure 3, the data processing device 100 issues an update instruction I to update the weighting 120 in the same manner as described above in process S30 of Figure 3. uThe system receives this information from the user. For example, the user may look at the hybrid image 40-1 displayed by the display device 50 and decide whether to increase or decrease the ratio between the weights 120 (or decide that it is not necessary to change the ratio between the weights 120, in which case process S60, which may be used as desired, is not performed).

[0041] Update instructions I u In process S70, which is used as desired in Figure 3 and received by the data processing device 100, the data processing device 100 performs the update instruction I in the same manner as described above in process S40. u Update the weight 120 using this.

[0042] After processes S60 and S70 as desired in Figure 3, the data processing device 100 may execute process S20 again in the same manner as described above, except for the weighting 120 updated by process S70, and execute process S50 as described above to control the signal S for the display device 50 to display the hybrid image thus generated. C2 These processes can be generated. These processes can be repeated as often as necessary to allow the user of the data processing device 100 to adjust the weights 120, and thus allow the user to adjust the weights 120 for the image at hand to provide the best compromise for clinical purposes between noise and texture.

[0043] In an alternative exemplary embodiment, in process S20 of Figure 3, the multiple hybrid images 40-1, 40-2, ..., 40-n may instead be generated by the data processing device 100 by synthesizing image 10-1 and denoising image 110-1 using different respective weights for each of the hybrid images 40-1, 40-2, ..., 40-n. For example, the multiple hybrid images 40-1, 40-2, ..., 40-n may also be the hybrid images 50-1, 50-2, ..., 50-9 in Figures 5A to 5I, which are generated using different pairs of weights resulting from different values ​​of α in Equation 1. Subsequently, in a manner similar to that described above in relation to process S50 of Figure 3, the display device 50 is given multiple control signals S for displaying the multiple hybrid images 40-1, 40-2, ..., 40-n. C1 ,..., S Cn The data processing device 100 can generate these. Therefore, continuing with the above example, multiple control signals S C1 ,..., S Cn The display device 50 can display each of the hybrid images 50-1, 50-2, ..., 50-9 shown in Figures 5A to 5I, for example, in a manner that can be arranged on the display device 50.

[0044] The user can compare the displayed hybrid images 40-1, 40-2, ..., 40-n and select the hybrid image 40-1, 40-2, ..., 40-n that is determined to offer the best compromise for clinical purposes between noise and texture. This selection can be used by the data processing unit 100 to update the weighting 120 (to the weighting used to generate the selected hybrid image) in the same manner as described above in process S70 of Figure 3, using update instruction I as described in process S60 above. uThis can be provided. Subsequently, the data processing device 100 can perform process S20 in Figure 3 in the same manner as described above, except for the weighting which is currently being updated, and can perform process S50 as described above to generate further control signals for the display device 50 to display the new hybrid image thus generated.

[0045] In summary, the aforementioned computer implementation method for processing an image of a portion of an eye 20 (e.g., the retina or anterior segment) acquired by an ophthalmic imaging device 30 (such as an ophthalmic FAF or OCT imaging device), wherein the image shows the texture T of that portion, and the method comprises processing the image using a machine learning-based noise reduction algorithm 110 to generate a denoised image 110-1 of that portion, wherein the texture T of that portion shown in the image is at least partially removed by the noise reduction algorithm 110 to generate a denoised image 110-1, and combining the image with the denoised image 110-1 to generate at least one hybrid image 40 of that portion showing more of the texture T of that portion than the denoised image 110-1, which has been previously described.

[0046] In process S20 of Figure 3, the data processing device 100 synthesizes image 10-1 with a denoised image 110-1 derived from image 10-1 to generate a hybrid image 40-1, which shows more retinal texture T than the denoised image 110-1, although this is not necessarily the case. In alternative exemplary embodiments, the denoised image may be synthesized with a retinal image other than the image from which the denoised image was derived. In particular, the data processing device 100 can process a plurality of images 10-1, 10-2, ..., 10-n of the retina of eye 20 acquired by the ophthalmic imaging device 30, including a first image 10-1 of the retina of eye 20 and a second image 10-2 of the retina of eye 20. These images 10-1, 10-2, ..., 10-n may be a sequence of repeating images of the retina of eye 20 that can be acquired at different (e.g., at narrow intervals and continuous) times. Since both the first image 10-1 and the second image 10-2 show the texture T of the eye 20, after processing the first image 10-1 to generate a denoised image 110-1, the data processing device 100 can instead combine the second image 10-2 (which may be adjacent to the first image 10-1 in the sequence of images, or may be another image in a different sequence from the first image 10-1) with the denoised image 110-1 to generate a hybrid image 40-1 that shows more retinal texture T than the denoised image 110-1. That is, by combining the second image 10-2 with the denoised image 110-1 to generate a hybrid image 40-1, the retinal texture T shown in the second image 10-2 is added to the denoised image 110-1 (for example, by the second image 10-2 showing more retinal texture T than the denoised image 110-1, as described above, and by the weighted average performed).

[0047] Accordingly, an alternative exemplary embodiment provides a computer implementation method for processing two or more images 10-1, 10-2 of a portion of an eye 20 (e.g., the retina or anterior segment) acquired by an ophthalmic imaging device 30 (such as an ophthalmic FAF or OCT imaging device), each of which images shows the texture T of that portion, and the method includes processing a first image 10-1 of the two or more images using a machine learning-based noise reduction algorithm 110 to generate a denoised image 110-1 of that portion, wherein the texture T of that portion shown in the first image 10-1 is at least partially removed by the noise reduction algorithm 110 to generate a denoised image 110-1, and combining a second image 10-2 of the two or more images (the second image being different from the first image 10-1) with the denoised image 110-1 to generate at least one hybrid image 40 of that portion showing more of the texture T of that portion than the denoised image 110-1.

[0048] According to exemplary embodiments, more generally, a computer implementation method is provided for processing at least one image 10 of a portion of an eye 20 (e.g., the retina or anterior segment) acquired by an ophthalmic imaging device 30 (such as an ophthalmic FAF or OCT imaging device), wherein the at least one image 10 shows the texture T of the portion, and the method is provided to process a first image 10-1 of the at least one image 10 using a machine learning-based noise reduction algorithm 110 to generate a denoised image 110-1 of the portion, wherein the texture T of the portion shown in the first image 10-1 is at least partially removed by the noise reduction algorithm 110 to generate the denoised image 110-1, and to synthesize a second image of the at least one image 10 with the denoised image 110-1 to generate at least one hybrid image 40 of the portion showing more of the texture T of the portion than the denoised image 110-1.

[0049] In the foregoing description, exemplary embodiments are described with reference to several exemplary embodiments. Therefore, this specification should be considered illustrative rather than restrictive. Similarly, the drawings illustrating the functions and advantages of the exemplary embodiments are presented for illustrative purposes only. The architecture of the exemplary embodiments is sufficiently flexible and configurable to be utilized in ways other than those shown in the accompanying drawings.

[0050] Some aspects of the examples presented herein, such as the functions of the data processing device 100, may be provided as one or more programs having instructions or instruction sequences contained in or stored in a product such as a machine-accessible or machine-readable medium, an instruction store, or a computer-readable storage device, each of which may be non-temporary in one exemplary embodiment. Programs or instructions in a non-temporary machine-accessible medium, a machine-readable medium, an instruction store, or a computer-readable storage device may be used to program a computer system or other electronic device. Machine or computer-readable mediums, instruction stores, and storage devices may include, but are not limited to, floppy diskettes, optical disks, and magneto-optical disks, or other types of mediums / machine-readable media / instruction stores / storage devices suitable for storing or transmitting electronic instructions. The techniques described herein are not limited to any particular software configuration. They may find applicability in any computing or processing environment. As used herein, the terms “computer-readable,” “machine-accessible medium,” “machine-readable medium,” “instruction store,” and “computer-readable storage device” include any medium capable of storing, encoding, or transmitting instructions or instruction sequences for execution by a machine, computer, or computer processor, causing the machine / computer / computer processor to perform any one of the methods described herein. Furthermore, in the art, it is common to say that software of any or other form (e.g., a program, procedure, process, application, module, unit, logic, etc.) takes action or produces a result. Such expressions are merely a simplified way of saying that the execution of software by a processing system causes a processor to perform an action to produce a result.

[0051] Some or all of the functions of the data processing device 100 may also be implemented by preparing application-specific integrated circuits, field-programmable gate arrays, or by interconnecting a suitable network of conventional component circuits.

[0052] Computer program products can be provided in the form of one or more storage media, instruction stores, or storage devices that store instructions that can be used to control a computer or computer processor, or to cause a computer or computer processor to perform any of the procedures of the exemplary embodiments described herein. Storage media / instruction stores / storage devices may include, but are not limited to, optical discs, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory, flash cards, magnetic cards, optical cards, nanosystems, molecular memory integrated circuits, RAIDs, remote data storage devices / archives / warehousing, and / or any other types of devices suitable for storing instructions and / or data.

[0053] Where stored in one or more computer-readable media, instruction stores, or storage devices, some implementations include software for controlling both hardware and the system, and enabling the system or microprocessor to interact with a human user or other mechanism using the results of the exemplary embodiments described herein. Such software may include, but is not limited to, device drivers, operating systems, and user applications. Finally, such computer-readable media or storage devices further include software for performing exemplary embodiments of the invention as described above.

[0054] The system programming and / or software includes software modules for performing the procedures described herein. In some exemplary embodiments, the modules include software, while in other exemplary embodiments, the modules include hardware, or a combination of hardware and software.

[0055] While various exemplary embodiments of the present invention have been described above, it should be understood that these are presented as examples and not limitations. Those skilled in the art will see that various modifications of form and detail are possible. Therefore, the present invention should not be limited by any of the exemplary embodiments described above, but should be defined solely in accordance with the following claims and their equivalents.

[0056] Furthermore, the purpose of the abstract is to enable the Patent Office and the general public, particularly scientists, engineers, and practitioners in the art who are not familiar with patent or legal terminology or expressions, to quickly determine the nature and essence of the technical disclosure of this application from a general search. The abstract is not intended to limit the scope of the exemplary embodiments presented herein. It should also be understood that the procedures described in the claims do not need to be performed in the order presented.

[0057] While this specification includes details of many specific embodiments, these should not be construed as limitations on the scope of any invention or claimable, but rather as descriptions of features specific to the particular embodiments described herein. Certain features described herein in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented separately or in any suitable partial combination in multiple embodiments. Furthermore, features may be described above as acting in a particular combination and may be initially claimed as such, but one or more features from a claimed combination may, in some cases, be removed from the combination, and the claimed combination may cover a partial combination or a variation of a partial combination.

[0058] In certain situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of various components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0059] While several exemplary embodiments and configurations have been described so far, it is clear that these are illustrative, not limiting, and are presented as examples only. In particular, many of the examples presented herein involve specific combinations of apparatus or software elements, but these elements may be combined in other ways to achieve the same purpose. Operations, elements, and features discussed only in relation to one embodiment are not intended to be excluded from similar roles in other embodiments or configurations.

Claims

1. A computer implementation method for processing at least one image (10) of the retina of an eye (20) acquired by an ophthalmic imaging device (30), wherein the at least one image (10) shows the texture (T) of the retina, and the computer implementation method The process involves using a machine learning-based noise reduction algorithm (110) to process a first image (10-1) of the at least one image (10) to generate a denoised image (110-1) of the retina, wherein the texture (T) of the retina shown in the first image (10-1) is at least partially removed by the noise reduction algorithm (110) to generate the denoised image (110-1) (S10), (S20) The process involves combining a second image (10-1, 10-2), which is an image to which the noise reduction algorithm (110) has not been applied, with the denoised image (110-1) to generate at least one hybrid image (40) of the retina that shows more of the texture (T) of the retina than the denoised image (110-1) and complements the texture (T) of the retina lost by the noise reduction algorithm (110), The sequence of repeated images of the retina, including the first image (10-1) and the second image (10-1, 10-2), having the same field of view region and acquired at different times from each other. The at least one hybrid image (40) of the retina is generated by combining the second image (10-1, 10-2) with the denoising image (110-1) using the respective weights (120) for the second image (10-1, 10-2) and the denoising image (110-1). A computer implementation method wherein the at least one hybrid image (40) of the retina is generated by calculating either a weighted sum or a weighted average of the second image (10-1, 10-2) and the denoising image (110-1) using the weighting (120).

2. The computer implementation method according to claim 1, further comprising receiving a setting instruction (Is) from a user for setting the weighting (120) (S30), and setting the weighting (120) using the setting instruction (Is) (S40).

3. The computer implementation method according to claim 1, further comprising the display device (50) generating a control signal (SC1) for displaying the at least one hybrid image (40) (S50).

4. A computer implementation method according to claim 3, wherein a plurality of hybrid images (40-1, 40-2, ..., 40-n) are generated by combining the second image (10-1, 10-2) with the denoised image (110-1) using different respective weights (120), and a plurality of control signals (SC1, ..., SCn) are generated for the display device (50) to display the plurality of hybrid images (40-1, 40-2, ..., 40-n).

5. The computer implementation method according to claim 3, further comprising receiving an update instruction (Iu) from a user to update the weighting (120) (S60), and updating the weighting (120) using the update instruction (Iu) (S70).

6. The computer implementation method according to claim 1, wherein the noise reduction algorithm (110) is based on a convolutional neural network.

7. The computer implementation method according to claim 1, wherein the at least one image (10) of the retina of the eye (20) is at least one fundus autofluorescence image of the retina of the eye (20).

8. A computer program (245) that, when executed by a processor (220), includes computer-readable instructions causing the processor (220) to perform the method according to claim 1.

9. A data processing device (100) configured to perform the method described in claim 1.

Images