Noise reduction of ophthalmic images

Abstract

The invention relates to noise reduction of ophthalmic images. A computer-implemented method of processing at least one image of a retina of an eye acquired by an ophthalmic imaging device, where the at least one image displays a texture of the retina. The method includes processing a first image of the at least one image using a machine learning-based noise reduction algorithm to generate a denoised image of the retina, where at least part of textures of the retina displayed in the first image are removed by the noise reduction algorithm to generate the denoised image. The method further includes combining a second image of the at least one image with the de-noised image to generate at least one blended image of the retina, the at least one blended image displaying more textures of the retina than the de-noised image.

Description

[0001] field

[0002] Example aspects herein relate generally to the field of noise reduction of ophthalmic images, and in particular, to techniques for reducing noise in ophthalmic images based on machine learning. background

[0004] Ophthalmic imaging devices employ various imaging techniques to image different parts of the eye (e.g., the retina), and are used by clinicians to diagnose and manage various eye conditions. Ophthalmic imaging devices include (but are not limited to) autofluorescence (AF) ophthalmic imaging devices, scanning laser ophthalmoscopy (SLO), optical coherence tomography (OCT) imaging devices, fundus cameras, and microperimetry devices (among other devices), or a combination of two or more of such devices.

[0005] Images acquired by such ophthalmic imaging devices are affected by noise sources (e.g., Gaussian noise, quantum noise, or speckle noise), which reduce the signal-to-noise ratio (SNR) of the acquired images. This may reduce the clinical value of the acquired images, as it may obscure important clinical information that may be useful in diagnosing various pathologies of the eye. Therefore, noise reduction algorithms are often used to improve the SNR in acquired ophthalmic images. Some modern noise reduction algorithms employ machine learning algorithms, such as, for example, convolutional neural networks (CNNs), which have been shown to effectively perform de-noising tasks.

[0006] Overview

[0007] According to a first example aspect of the present invention, there is provided a computer-implemented method of processing at least one image of a retina of an eye acquired by an ophthalmic imaging device, wherein the at least one image displays texture of the retina. The method includes processing a first image of the at least one image using a machine learning-based noise reduction algorithm to generate a denoised image of the retina, wherein the texture of the retina displayed in the first image is at least partially removed by the noise reduction algorithm to generate the denoised image. The method also includes combining a second image of the at least one image with the denoised image to generate at least one blended image of the retina, the at least one blended image displaying more texture of the retina than the denoised image.

[0008] The at least one mixed image of the retina can be generated by combining the second image with the denoised image using their respective weights. In addition, the at least one mixed image of the retina can be generated by calculating one of a weighted sum or a weighted average of the second image and the denoised image using the weights.

[0009] The computer-implemented method of the first example aspect or any of the example implementations thereof set forth above may also include: receiving a setting instruction from a user for setting the weight, and using the setting instruction to set the weight. Additionally or alternatively, the method may include generating a control signal for a display device to display the at least one mixed image.

[0010] In some example implementations, a plurality of mixed images are generated by combining the second image with the denoised image using different corresponding weights, and a plurality of control signals are generated for a display device to display the plurality of mixed images.

[0011] The computer-implemented method of the first example aspect or any of the example implementations thereof set forth above may further include receiving an update indication from a user for updating the weights, and updating the weights using the update indication.

[0012] In any of the foregoing, the noise reduction algorithm may be based on a convolutional neural network, and the at least one image of the retina of the eye may be at least one fundus autofluorescence image of the retina of the eye.

[0013] According to a second exemplary aspect of the present invention, a computer program is also provided, the computer program comprising computer readable instructions, which, when executed by a processor, causes the processor to perform the method according to the first exemplary aspect or any one of the exemplary implementations thereof set forth above. The computer program may be stored on a non-transitory computer readable storage medium (e.g., such as a computer hard disk or CD), or may be carried by a computer readable signal.

[0014] According to a third example aspect of the present invention, there is also provided a data processing device, which is arranged to perform the method according to the first example aspect or any one of the example implementations thereof set forth above. The data processing device may include at least one processor and at least one memory storing computer-readable instructions, which, when executed by at least one processor, causes the at least one processor to perform the method according to the first example aspect or any one of the example implementations thereof set forth above. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Example embodiments will now be explained in detail, by way of non-limiting examples only, with reference to the accompanying drawings described below. Unless otherwise specified, like reference numerals appearing in different drawings represent identical or functionally similar elements.

[0017] Figure 1 is a schematic diagram of a data processing apparatus 100 according to an example embodiment herein.

[0018] Figure 2 is a schematic diagram of an example implementation of a data processing apparatus 100 of an example embodiment in programmable signal processing hardware 200 .

[0019] Figure 3 is a flowchart illustrating a process in which the data processing apparatus 100 processes at least one image according to an example embodiment of this document.

[0020] Figure 4A and Figure 4B They are Figure 1 An enlarged version of image 10 - 1 and denoised image 110 - 1 are shown.

[0021] FIG. 5A to FIG. 5I , 50 - 9 are mixed images 50 - 1 , 50 - 2 , . . . , 50 - 9 , which are generated using Equation 1 described herein by using the image 10 - 1 , the denoised image 110 - 1 , and corresponding α values ​​of 0.1 , 0.2, . . . , 0.9, respectively.

[0022] Detailed Description of Example Embodiments

[0023] The present inventors have recognized that using conventional noise reduction algorithms such as those described above to reduce noise in retinal images often results in a loss of some or all of the texture of the retina displayed in the image. This appears to be caused by the difficulty that the noise reduction algorithm has in distinguishing between noise and texture within the acquired image, which often have a similar appearance. This loss of texture may result in the loss of important clinical information from the acquired image that may otherwise be useful in diagnosing various pathologies of the eye, and may result in images that appear unrealistic to a clinician (e.g., the images appear "flat" to the clinician).

[0024] To address the above-mentioned problems recognized by the inventors, the inventors have devised a computer-implemented method for processing at least one image of the retina of an eye acquired by an ophthalmic imaging device, wherein the at least one image displays the texture of the retina. The method includes processing a first image of the at least one image using a machine learning-based denoising algorithm to generate a denoised image of the retina, wherein the texture of the retina displayed in the first image is at least partially removed by the denoising algorithm to generate the denoised image. The method also includes combining a second image of the at least one image with the denoised image to generate at least one blended image of the retina, the at least one blended image displaying more texture of the retina than the denoised image. This improvement in the amount of texture displayed in the blended image relative to the amount of texture displayed in the denoised image may be clinically important because information about the texture of the retina may be important clinical information useful for diagnosing various pathologies of the eye. In addition, due to the increase in the amount of texture of the retina displayed in the blended image, the blended image may appear more realistic to a clinician (e.g., the blended image may not appear "flat"). The shortcomings of the conventional noise reduction algorithms discussed above occur not only in the processing of images of the retina, but also in the processing of images of other parts of the eye, in particular in the processing of images of the anterior segment. The computer-implemented method described herein is also applicable to the processing of such images.

[0025] Figure 1 is a schematic diagram of a data processing apparatus 100 arranged to process at least one image 10 of a retina of an eye 20 acquired by an ophthalmic imaging device 30 .

[0026] As in the present exemplary embodiment, at least one image 10 may be at least one fundus autofluorescence (FAF) image acquired by a FAF ophthalmic imaging device as an ophthalmic imaging device 30 (as shown in the figure, see also Figure 4A ). For example, at least one FAF image may be obtained by Optos TMAt least one green (i.e., green laser) FAF image acquired by a California or Silverstone imaging device, or may be at least one blue (i.e., blue laser) FAF image. However, the at least one image 10 is not limited thereto, and may take other forms showing the texture T of the retina of the eye 20. For example, the at least one image 10 may alternatively be at least one optical coherence tomography (OCT) image (e.g., B-scan) acquired by an OCT imaging device as the ophthalmic imaging device 30. As another example, the at least one image 10 may be at least one reflectance image (e.g., a red reflectance image, a blue reflectance image, or a green reflectance image) of the retina acquired by a fundus camera, an SLO, or other ophthalmic imaging device for acquiring a reflectance image. The portion of the eye 20 imaged in the image 10 does not necessarily include the retina, but may include the anterior segment (AS) of the eye 20.

[0027] At least one image 10 of the retina of the eye 20 shows a texture T of the retina. In this context, the texture is an anatomical texture associated with the retina, which indicates an anatomical structure in the retina of the eye 20. For example, in the case where the image acquired by the ophthalmic imaging device 30 is a FAF image, as in the present exemplary embodiment, the structure may be defined by the spatial distribution of fluorophores on the retina. As another example, in the case where the image acquired by the ophthalmic imaging device 30 is an OCT image, the structure may include the physical structure of one or more layers of the retina. As another example, in the case where the image acquired by the ophthalmic imaging device 30 is a reflection image of the retina, the structure may include the upper surface of the retina, so that the texture is a physical texture of the surface reflecting the topography of the surface. As in the present exemplary embodiment, as described below, the at least one image 10 processed by the data processing device 100 may be one image 10-1 acquired by the ophthalmic imaging device 30, but the data processing device 100 may instead process multiple images 10-1, 10-2, ..., 10-n of the retina.

[0028] As in the present example embodiment, as described in more detail below, the data processing apparatus 100 may be arranged to process the image 10-1 using a machine learning based noise reduction algorithm 110 to generate a denoised image 110-1 of the retina. The data processing apparatus 100 may be arranged to receive the image 10-1 from the ophthalmic imaging device 30, but the data processing apparatus 100 may alternatively be arranged to acquire the image 10-1 by controlling the ophthalmic imaging device 30 to capture the image 10-1. For example, this may be the case where the functionality of the data processing apparatus 100 is implemented by a processor of the ophthalmic imaging device 30, which controls the acquisition of the image by the ophthalmic imaging device 30, or this may be the case where the data processing apparatus 100 is arranged to control the functionality of the processor of the ophthalmic imaging device 30, which controls the acquisition of the image by the ophthalmic imaging device 30.

[0029] As in the present example embodiment, the data processing apparatus 100 may also be arranged to combine the (single) image 10-1 with the denoised image 110-1 derived therefrom to generate a first mixed image 40-1 (as in the example embodiment for the present example embodiment). Figure 1 is shown as a FAF image in , see also Figure 4B ), the first mixed image 40-1 shows more texture T than the denoised image 110-1. However, as described below, the data processing device 100 may more generally generate at least one mixed image 40, and thus may generate a plurality of mixed images 40-1, 40-2, ..., 40-n. As will be described later, the data processing device 100 may use the weights w1 and w2 (in Figure 1 120 ), the first mixed image 40 - 1 shows more texture T than the denoised image 110 - 1 .

[0030] As in the present exemplary embodiment, the data processing apparatus 100 may also be arranged to generate a control signal S for the display device 50 to display at least one mixed image 40. C1 ,……,S Cn For example, the display device 50 may be a part of the ophthalmic imaging device 30, or may be a screen of an external computer.

[0031] As in the present example embodiment, the data processing apparatus 100 may also be arranged to receive an indication from a user of the data processing apparatus 100. u ,I s At least one of is used to perform at least one of the following operations: setting weight 120 or updating weight 120, which will be described later.

[0032] The data processing device 100 may be provided in any suitable form, for example Figure 2 1. The programmable signal processing hardware 200 of the type schematically illustrated in FIG. 1 may include components of the programmable signal processing hardware 200 in the data processing apparatus 100. The programmable signal processing apparatus 200 includes a communication interface (I / F) 210 for receiving at least one image 10 and receiving the above-mentioned indication I u ,I s and outputting at least one mixed image 40 and the control signal S C1 ,……,S Cn The signal processing hardware 200 further includes a processor 220 (e.g., a central processing unit CPU and / or a graphics processing unit GPU), a working memory 230 (e.g., a random access memory), and an instruction storage device 240 storing a computer program 245, which includes computer readable instructions that, when executed by the processor 220, cause the processor 220 to perform various functions of the data processing device 100 described herein.

[0033] The working memory 230 stores information used by the processor 220 during the execution of the computer program 245, including the noise reduction algorithm 110 and optionally the weights w1 and w2. The instruction storage device 240 may include a ROM (e.g., in the form of an electrically erasable programmable read-only memory (EEPROM) or flash memory) preloaded with computer-readable instructions. Alternatively, the instruction storage device 240 may include a RAM or similar type of memory, and the computer-readable instructions of the computer program 245 may be input to the instruction storage device 240 from a computer program product (e.g., a non-transitory computer-readable storage medium 250 in the form of a CD-ROM, DVDROM, etc.) or a computer-readable signal 260 carrying computer-readable instructions. In any case, the computer program 245, when executed by the processor 220, causes the processor 220 to perform the functions of the data processing device 100 described herein. In other words, the data processing device 100 of this example embodiment may include a computer processor 220 and a memory 240 storing computer-readable instructions, which, when executed by the computer processor 220, causes the computer processor 220 to perform the functions of the data processing device 100 as described herein.

[0034] However, it should be noted that the data processing apparatus 100 may alternatively be implemented in non-programmable hardware (such as an ASIC, FPGA or other integrated circuit dedicated to performing the functions of the data processing apparatus 100 described herein), or in such non-programmable hardware and as described above with reference to Figure 2Furthermore, in some example embodiments, the programmable signal processing hardware 200 may also perform the functions of the ophthalmic imaging device 30 and may control the display device 50 .

[0035] Figure 3 1 is a flowchart showing a process in which the data processing device 100 processes the image 10-1. Figure 3 The processes in FIG. 1 are shown in a specified order, but those skilled in the art will understand that Figure 3 The process is not limited to this order, and one or more processes in the process may be performed in parallel (for example, processes S30 and S40 may be performed before or in parallel with process S10). As described below, the process represented by a dotted box in the flowchart is optional and may be omitted.

[0036] exist Figure 3 In the process S10, the data processing device 100 uses the noise reduction algorithm 110 based on machine learning to process the image 10-1 to generate the denoised image 110-1. The noise in the image 10-1 is at least partially removed by the noise reduction algorithm 110 to generate the denoised image 110-1. However, in this process, the texture T of the retina shown in the image 10-1 is also at least partially removed by the noise reduction algorithm 110. As explained previously, such loss of the texture T of the retina in the denoised image 110-1 is undesirable.

[0037] As in the present example embodiment, the denoising algorithm 110 can be based on a convolutional neural network (CNN). For example, the denoising algorithm 110 can be a denoising autoencoder, a generative adversarial network (GAN), or one of the denoising CNNs described in the following document: Ilesanmi, AE, Ilesanmi, TO, "Methods for image denoising using convolutional neural network: a review.", Complex Intell. Syst. 7, 2179–2198 (2021), the contents of which are incorporated herein by reference in their entirety. The denoising algorithm 110 may be a U-NET CNN as described in Ronneberger, O., Fischer, P., and Brox, T., 2015, “U-net: Convolutional networks for biomedical image segmentation”, in Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18 (pp. 234-241), Springer International Publishing, or the denoising algorithm 110 may be a U-NET CNN as described in Lehtinen, J., Munkberg, J., Hasselgren, J., Laine, S., Karras, T., Aittala, M., and Aila, T., 2018, “Noise2Noise: Learning image restoration without clean data”, arXiv preprint arXiv:1803.04189, the contents of which are incorporated herein by reference in their entirety. However, the form of the denoising algorithm 110 is not limited thereto and may instead be based on other machine learning algorithms. For example, the denoising algorithm 110 may instead be a Bayesian image denoising algorithm, such as the Bayesian image denoising algorithm described in the following document: Kataoka, S., Yasuda, M., "Bayesian Image Denoising with Multiple Noisy Images", Rev Socionetwork Strat 13, 267-280 (2019), the contents of which are incorporated herein by reference in their entirety.The denoising algorithm 110 may be pre-trained before it is stored in the memory of the data processing device 100 , or may be trained by the data processing device 100 upon receiving a training data set for training the denoising algorithm 110 .

[0038] Figure 4A and Figure 4B They are Figure 1 1 and a zoomed-in version of the image 10-1 and the denoised image 110-1 shown in . It can be seen that the noise visible in the image 10-1 has been reduced in the denoised image 110-1 due to the denoising algorithm 110 (in this case, the algorithm is based on a U-NET CNN similar to those described above).

[0039] Reference again Figure 3 , in process S20, the data processing device 100 combines the image 10-1 with the denoised image 110-1 to generate a mixed image 40-1, which shows more texture T than the denoised image 110-1. That is, by combining the image 10-1 with the denoised image 110-1, the texture T of the retina shown in the image 10-1 is added to the denoised image 110-1, thereby generating the mixed image 40-1 (so the mixed image 40-1 shows more texture T of the retina than the denoised image 110-1). As in the first exemplary embodiment, the mixed image 40-1 can be generated by combining the image 10-1 with the denoised image 110-1 using the weight 120. For example, the combination of the image 10-1 and the denoised image 110-1 using the weight 120 can be a weighted sum or a weighted average of the image 10-1 and the denoised image 110-1. However, alternatively, the mixed image 40-1 may be combined by simply averaging or summing the image 10-1 and the denoised image 110-1 (i.e., by performing an unweighted average or sum) without the weights 120. That is, the data processing apparatus 100 may calculate the average value or sum of the image 10-1 and the denoised image 110-1 so that the corresponding pixel value of each pixel in the resulting average image (or summed image, as the case may be) is the mean (or sum, as the case may be) of the pixel values ​​of the pixels correspondingly located in the image 10-1 and the denoised image 110-1 corresponding to the corresponding common position on the retina.

[0040] The amount of texture in the mixed image 40-1 and the denoised image 110-1 can be quantified using an algorithm as described in the following article: "Detection of Textured Areas in Images Using a Disorganization Indicator Based on Component Counts", R. Bergman et al., J. Electronic Imaging., 17.043003, 2008, the contents of which are incorporated herein by reference in their entirety. The texture detector proposed in the article is based on the intuition that texture in natural images is "disorganized". The measurement used to detect texture examines the structure of local areas of the image. This structural approach allows detection of structured and unstructured textures at many scales. In addition, the method distinguishes between edges and texture, and also distinguishes between texture and noise. The automatic detection results are shown in the article to match human classification of corresponding image areas. The amount of texture in the mixed image 40-1 and the denoised image 110-1 can be compared by comparing the areas in these images that are designated as "texture" by the algorithm. An indication of the amount of texture in the mixed image 40 - 1 and the denoised image 110 - 1 may be obtained using a measure of the SNR or the structural similarity index metric (SSIM), since noise and texture may have a similar appearance.

[0041] In the case where the combination of the image 10-1 and the denoised image 110-1 using the weights 120 is a weighted average, the weighted average may be expressed as:

[0042] X′=(1-α)X+αN(X) Equation 1

[0043] Where X' is the mixed image 40-1, X is the image 10-1, N(X) is the denoised image 110-1 generated by the denoising algorithm 110, and α is a predetermined constant between values ​​0 and 1. Note that when α is 1, the mixed image 40-1 is the same as the denoised image 110-1, and when α is 0, the mixed image 40-1 is the same as the image 10-1. The weights in this case are (1-α) and α, and the sum of the weights is equal to 1.

[0044] FIG. 5A to FIG. 5I are mixed images 50-1, 50-2, ..., 50-9, which are respectively Figure 4A Image 10-1, Figure 4BThe denoised image 110-1 in FIG. 1 , the denoising algorithm 110 based on the U-NET CNN and the corresponding α values ​​are 0.1, 0.2, ..., 0.9 and generated using the above equation 1. As shown in the figure, as the value of α increases, the texture T of the retina is gradually added more to the denoised image 110-1, but also at the expense of adding some noise.

[0045] The weights 120 may be set at manufacture, manually entered, downloaded from an external server, or determined by a processor of the ophthalmic imaging device 30 to give an optimal compromise between noise and texture in the mixed image 40-1 for clinical purposes given the capabilities of the ophthalmic imaging device 30. However, the weights 120 may alternatively be determined by a user of the data processing apparatus 100 (e.g., a clinician) via a Figure 3 The optional processes S30 and S40 described below before the process S20 in the embodiment are set.

[0046] exist Figure 3 In the optional process S30, the data processing device 100 receives a setting instruction I from the user to set the weight 120. s For example, a setting instruction from the user s The weight 120 may be included as selected by a user, or may be a value used to set the weight 120 (eg, the value of α in Equation 1 above).

[0047] exist Figure 3 In the optional process S40, the data processing device 100 uses the setting instruction I s To set the weight to 120. For example, in setting the instruction I s When the weights 120 selected by the user are included, the data processing device 100 sets the weights 120 to these user selected weights. s When α is a value for setting the weight 120 , the data processing apparatus 100 uses the value to set the weight 120 (eg, the data processing apparatus 100 sets the weight in Equation 1 by using the received value of α).

[0048] Once the mixed image 40-1 has been described above Figure 3 If the mixed image is generated by the data processing device 100 in the process S20, the mixed image may be stored in the memory of the data processing device 100 or may be transmitted to an external storage device. Figure 3 In the optional process S50, the data processing apparatus 100 generates a control signal S for the display device 50 to display the mixed image 40-1. C1 For example, the control signal S C1The mixed image 40-1 and an indication that the mixed image 40-1 is to be displayed by the display device 50 may be included, or an indication that the display device 50 is to call the mixed image 40-1 from a memory storing the mixed image 40-1 and is to display the mixed image 40-1 may be included.

[0049] exist Figure 3 In the optional process S60, the data processing device 100 performs the above Figure 3 In a manner similar to that described in the process S30, an update instruction for updating the weight 120 is received from the user. u For example, the user may view the mixed image 40 - 1 displayed by the display device 50 and decide whether the ratio between the weights 120 should be increased or decreased (or that it does not need to be changed, in which case the optional process S60 will not be performed).

[0050] exist Figure 3 In the optional process S70, in the update indication I u When the update indication has been received by the data processing apparatus 100, the data processing apparatus 100 uses the update indication I in a manner similar to that described above in process S40. u to update the weight to 120.

[0051] exist Figure 3 After the optional processes S60 and S70 in the above, the data processing apparatus 100 may again perform the process S20 in the same manner as described above, except that the weight 120 has been updated by the above process S70, and the data processing apparatus 100 may perform the above process S50 to generate a control signal S for causing the display device 50 to display the mixed image thus generated. C2 These processes may be repeated as often as necessary to allow a user of the data processing apparatus 100 to adjust the weights 120 and may therefore allow the user to tailor the weights 120 to the image 10 - 1 to be generated to provide an optimal compromise between noise and texture for clinical purposes.

[0052] In an alternative exemplary embodiment, Figure 3 In the process S20, the plurality of mixed images 40-1, 40-2, ..., 40-n may alternatively be generated by the data processing device 100 by combining the image 10-1 and the denoised image 110-1 using different corresponding weights for each of the mixed images 40-1, 40-2, ..., 40-n. For example, the plurality of mixed images 40-1, 40-2, ..., 40-n may be FIG. 5A to FIG. 5I The mixed images 50-1, 50-2, ..., 50-9 are generated using different weight pairs generated by different corresponding α values ​​in equation 1. Subsequently, the mixed images 50-1, 50-2, ..., 50-9 can be generated in the same way as above. Figure 3In a similar manner to the process S50 described above, the data processing apparatus 100 generates a plurality of control signals S for the display device 50 to display the plurality of mixed images 40 - 1 , 40 - 2 , . . . , 40 - n. C1 ,……,S Cn Continuing with the previous example, multiple control signals S C1 ,……,S Cn Therefore, the display device 50 can be made to display, for example, the images displayed side by side on the display device 50 FIG. 5A to FIG. 5I Each of the mixed images 50-1, 50-2, ..., 50-9.

[0053] The user can compare the displayed multiple mixed images 40-1, 40-2, ..., 40-n, and select the mixed image of the multiple mixed images 40-1, 40-2, ..., 40-n that is judged to provide the best compromise between noise and texture for clinical purposes. This selection can provide an update indication I as described above in process S60. u , update indication I u can be used by the data processing device 100 to Figure 3 The weight 120 is updated in the same manner as described in the process S70 of FIG. 1 (the weight 120 is updated to the weight used to generate the selected mixed image). Thereafter, the data processing apparatus 100 may execute the process in the same manner as described above. Figure 3 The process S20 is different in that the currently updated weight is used, and the process S50 can be performed as described above to generate another control signal for the display device 50 to display the new mixed image generated thereby.

[0054] In summary, a computer-implemented method for processing an image of a portion of an eye 20 (e.g., the retina or the anterior segment of the eye) acquired by an ophthalmic imaging device 30 (such as an ophthalmic FAF or OCT imaging device) has been described in the foregoing, wherein the image displays a texture T of the portion, the method comprising: processing the image using a machine learning-based denoising algorithm 110 to generate a denoised image 110-1 of the portion, wherein the texture T of the portion displayed in the image is at least partially removed by the denoising algorithm 110 to generate the denoised image 110-1; and combining the image with the denoised image 110-1 to generate at least one hybrid image 40 of the portion, the at least one hybrid image 40 displaying more texture T of the portion than the denoised image 110-1.

[0055] Despite Figure 3In the process S20, the data processing apparatus 100 combines the image 10-1 with the denoised image 110-1 derived from the image 10-1 to generate a mixed image 40-1, the mixed image 40-1 showing more texture T of the retina than the denoised image 110-1, but this is not necessarily the case. In an alternative example embodiment, the denoised image may be combined with an image of the retina other than the image from which the denoised image is derived. Specifically, the data processing apparatus 100 may process a plurality of images 10-1, 10-2, ..., 10-n of the retina of the eye 20 acquired by the ophthalmic imaging device 30, the plurality of images including a first image 10-1 of the retina of the eye 20 and a second image 10-2 of the retina of the eye 20. These images 10-1, 10-2, ..., 10-n may be a sequence of repeated images of the retina of the eye 20, which may be acquired, for example, at different (e.g., closely spaced and consecutive) corresponding times. Both the first image 10-1 and the second image 10-2 show the texture T of the eye 20, so after processing the first image 10-1 to generate the denoised image 110-1, the data processing device 100 may alternatively combine the second image 10-2 (the second image 10-2 may be immediately adjacent to the first image 10-1 in the image sequence, or another image different from the first image 10-1 in the sequence) with the denoised image 110-1 to generate a mixed image 40-1 that shows more texture T of the retina than the denoised image 110-1. That is, the texture T of the retina shown in the second image 10-2 is added to the denoised image 110-1 by combining the second image 10-2 with the denoised image 110-1, thereby generating the mixed image 40-1 (for example, by the second image 10-2 showing more texture T of the retina than the denoised image 110-1 and performing weighted averaging as described above).

[0056] Thus, an alternative exemplary embodiment provides a computer-implemented method for processing two or more images 10-1, 10-2 of a portion of an eye 20 (e.g., the retina or the anterior segment of the eye) acquired by an ophthalmic imaging device 30 (such as an ophthalmic FAF or OCT imaging device), wherein each image displays a texture T of the portion, the method comprising: processing a first image 10-1 of the two or more images using a machine learning-based denoising algorithm 110 to generate a denoised image 110-1 of the portion, wherein the texture T of the portion displayed in the first image 10-1 is at least partially removed by the denoising algorithm 110 to generate the denoised image 110-1; and combining a second image 10-2 of the two or more images (the second image being different from the first image 10-1) with the denoised image 110-1 to generate at least one mixed image 40 of the portion, the at least one mixed image 40 displaying more texture T of the portion than the denoised image 110-1.

[0057] According to example embodiments of the present invention, there is more generally provided a computer-implemented method for processing at least one image 10 of a portion of an eye 20 (e.g., a retina or anterior segment of the eye) acquired by an ophthalmic imaging device 30 (such as an ophthalmic FAF or OCT imaging device), wherein the at least one image 10 displays a texture T of the portion, the method comprising: processing a first image 10-1 of the at least one image 10 using a machine learning-based denoising algorithm 110 to generate a denoised image 110-1 of the portion, wherein the texture T of the portion displayed in the first image 10-1 is at least partially removed by the denoising algorithm 110 to generate the denoised image 110-1; and combining a second image of the at least one image 10 with the denoised image 110-1 to generate at least one mixed image 40 of the portion, the at least one mixed image 40 displaying more texture T of the portion than the denoised image 110-1.

[0058] In the foregoing description, example aspects have been described with reference to several example embodiments. Therefore, the description should be regarded as illustrative rather than restrictive. Similarly, the drawings shown in the accompanying drawings highlighting the features and advantages of the example embodiments are presented for illustrative purposes only. The architecture of the example embodiments is sufficiently flexible and configurable that it can be utilized in ways other than those shown in the accompanying drawings.

[0059] In an example embodiment, some aspects of the examples given herein, such as the functionality of the data processing apparatus 100, may be provided as a computer program or software, such as one or more programs having instructions or sequences of instructions, included or stored in an article of manufacture, such as a machine-accessible or machine-readable medium, an instruction storage device, or a computer-readable storage device, each of which may be non-transitory. The program or instructions on a non-transitory machine-accessible medium, a machine-readable medium, an instruction storage device, or a computer-readable storage device may be used to program a computer system or other electronic device. The machine-readable medium or computer-readable medium, the instruction storage device, and the storage device may include, but are not limited to, a floppy disk, an optical disk, and a magneto-optical disk or other types of media / machine-readable media / instruction storage devices / storage devices suitable for storing or transmitting electronic instructions. The techniques described herein are not limited to any particular software configuration. They may find application in any computing or processing environment. As used herein, the terms "computer-readable", "machine accessible medium", "machine-readable medium", "instruction storage", and "computer-readable storage device" shall include any medium that is capable of storing, encoding, or transmitting instructions or sequences of instructions to be executed by a machine, computer, or computer processor and causing the machine / computer / computer processor to perform any of the methods described herein. Furthermore, it is common in the art to refer to software in one or another form (e.g., program, procedure, process, application, module, unit, logic, etc.) as taking an action or causing a result. Such an expression is merely a shorthand way of stating that execution of the software by a processing system causes the processor to perform an action to produce a result.

[0060] Some or all of the functions of data processing apparatus 100 may also be implemented by preparing application specific integrated circuits, field programmable gate arrays, or by interconnecting an appropriate network of conventional component circuits.

[0061] A computer program product may be one or more storage media, instruction storage means, or storage devices on or in which instructions are stored that can be used to control or cause a computer or computer processor to perform any of the processes of the example embodiments described herein. Storage media / instruction storage means / storage devices may include, by way of example and without limitation, optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memories, flash memory cards, magnetic cards, optical cards, nanosystems, molecular memory integrated circuits, RAIDs, remote data storage / archiving / warehousing devices, and / or any other type of device suitable for storing instructions and / or data.

[0062] Some implementations stored on any of one or more computer-readable media, instruction storage devices, or storage devices include hardware for controlling the system and software for enabling the system or microprocessor to interact with a human user or other mechanism using the results of the example embodiments described herein. Such software may include, without limitation, device drivers, operating systems, and user applications. Ultimately, as described above, such computer-readable media or storage devices also include software for performing example aspects of the present invention.

[0063] Software modules for implementing the processes described herein are included in the programming and / or software of the system. In some example embodiments herein, the modules include software, but in other example embodiments herein, the modules include hardware or a combination of hardware and software.

[0064] Although a number of example embodiments of the present invention have been described above, it should be understood that they are presented as examples rather than limitations. It will be apparent to those skilled in the relevant art that various changes may be made in form and detail. Therefore, the present invention should not be limited to any of the above example embodiments, but should only be defined in accordance with the appended claims and their equivalents.

[0065] Furthermore, the purpose of the Abstract is to enable patent offices and the public in general, and especially scientists, engineers, and practitioners in the art who are not familiar with patent or legal terminology or wording, to quickly determine the nature and essence of the technical disclosure of the present application based on a cursory inspection. The Abstract is not intended to be limiting in any way with respect to the scope of the example embodiments presented herein. It should also be understood that any process recited in the claims need not be performed in the order presented.

[0066] Although this specification contains many specific embodiment details, these should not be understood as limitations on the scope of any invention or content that may be claimed, but should be understood as descriptions of features specific to the specific embodiments described herein. Certain features described in this specification in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. In addition, although features may be described above as acting in a specific combination and even initially claimed as such, one or more features from the claimed combination may be deleted from the combination in some cases, and the claimed combination may be directed to a sub-combination or a variant of the sub-combination.

[0067] In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of the various components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0068] Now that some illustrative embodiments and embodiments have been described, it is apparent that the foregoing embodiments are illustrative rather than restrictive and have been given by way of example. Specifically, although many of the examples presented herein relate to specific combinations of devices or software elements, these elements may be combined in other ways to achieve the same purpose. Actions, elements, and features discussed only in conjunction with one embodiment are not intended to be excluded from similar roles in the embodiments or other embodiments.

Claims

1. A computer-implemented method for processing at least one image (10) of a retina of an eye (20) acquired by an ophthalmic imaging device (30), wherein: The at least one image (10) shows a texture (T) of the retina, the method comprising: Processing (S10) a first image (10-1) of the at least one image (10) using a machine learning-based noise reduction algorithm (110) to generate a denoised image (110-1) of the retina, wherein a texture (T) of the retina shown in the first image (10-1) is at least partially removed by the noise reduction algorithm (110) to generate the denoised image (110-1); and A second image (10-1; 10-2) of the at least one image (10) is combined (S20) with the denoised image (110-1) to generate at least one mixed image (40) of the retina, the at least one mixed image showing more texture (T) of the retina than the denoised image (110-1).

2. The computer-implemented method of claim 1 , wherein: The at least one mixed image (40) of the retina is generated by combining the second image (10-1; 10-2) with the denoised image (110-1) using respective weights (120) for the second image (10-1; 10-2) and the denoised image (110-1).

3. The computer-implemented method of claim 2, wherein: The at least one mixed image (40) of the retina is generated by calculating one of a weighted sum or a weighted average of the second image (10-1; 10-2) and the denoised image (110-1) using the weights (120).

4. The computer-implemented method according to claim 2 or claim 3, further comprising receiving (S30) a setting instruction (I100) from a user for setting the weight (120). s ), and using the setting indication (I s ) to set (S40) the weight (120).

5. The computer-implemented method according to any one of claims 2 to 4, further comprising generating (S50) a control signal (S51) for a display device (50) to display the at least one mixed image (40). C1 ).

6. The computer-implemented method of claim 5, wherein: A plurality of mixed images (40-1, 40-2, ..., 40-n) are generated by combining the second image (10-1; 10-2) with the denoised image (110-1) using different corresponding weights (120), and wherein a plurality of control signals (S C1 ,……,S Cn ) are generated for a display device (50) to display the plurality of mixed images (40-1, 40-2, ..., 40-n).

7. The computer-implemented method according to claim 5 or claim 6, further comprising receiving (S60) an update instruction (I 60) from a user for updating the weight (120). u ), and using the update indication (I u ) to update (S70) the weight (120).

8. A computer-implemented method according to any preceding claim, wherein: The denoising algorithm (110) is based on a convolutional neural network.

9. A computer-implemented method according to any preceding claim, wherein: The at least one image (10) of the retina of the eye (20) is at least one fundus autofluorescence image of the retina of the eye (20).

10. A computer-implemented method according to any preceding claim, wherein: The second image (10-1) is identical to the first image (10-1).

11. A non-transitory computer-readable storage medium having stored thereon a computer program (245), the computer program comprising computer-readable instructions which, when executed by a processor (220), cause the processor (220) to perform a method according to any preceding claim.

12. A data processing device (100) arranged to perform the method according to any one of claims 1 to 10.

Images