Computer-implemented method, computer program, and system for processing images

Abstract

A computer-implemented method for processing images includes: downsampling a plurality of first images having a first resolution to obtain a plurality of second images having a second resolution, the first resolution being higher than the second resolution, and each of the plurality of second images being a downsampled version of one of the plurality of first images; training an artificial neural network (ANN) model (40) to process the input images and output an output image having a higher resolution than the input images, the training data for training including pairs of images, each pair of images including one of the plurality of second images as an input to the ANN model (40) and one of the plurality of first images that corresponds to one of the plurality of second images as a desired output from the ANN model (40) when one of the plurality of second images is input to the ANN model (40); inputting at least one of the plurality of first images to the trained ANN model (40); and obtaining at least one output image from the trained ANN model (40) having a third resolution higher than the first resolution.

Description

[Technical Field]

[0001] The present application relates to computer-implemented methods, computer program products, and systems for processing images. [Background technology]

[0002] The field of super-resolution (SR) aims to increase the signal resolution beyond the sensor resolution, i.e., to approximate the signal using more sampling points, such as pixels for 2D images or voxels for 3D images. The SR concept can also be applied to 1D or 2D spectra, or any type of data with a defined spatial resolution.

[0003] Several heuristic methods are available for achieving higher-resolution approximations. For example, commonly used methods for image upsampling include nearest-neighbor interpolation, bilinear interpolation, bicubic interpolation, etc. In such methods, interpolated values can be derived from original pixel values by applying a context-independent upsampling operation. For example, one such algorithm may involve inserting a new pixel and assigning it the average value of the previously existing pixels surrounding the new pixel. These heuristic methods often produce approximations that can be perceived as "pixelated" or "blurry" because the actual high-resolution (HR) version of the image may not resemble those approximations. For example, a sharp line may not remain sharp because missing pixels along a line edge do not have values that are an interpolation between the line and its surroundings, but rather belong to either the line or its surroundings.

[0004] An alternative approach may be offered by the field of deep learning, in which a deep artificial neural network (ANN) model is trained to approximate the HR version of the input with high fidelity (see, e.g., W. Yang, X. Zhang, Y. Tian, W. Wang, and J.-H. Xue, "Deep Learning for Single Image Super-Resolution: A Brief Review," IEEE Trans. Multimed., pp. 1-1, 2019). Using an optimization algorithm, the ANN can be trained to perform this upsampling by updating its parameters to improve the approximation of the target HR image. For training, a dataset of training samples may need to be provided. Training can be either supervised or unsupervised. In supervised training, the model can access and enhance the HR version of the input image on which it is training (see, e.g., C. Dong, C. C. Loy, K. He, and X. Tang, "Learning a Deep Convolutional Network for Image Super-Resolution," 2014). For unsupervised training, the model may have access only to original resolution (OR) images (see, e.g., WO 2019 / 102476), or in most cases, HR images that are not actual HR versions of the original resolution input images (see, e.g., Y. Yuan, S. Liu, J. Zhang, Y. Zhang, C. Dong, and L. Lin, "Unsupervised Image Super-Resolution using Cycle-in-Cycle Generative Adversarial Networks," ArXiv180900437 Cs, Sep. 2018).

[0005] Compared to heuristic upsampling techniques such as those described above, ANN-based upsampling is both domain- and context-aware. For example, an ANN model can learn to upsample high-complexity images, such as checkerboards and areas with sharp lines, differently from images of clouds and areas with smooth lines. However, known ANN-based upsampling methods often require HR images to train the ANN model, which are not always readily available. Summary of the Invention

[0006] According to one aspect, the problem relates to facilitating image resolution extrapolation with improved quality. This problem is solved by the features disclosed in the independent claims. Further exemplary embodiments are defined by the dependent claims.

[0007] According to one aspect, a computer-implemented method for processing an image is provided, the method comprising: downsampling a plurality of first images having a first resolution to obtain a plurality of second images having a second resolution, the first resolution being higher than the second resolution, and each of the plurality of second images being a downsampled version of one of the plurality of first images; Training an artificial neural network (ANN) model to process input images and output output images with a higher resolution than the input images, wherein training data for the training includes pairs of images, each pair of images comprising: one of the plurality of second images as an input to the ANN model; and training, including one of the plurality of first images corresponding to one of the plurality of second images as a desired output from the ANN model when one of the plurality of second images is input to the ANN model; inputting at least one of the plurality of first images into a trained ANN model; Obtaining at least one output image from the trained ANN model, the at least one output image having a third resolution higher than the first resolution.

[0008] In this disclosure, the term "image" can be understood as data for which a spatial resolution is defined. For example, an "image" can be a 2D digital image containing pixels or a 3D digital image containing voxels. In such examples, each pixel or voxel in the image can represent an intensity or gray scale. In other examples, an "image" can be a 1D or 2D spectrum.

[0009] In various embodiments and examples described herein, an "ANN model" may comprise an ANN where the operation of the ANN is independent of the resolution of the input data and output data. In other words, an ANN model may comprise an ANN that is capable of processing input data having different resolutions and outputting data having different resolutions.

[0010] The method according to the above-described embodiment can generate an image having a target resolution higher than an image input to the ANN model (e.g., a third resolution) without using an image having the target resolution for training the ANN model. Furthermore, the use of the ANN model can improve the quality of the obtained image having the target resolution compared to using known heuristic methods such as nearest neighbor interpolation or bicubic interpolation.

[0011] In the method according to the above-described aspect, the ANN model may include a convolutional neural network.

[0012] Furthermore, in the method according to the above-described embodiment, the ANN model is a generator comprising a first ANN that processes an input image and outputs an output image having a higher resolution than the input image; a classifier comprising a second ANN for distinguishing images generated by the first ANN from real images that are considered to be desired outputs from the first ANN; The ANN model is training a generator using the training data to generate, from an input image, an output image having a higher resolution than the input image; Using the images output from the trained generator and at least some of the plurality of first images, the images input to the second ANN are: The image output from the trained generator, or training a classifier to determine whether the first image is one of a plurality of first images; Further training the generator and the discriminator, using the output from the trained classifier to update parameter values of the first ANN to increase the error rate of the second ANN; The generator and discriminator can be trained by iterating using the output from the trained generator to update the parameter values of the second ANN to reduce the error rate of the second ANN, and further training the generator and discriminator by repeating the above steps.

[0013] In this disclosure, the term "parameter values" of an ANN may be understood as values of parameters used in operations performed in the ANN. For example, "parameter values" may include values of weights of connections between nodes in the ANN.

[0014] In an exemplary embodiment in which the ANN model comprises a generator and a discriminator, the first ANN and / or the second ANN may be a convolutional neural network.

[0015] Furthermore, in an exemplary embodiment in which the ANN model comprises a generator and a classifier, the method further comprises: parameter values of the first ANN obtained by training the generator; and the updated parameter values of the first ANN obtained by further training the generator and the discriminator; The first ANN having the weighted average value as the parameter value can be used as a trained ANN model to obtain at least one output image.

[0016] Additionally, any one of the methods according to the above-mentioned aspects and various embodiments thereof may further comprise: applying an image enhancement pre-processing step to the plurality of first images prior to downsampling of the plurality of first images; The image enhancement pre-processing step is Blur, Adding noise to each pixel, Applying out-of-focus distortion, and applying motion blur.

[0017] Performing an image enhancement preprocessing step can enhance the ability of the ANN model to robustly enhance image resolution.

[0018] Further, in the method according to any one of the above-mentioned aspects and various embodiments thereof, the first resolution may be higher than the second resolution by a scaling factor (S), and the third resolution may be higher than the first resolution by a scaling factor (S).

[0019] In various aspects and embodiments described herein, the scaling factor (S) may be a real number having a value greater than 1.0. In some exemplary embodiments, the scaling factor (S) may be a value between 2.0 and 4.0. In further exemplary embodiments, the scaling factor (S) may be a value greater than 4.0.

[0020] In various aspects and embodiments described herein, the plurality of first images may include microscopic images of cells. When the plurality of first images includes microscopic images of cells, it is preferable to set the scaling factor (S) to a value less than 3.0, and in some circumstances, it may be preferable to set it to a value of 2.0 or less.

[0021] According to another aspect, there is provided a computer program product comprising computer readable instructions that, when loaded and executed on a computer, cause the computer to perform a method according to any one of the above-mentioned aspects and their various embodiments.

[0022] According to yet another aspect, there is provided a system for processing an image, the system comprising: a storage medium storing a plurality of first images having a first resolution and an ANN model, which is an artificial neural network, for processing the input images to output an output image having a higher resolution than the input images; 1. A processor, comprising: downsampling the plurality of first images to obtain a plurality of second images having a second resolution, the first resolution being higher than the second resolution, and each of the plurality of second images being a downsampled version of one of the plurality of first images; training an ANN model using training data comprising pairs of images, each pair of images comprising: one of the plurality of second images as an input to the ANN model; and training, including one of the plurality of first images corresponding to one of the plurality of second images as a desired output from the ANN model when one of the plurality of second images is input to the ANN model; inputting at least one of the plurality of first images into a trained ANN model; and a processor configured to obtain at least one output image from the trained ANN model, wherein the at least one output image has a third resolution higher than the first resolution.

[0023] In the system according to the above-described aspect, the ANN model may include a convolutional neural network.

[0024] Furthermore, in the system according to the above-described aspect, the ANN model is a generator comprising a first ANN that processes an input image and outputs an output image having a higher resolution than the input image; a classifier comprising a second ANN for distinguishing images generated by the first ANN from real images that are considered to be desired outputs from the first ANN; The ANN model is training a generator using the training data to generate, from an input image, an output image having a higher resolution than the input image; Using the images output from the trained generator and at least some of the plurality of first images, the images input to the second ANN are: The image output from the trained generator, or training a classifier to determine whether the first image is one of a plurality of first images; Further training the generator and the discriminator, using the output from the trained classifier to update parameter values of the first ANN to increase the error rate of the second ANN; The generator and discriminator can be trained by iterating using the output from the trained generator to update the parameter values of the second ANN to reduce the error rate of the second ANN, and further training the generator and discriminator by repeating the above steps.

[0025] Furthermore, when the ANN model comprises a generator and a discriminator, the processor parameter values of the first ANN obtained by training the generator; and the updated parameter values of the first ANN obtained by further training the generator and the discriminator; The first ANN having the weighted average value as the parameter value can be used as a trained ANN model to obtain at least one output image.

[0026] In a system according to any one of the above aspects and various embodiments thereof, a processor and further configured to apply an image enhancement pre-processing step to the plurality of first images prior to downsampling of the plurality of first images; The image enhancement pre-processing step is Blur, Adding noise to each pixel, Applying out-of-focus distortion, and applying motion blur.

[0027] Further, in a system according to any one of the above-described aspects and various embodiments thereof, the first resolution may be higher than the second resolution by a scaling factor (S), and the third resolution may be higher than the first resolution by a scaling factor (S).

[0028] In a system according to any one of the above-described aspects and various embodiments thereof, the plurality of first images can include microscopic images of cells.

[0029] The subject matter described herein may be implemented as a method or system, possibly in the form of one or more computer program products. The subject matter described herein may be implemented in a data signal or machine-readable medium, where the medium is embodied in one or more information carriers, such as a CD-ROM, a DVD-ROM, a semiconductor memory, or a hard disk. Such a computer program product may cause a data processing device to perform one or more of the operations described herein.

[0030] Furthermore, the subject matter described herein can also be implemented as a system including a processor and a memory coupled to the processor. The memory can encode one or more programs for causing the processor to perform one or more of the methods described herein. In some examples, the system can be a general-purpose computer system. In other examples, the system can be a special-purpose computer system, including an embedded system.

[0031] Depending on the circumstances, any one of the above-described aspects, as well as any one of the various embodiments and examples described herein, can provide one or more of the following advantages: Achieving improved quality of image resolution extrapolation compared to heuristic methods such as nearest neighbor or bicubic interpolation; - eliminating the need to use images of a target resolution (e.g., high resolution) in the training process, as is the case with known deep learning methods; - Achieving improved image quality facilitates and / or provides for improved further processing of the images, such as cell segmentation. [Brief explanation of the drawings]

[0032] The details of one or more implementations are set forth in the illustrative drawings and description that follow. Other features will become apparent from the description, drawings, and claims. However, even if embodiments are described separately, it should be understood that single features of different embodiments may be combined into further embodiments.

[0033] [Figure 1] 1 shows a schematic diagram illustrating a method according to the present disclosure. [Figure 2] 1 shows a schematic diagram of an exemplary system for processing images. [Figure 3] 3 shows a flowchart illustrating an exemplary process performed by the system shown in FIG. 2. [Figure 4] 1 shows a flowchart illustrating an exemplary process for training an ANN model in accordance with the present disclosure. [Figure 5] 1 shows a schematic diagram illustrating an exemplary generative adversarial network (GAN) model according to the methodology. [Figure 6] 6 shows a flowchart illustrating an example process for training the example ANN model shown in FIG. 5. [Figure 7] 1 shows an exemplary visualization of the training and evaluation steps using natural images. [Figure 8] 1 shows examples of natural images for comparison with the method according to the present disclosure and images obtained by bicubic interpolation. [Figure 9] 1 shows example cell images for comparison of images obtained by the method of the present disclosure and by bicubic interpolation. [Figure 10] 1 shows an example of a cell image when training an ANN model according to the present disclosure. [Figure 11] 1 shows a conceptual diagram illustrating the relationship between magnification, number of cells in the field of view, and image detail in live cell imaging. [Figure 12] 1 shows exemplary images of neutrophil-like cells undergoing NETosis, including captured images and images processed by methods according to the present disclosure. [Figure 13] 1 shows example images for comparison of images obtained by the method according to the present disclosure, nearest neighbor interpolation, and bicubic interpolation. [Figure 14] 10 shows further example images for comparison of images obtained by the method according to the present disclosure, nearest neighbor interpolation, and bicubic interpolation. [Figure 15] 1 illustrates an exemplary hardware configuration of a computer that can be used to implement at least a portion of a system according to the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0034] In the following, the embodiments will be described in detail with reference to the drawings. It will be understood that various modifications can be made to the embodiments. In particular, one or more elements of one embodiment can be combined and used in another embodiment to form a new embodiment.

[0035] While exemplary embodiments are described below with respect to processing 2D images, it should be noted that various aspects and exemplary embodiments may be applicable to other types of data for which a spatial resolution can be defined. Examples of other types of data may include, but are not limited to, 3D images, 1D and / or 2D spectra, etc.

[0036] Signal quality can be important for all analyses. In the context of image analysis, image quality and resolution can define what information is extracted from an image and / or how easy it is to extract information, both perceptually and algorithmically. For example, it is easier to see what is in a high-resolution, clear image compared to a low-resolution, blurry image. Thus, the simplest way to improve the quality of downstream analysis of a given problem can be to improve the quality of the signal used for the analysis. Various aspects and exemplary embodiments described herein can provide for the enhancement of input signals in a qualitative manner.

[0037] ANNs are commonly used to enhance image resolution (see, e.g., W. Yang, X. Zhang, Y. Tian, W. Wang, and J.-H. Xue, "Deep Learning for Single Image Super-Resolution: A Brief Review," IEEE Trans. Multimed., pp. 1-1, 2019; C. Dong, C. C. Loy, K. He, and X. Tang, "Learning a Deep Convolutional Network for Image Super-Resolution," 2014; B. Lim, S. Son, H. Kim, S. Nah, and K. M. Lee, "Enhanced Deep Residual Networks for Single Image Super-Resolution," ArXiv170702921 Cs, Jul. 2017). However, methods used today often rely on high-resolution images to train networks using low-resolution (LR)-HR image pairs, or by providing unpaired LR and HR images of the same domain as in generative adversarial networks (GANs) (see, e.g., Y. Yuan, S. Liu, J. Zhang, Y. Zhang, C. Dong, and L. Lin, "Unsupervised Image Super-Resolution using Cycle-in-Cycle Generative Adversarial Networks," ArXiv180900437 Cs, Sep. 2018).

[0038] Various aspects and exemplary embodiments described herein can enhance image resolution without requiring any HR images to participate in the training process.

[0039] FIG. 1 shows a schematic conceptual diagram illustrating a method according to the present disclosure. Referring to step A shown in FIG. 1, in some exemplary embodiments, an image having an original resolution (e.g., a first resolution) may be downsampled to an image having a lower resolution (e.g., a second resolution). Hereinafter, the image having the original resolution may also be referred to as an "OR image," and the image having the lower resolution may also be referred to as an "LR image." The downsampling may be performed, for example, by bicubic interpolation. Subsequently, referring to step B shown in FIG. 1, an ANN model 40 may be trained to reconstruct the OR image from the LR image. The ANN model 40 may have an ANN, for example, a convolutional neural network (CNN) architecture for image processing (see, for example, A. Krizhevsky, I. Sutskever, and G. Hinton, "ImageNet Classification with Deep Convolutional Neural Networks," Neural Inf. Process. Syst., vol. 25, Jan. 2012). Once training is complete, the ANN model 40 having the operations learned from upsampling the LR image to the OR image can be used to extend the OR image to a higher resolution image (e.g., a third resolution higher than the original resolution), as shown in step C of Figure 1, in other words, to increase the image resolution from the original resolution to a higher resolution. Hereinafter, the image with a higher resolution may also be referred to as an "HR image."

[0040] System Configuration 2 shows a schematic diagram of an exemplary system for image processing. The exemplary system shown in FIG. 2 includes an imaging system 10, a computing device 20, and a data storage device 30.

[0041] Imaging system 10 can be configured to capture images and provide the captured images to computing system 20. The images can be, for example, natural images of one or more scenes. In some circumstances, the images can be microscopic images of cells, in which case imaging system 10 can include a microscopic imager (not shown), such as an optical microscope, a fluorescent microscope, or an electron microscope.

[0042] The computing device 20 may be (a) a computer connected to the imaging system 10 via a wired and / or wireless communication network. The computing device 10 may receive images captured by the imaging system 10. The computing device 20 may be configured to perform methods according to various embodiments and examples described herein. The data storage device 30 may store information used by and / or generated by the computing device 20. For example, the storage device 30 may store an ANN model 40 (see, e.g., FIG. 1 ) used in methods according to various embodiments and examples described herein.

[0043] It should be noted that imaging system 10, computing device 20, and data storage device 30 may be incorporated into a single device having a single body, or may be implemented by multiple separate devices. Furthermore, computing device 20 may be implemented by multiple computers connected to each other via a wired and / or wireless communication network.

[0044] In some exemplary embodiments, computing device 20 does not necessarily receive the images to be processed directly from the imaging system 10 that captured the images. For example, computing device 20 may receive the captured images from a database system (not shown) that stores images captured by imaging system 10.

[0045] Unsupervised Image Augmentation Figure 3 shows a flowchart illustrating an exemplary process performed by the system shown in Figure 2. The exemplary process may be performed by computing device 20 shown in Figure 2. The exemplary process may be initiated, for example, when computing device 20 receives an instruction from a user via an input device (not shown) to initiate the exemplary process.

[0046] In step S10, the computing device 20 may optionally perform image enhancement preprocessing on the OR image. The image enhancement preprocessing may include, for example, one or more of blurring, adding pixel-by-pixel noise, applying out-of-focus distortion, applying motion blur, etc. The OR image may be provided (directly or indirectly) from the imaging system 10 and stored in the data storage device 30. After step S10, the process may proceed to step S20. It should be noted that step S10 is an optional step that may be skipped. If step S10 is skipped, the process may start from step S20.

[0047] In step S20, computing device 20 may downsample the OR images (e.g., stored in data storage device 30 as described above) to obtain LR images. Downsampling may be performed, for example, using bicubic interpolation. Each of the LR images obtained in step S20 may be a downsampled version of one of the OR images. After step S20, the process may proceed to step S30.

[0048] In step S30, computing device 20 may train an ANN model, for example, ANN model 40 shown in Figures 1 and 2, using the LR images and OR images. The training data for training in step S30 may include pairs of images, where each pair of images may include one of the LR images as an input to ANN model 40 and a corresponding one of the OR images as a desired output from ANN model 40, if one of the LR images is input to ANN model 40.

[0049] Figure 4 shows a flowchart illustrating a more detailed example process of training in step S30 of Figure 3. The example process of Figure 4 can begin when computing device 20 begins execution of step S30 of Figure 3.

[0050] 4, in step S300, an LR image from the training data may be input to the ANN of the ANN model 40 to obtain an output image with original resolution. After step S300, the process may proceed to step S302.

[0051] In step S302, the computing device 20 may calculate how far the output image is from the corresponding OR image. This metric may be calculated, for example, using a pixel-wise mean absolute error (MAE) or mean squared error (MSE). This metric is sometimes referred to as a "loss" in the field of deep learning. After step S302, the process may proceed to step S304.

[0052] In step S304, computing device 20 may update the ANN weights to minimize the difference between the ANN output image and the original OR image. In other words, the ANN weights may be updated to minimize the loss calculated in step S302. More specifically, for example, the weight updates may be performed using a variant of gradient descent (e.g., stochastic, batch, or mini-batch gradients are appropriate) and backpropagation algorithms on the current network weights, activations, and calculated loss to minimize the difference between the ANN output image and the original OR image.

[0053] After step S304, the computing device 20 may determine in step S306 whether to terminate the training process. For example, the determination in step S306 may be based on whether a specified number of iterations of the training steps S300, S302, and S304 have been performed. Alternatively or additionally, the determination in step S306 may be based on whether the loss of the ANN has converged. If the specified number of iterations has been performed and / or the loss of the ANN has converged, the computing device 20 may determine to terminate the training process. Otherwise, the computing device 20 may determine to continue executing the training process.

[0054] If the computing device 20 determines to continue the training process (No in step S306), the process may return to step S300. If the computing device 20 determines to end the training process (Yes in step S306), the process shown in FIG. 4 may end.

[0055] After the training process shown in FIG. 4 is complete, step S30 of FIG. 3 may end and computing device 20 may proceed to perform step S40 of FIG.

[0056] 3 , in step S40, the computing device 20 may input at least one of the OR images (e.g., stored in the data storage device 30) to the trained ANN model 40. At least one of the OR images may be expanded by the ANN model 40 to a target resolution higher than the original resolution, and at least one HR image corresponding to at least one of the OR images may be output from the ANN model 40.

[0057] In step S50, the computing device 20 may obtain at least one HR image from the trained ANN model. The HR image output from the trained ANN model may have higher quality (e.g., resolution) compared to the OR image input to the ANN model.

[0058] The process shown in FIG. 3 may end after step S50.

[0059] In the exemplary process shown in Figure 3, steps S10 and S20 may correspond to step A of Figure 1, and step S30 may correspond to step B of Figure 1. Additionally, steps S40 and S50 of Figure 3 may correspond to step C of Figure 1.

[0060] In some exemplary embodiments, the ANN model 40 may be constructed and trained according to a generative adversarial network (GAN) methodology (see, e.g., Goodfellow, Ian; Pouget-Abadie, Jean; Mirza, Mehdi; Xu, Bing; Warde-Farley, David; Ozair, Sherjil; Courville, Aaron; Bengio, Yoshua, "Generative Adversarial Networks," Proceedings of the International Conference on Neural Information Processing Systems (NIPS 2014), pp. 2672-2680, available online at https: / / arxiv.org / abs / 1406.2661). In such exemplary embodiments, the ANN model 40 may be constructed and trained as described below with reference to FIGS. 5 and 6.

[0061] FIG. 5 illustrates an exemplary configuration of an ANN model 40 in an exemplary embodiment in which a GAN methodology is applied for the ANN model 40 .

[0062] Referring to FIG. 5, the ANN model 40 may comprise a generator 42 having a first ANN 420 and a classifier 44 having a second ANN 440 .

[0063] The first ANN 420 of the generator 42 may be for processing an input image and outputting an output image having a higher resolution than the input image. The first ANN 420 may be a CNN and may be trained in the same manner as the exemplary training process shown in FIG.

[0064] The second ANN 440 of the classifier 44 may be for distinguishing the image generated by the first ANN 420 from a real image that is considered the desired output from the first ANN 420. In other words, the purpose of the classifier 44 with the second ANN 440 may be to classify whether the input provided to the classifier 44 is real or synthetic. The second ANN 440 may be a CNN.

[0065] 5, an LR image can be input to the generator 42 and processed by the first ANN 420, and then an OR image can be output from the generator 42. These OR images output from the generator 42 and the original OR images, in other words, the OR images from which the LR images were generated by downsampling, can be used to train the classifier 44 to determine whether the input to the classifier 44 is the output from the generator 42 or the original OR image.

[0066] Figure 6 shows a flowchart of an exemplary training process for an ANN model 40 including a generator 42 and a classifier 44. The process shown in Figure 6 may be performed by the computing device 20, for example, as part of the training step S30 shown in Figure 3. Furthermore, in some circumstances, the generator 42 may be trained by the exemplary process shown in Figure 4 before commencing the exemplary process shown in Figure 6.

[0067] 6, in step S310, the computing device may pre-train the classifier 44 to distinguish between the output from the generator 42 and the original OR image. For example, the OR image output from the generator 42 and the original OR image corresponding to the LR image input to the generator 42 to obtain the OR image may be used as training data for the pre-training in step S310 (see also FIG. 5). After step S310, the process may proceed to step S312.

[0068] In step S312, the computing device 20 may use output from the trained classifier 44 to update parameter values of the first ANN 420 of the generator 42 to increase the error rate of the second ANN 440 of the classifier 44. In other words, the parameter values of the first ANN 420 may be updated to make the classifier 44 less accurate. After step S312, the process may proceed to step S314.

[0069] In step S314, the computing device 20 may use the output from the trained generator 42 (using the updated parameter values in step S312) to update the parameter values of the second ANN 440 of the classifier 44 to reduce the error rate of the second ANN 440. In other words, the parameter values of the second ANN 440 may be updated to make it more difficult for the generator 42 to produce an output that the classifier 44 cannot distinguish from the original image. After step S314, the process may proceed to step S316.

[0070] In step S316, the computing device 20 may determine whether to terminate the training process. For example, the computing device 20 may determine whether one or more conditions for terminating the training process are met. The one or more conditions may include, for example, whether the update steps S312 and S314 have been iterated a specified number of times, whether a metric used in the update steps S312 and / or S314 has converged, etc. Alternatively or additionally, in some situations, the one or more conditions may include convergence of a secondary metric other than the metric used in the update steps S312 and / or S314, such as a peak signal-to-noise ratio (PSNR). If the one or more conditions for terminating the training process are met, the computing device 20 may determine to terminate the training process. If not, the computing device 20 may determine to continue the training process.

[0071] If the computing device 20 determines to continue the training process (No at step S316), the process may return to step S312 and the update steps S312 and S314 may be repeated. If the computing device 20 determines to terminate the training process (Yes at step S316), the process may proceed to step S318.

[0072] In step S318, the computing device 20 may optionally calculate a weighted average between the parameter values of the trained generator 42 used in S310 and the updated parameter values of the generator 42 after the computing device 20 determines to terminate the training process in step S316. The first ANN 420 having the calculated weighted average may be used as the ANN model 40 for generating an HR image from an OR image. It should be noted that step S318 is an optional step that may be skipped.

[0073] After a YES determination is made in step S316 or after step S318, the process shown in FIG. 6 may end.

[0074] In an exemplary embodiment in which the ANN model 40 is trained in the exemplary process shown in FIG. 6 , the first ANN 420 of the generator 42 having parameter values as a result of performing the training process of FIG. 6 can be used as the ANN of the ANN model 40 for generating an HR image from an OR image (see also, e.g., step S50 of FIG. 3 ).

[0075] It will be appreciated that in an exemplary embodiment of the GAN methodology as described above with reference to FIGS. 5 and 6, the first ANN 420 of the generator 420 is trained, for example, by a training process such as that shown in FIG. 4, and then further fine-tuned by a training process as shown in FIG. 6.

[0076] The image augmentation process as described above with reference to Figures 1 to 6 can be considered an unsupervised and / or self-supervised method in the sense that HR images with the target resolution are not required to train the ANN model 40.

[0077] experiment The following provides the setup and results of experiments conducted by the inventors for the method according to the present disclosure. In the experiments, ANN models were trained at different resolution scales (2x, 3x and 4x) and different downsampling methods were tested.

[0078] a) Dataset Two different datasets were used for the experiments: A natural image dataset consisting of 900 images from the DIV2K dataset (e.g., R. Timofte, S. Gu, L. Van Gool, L. Zhang, and M.-H. Yang, "NTIRE 2018 Challenge on Single Image Super-Resolution: Methods and Results," in 2018 IEEE / CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), June 2018, pp. 965-96511, doi:10.1109 / CVPRW.2018.00130; E. Agustsson and R. Timofte, "NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study," in 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, HI, USA, July 2017, pp. 1122-1131, doi:10.1109 / CVPRW.2017.150), and - A cell microscopy image dataset with 3816 images captured by an IncuCyte® S3 at 10x magnification.

[0079] The DIV2K dataset was divided into 750 training images, 50 validation images, and 100 test images. The cell microscopy image dataset was divided into 2124 training images, 531 validation images, and 1160 test images.

[0080] b) Experimental setup The experimental setup was different for two datasets: a natural image dataset and a cell microscopy image dataset. In the general workflow, we assumed that original-resolution (OR) images were available, with the goal of improving OR images to high resolution (HR) without access to the actual HR images. If the magnitude of the resolution improvement from OR to HR is represented by a scaling factor S, then equivalent downscaling (i.e., downsampling) by a factor of S was performed on the OR images to obtain LR images. An ANN model was trained to recreate OR images from LR images and then used to extrapolate the data in the OR images to HR.

[0081] Three ANN models were trained on a cell microscopy image dataset, and the extrapolation step was compared with bicubic interpolation (BI) upscaling. The ANN models were trained with scaling factors of 2x, 3x, and 4x, using nearest neighbor (NN) downscaling for LR generation.

[0082] For natural images (DIV2K), a slightly different approach was taken. Instead of using the original images as the OR resolution, the original images in the DIV2K dataset were downscaled by BI to create a downscaled OR dataset. The downscaled OR images were used as OR images to train the ANN model. Once the ANN model was fully trained, the downscaled OR test images were upscaled to HR images using the ANN model. The peak signal-to-noise ratio (PSNR) and mean absolute error (MAE) were calculated between the extrapolated HR images output from the ANN model and the ground truth HR images, which were the original images in the DIV2K dataset. This evaluation method can potentially yield comparable metrics. However, the information loss for LR images can increase as the scaling factor S increases, which can make it more difficult to train the ANN model. If the original images are considered as images with a target resolution HR, the low-resolution LR of the training input images can be calculated as LR = HR / S. 2 Therefore, to obtain the LR image of the ANN model, which increases the resolution by a factor of 4, the original image had to be downscaled by a factor of 16.

[0083] As the architecture of the ANN model in our experiments, we adopted a ResNet network called the Enhanced Deep Super-Resolution Network (EDSR) (see, for example, B. Lim, S. Son, H. Kim, S. Nah, and K. M. Lee, "Enhanced Deep Residual Networks for Single Image Super-Resolution," July 2017, available online at https: / / arxiv.org / abs / 1707.02921).

[0084] Figure 7 shows an example visualization of different resolutions for an ANN model using a scaling factor S = 3. More specifically, Figure 7 shows an example visualization of the training and evaluation steps using natural images with an ANN model that increases the image resolution by a factor of 3. The original HR images were downscaled to simulated OR images, which were further downscaled to LR images for training purposes.

[0085] For the DIV2K image dataset, six different experiments were performed: one ANN model with BI downscaling and one ANN model with NN downscaling, for each of the scaling factors S = 2, 3, and 4. Furthermore, for each combination of scaling factor and downscaling method, a non-converged version of the ANN model was saved after 50 epochs, resulting in a total of 12 ANN models.

[0086] In all experiments, the ANN model was optimized to minimize the MAE loss using the Adam optimizer (see, e.g., D.P. Kingma and J. Ba, "Adam: A method for stochastic optimization," ArXiv Prepr. ArXiv14126980, 2014). -4 The ANN models were trained for 1000 epochs with a learning rate of 0. For the cell microscopy dataset, the ANN model used a batch size of 5, and for the DIV2K image dataset, the ANN model used a batch size of 12.

[0087] c) Result Table 1 shows the peak signal-to-noise ratio (PSNR) results for the ANN model according to the present disclosure, as well as comparative examples of a non-converged model and known heuristic methods, BI and NN. In Table 1, the BI and NN designations for the ANN model and non-converged model represent downscaling methods, while the BI and NN designations for the heuristic methods represent upscaling methods. The term "non-converged model" in Table 1 should be understood as the non-converged version of the ANN model trained for 50 epochs. PSNR was calculated between the extrapolated high-resolution image and the corresponding ground truth image.

[0088] [Table 1]

[0089] As can be seen from Table 1, in terms of PSNR (higher is better), the ANN model according to the present disclosure outperforms known heuristic methods, BI and NN, and non-converged models. However, there does not appear to be a clear difference between fully converged models depending on which downscaling method is used. The results of the mean absolute error (MAE) metric, shown in Table 2 below, follow the same pattern.

[0090] [Table 2]

[0091] Table 2 shows MAE comparisons of the ANN model according to the present disclosure, as well as comparisons of non-converged models and known heuristic methods, BI and NN. Errors were calculated as the mean absolute pixel difference between the extrapolated high-resolution image and the corresponding ground truth image. Also, in Table 2, the BI and NN designations for the ANN model and non-converged model represent downscaling methods, while the BI and NN designations for the heuristic methods represent upscaling methods. Furthermore, the term "non-converged model" in Table 2 should be understood as the non-converged version of the ANN model trained for 50 epochs.

[0092] FIG. 8 shows example natural images for comparing images obtained from an ANN model according to the present disclosure with images obtained by bicubic interpolation (BI). FIG. 8 shows exemplary outputs from an ANN model trained with 2x, 3x, and 4x resolution enhancements (corresponding to the image rows in FIG. 8). In FIG. 8, the images in column (a) are input images, column (b) are output images from the ANN model, column (c) is a ground truth image, and column (d) is the resulting image with BI upscaling. The ANN model took the input image shown in column (a) of FIG. 8 as an OR input image and increased the resolution to HR by the respective scaling factors. Compared to BI upscaling, the output image of the ANN model appears sharper and therefore more similar to the ground truth image. As the input resolution decreases, the ability to recreate the ground truth HR image also decreases.

[0093] 9 shows example cell images for comparing images obtained from an ANN model according to the present disclosure with images obtained by bicubic interpolation. The image rows in FIG. 9 correspond to resolution increases of 2x, 3x, and 4x (i.e., scaling factors of 2, 3, and 4). In FIG. 9, the images in column (a) show input images, column (b) show output images from the ANN model, and column (c) show images upscaled by BI upscaling as a comparative example.

[0094] As can be seen in Figure 9, the ANN model clearly improved the quality of the cell microscopy images when using a 2x resolution enhancement and BI resolution enhancement compared to the input image. However, as the resolution increased by 3x and 4x, the quality of the ANN model output did not improve; rather, the opposite was true. The ANN models with 3x and 4x increased resolution were still able to capture cell boundaries.

[0095] FIG. 10 shows examples of cell images used in training an ANN model according to the present disclosure. Specifically, FIG. 10 illustrates the ability of different ANN models to recreate images with higher target resolutions from images with lower resolutions. The ANN models were trained at 2x, 3x, and 4x resolution increases (corresponding to the image rows in FIG. 10 ), and the output image from the ANN model (see column (b) in FIG. 10 ) can be compared with an image obtained by bicubic interpolation upscaling (see column (d) in FIG. 10 ) derived from the target image (see column (c), ground truth) and the input image (see column (a) in FIG. 10 ).

[0096] From Figure 10, it appears that training the ANN model (LR to OR) causes the LR image to suffer a sharp information loss at the 3x and 4x settings. Much of the detail in the ground truth image is missing from its LR counterpart. In this setting, the reconstruction was even much better than BI.

[0097] Live cell imaging applications In live-cell imaging, image quality can be important for making accurate decisions. However, increasing resolution often requires reducing the field of view (FOV) by zooming in. Ideally, instruments capture high resolution over a wide field of view. However, this may not be feasible due to various limitations ranging from instrument throughput, hardware limitations, and hard drive storage limitations.

[0098] Therefore, image acquisition at higher magnifications can result in clearer and more detailed images, while lower magnifications can allow for capturing a larger number of cells within the FOV. As the magnification increases, the number of cells within the FOV decreases, as shown in Figure 11.

[0099] By increasing the effective resolution, more detailed images can be acquired without sacrificing the number of cells in the FOV. Thus, rare cellular events can be more likely to be captured, increasing the statistical relevance of the overall population study. Replicating this with traditional methods (e.g., by acquiring more high-magnification images and combining them) can be time-consuming and tedious, and can potentially damage cells due to increased phototoxicity.

[0100] Applying the disclosed method to images acquired at 20x magnification can improve images at the cellular level. Resolution of cell edges can be enhanced, allowing for more accurate cell segmentation, and subcellular features, for example, can have improved texture and clarity, reflecting biological structures. While accurate label-free quantification of intracellular processes in 20x images is not yet possible, improving image quality can allow for clearer visualization of intracellular organelles, such as the nucleus. Furthermore, texture, periphery, and three-dimensional features can be more accurately quantified. This is demonstrated in Figure 12, which shows a magnified image of a cell undergoing NETosis, a process in which the nucleus enlarges and changes shape before releasing its contents into the cytoplasm.

[0101] Figure 12 shows a neutrophil-like cell undergoing NETosis. The top row of images in Figure 12 shows images acquired at 20x magnification from 0 to 90 minutes (columns (a), (b), (c), and (d)), while the bottom row of images in Figure 12 shows the same images processed using an ANN model trained according to the present disclosure. As can be seen in Figure 12, the processed image at 0 minutes has more visible contours, consistent with this type of cell, and the cell contents have increased texture. In other words, the visible texture within the cell, the outer cell boundary, and the cell shape are more distinct in the processed image at 0 minutes. At 30 minutes, the nucleus is enlarged and is also more prominent in the processed image. For example, in the processed image at 30 minutes, the outer cell boundary is more distinct, and the nucleus is more prominent, reflecting the biological process of DNA decondensation. At 60 and 90 minutes, the processed images again show better texture in areas where the nucleus has expelled its contents into the cytoplasm and a much clearer cell boundary. For example, in the 60-minute processed image, the intracellular texture is more visible and the cell outline is clearer. Furthermore, in the 90-minute processed image, the outer cell boundary is more clearly defined, the texture and cell contents are visible, and the cell shape is more clearly defined.

[0102] Further experiments on live cell imaging The following provides an exemplary workflow in which images are acquired at standard quality and then augmented in silico using an ANN model according to the present disclosure.

[0103] For these experiments, a dataset of phase contrast images at 10x magnification of cell cultures was provided by Essen Instruments / Inc. dba Essen Bioscience, Inc. The dataset consisted of images obtained from an IncuCyte® S3 and consisted of seven different cell types: A549 HeLa HT1080 HUVEC Jurkat MDAMB231 PC3 SKOV3

[0104] The dataset consisted of 1,746 images and was divided into training, validation, and test datasets. The training dataset consisted of 1,037 images from all cell types, and the validation dataset had 259 images. The test dataset consisted of 450 images from all cell cultures, with some images coming from the same well but separated by wells in the training and validation datasets.

[0105] A CNN with the ResNet architecture, as used in B. Lim, S. Son, H. Kim, S. Nah, and K. M. Lee, "Enhanced Deep Residual Networks for Single Image Super-Resolution," ArXiv1707.02921 Cs, July 2017 (available online at https: / / arxiv.org / abs / 1707.02921), was trained by taking training image patches, downsampling the image patches to half the resolution using bicubic interpolation, and then recreating the original image patches and tasking the network. This was done with the Adam optimizer for 2000 epochs, a batch size of 20, and a learning rate of 0.0001. Pixel mean absolute error (MAE) was used as the loss function, and the network was evaluated by both MAE, peak signal-to-noise ratio (PSNR), and visual inspection of image quality on validation data.

[0106] Once the network was fully trained, the model with the highest MAE and PSNR was selected and further validated against the test data to ensure generalizability. In this final step, the network was modified to take the original images as input instead of the downsampled training and validation images. Thus, the resolution of the test images was increased from 10x to 20x, and these HR images were then compared with upsampled images from the test set using bicubic and nearest-neighbor interpolation. Visual inspection showed higher fidelity in the 20x images generated by our model than in the comparison.

[0107] 13 and 14 show image comparisons of image augmentation according to the ANN model of the present disclosure (see FIG. 13 and FIG. 14(b)) compared to the original image (see FIG. 13(a) and FIG. 14(d)), nearest neighbor interpolation (see FIG. 13 and FIG. 14(c)), and bicubic interpolation (see FIG. 13(d) and FIG. 14(a)).

[0108] Possible further implementations As can be seen from the experimental results described above with reference to the figures, if the information loss to the LR is too large, for example as shown in Figures 9 and 10, the ANN model according to the present disclosure may not be able to learn how to reconstruct finer details of the image at the target resolution (e.g., HR). This problem does not arise in the same way when actually accessing LR-HR pairs where the model can learn the mapping directly.

[0109] In this regard, one possible implementation of the present disclosure to achieve higher resolution improvement can be to use the method disclosed herein as a pre-training technique for later fine-tuning the model on actual HR images. An advantage of this possible implementation can be that the amount of HR images required can be smaller than without pre-training.

[0110] In some situations, the information loss to LR can also be somewhat mitigated by alternative methods of downscaling that preserve information about important features the model wants to learn. The importance may not necessarily be to have an exact replica of what the actual LR image looks like, but rather to preserve information about such important features. Thus, for example, instead of downscaling OR images using a BI or NN as described above, an additional ANN can be used and trained to downscale OR images to LR images, and then an ANN model according to the present disclosure can be trained to restore OR images from LR images. The additional ANN for downscaling OR images can be trained to preserve information used in the upscaling task of the ANN model according to the present disclosure.

[0111] The ANN model according to the present disclosure appears to perform significantly better on natural images than on cellular microscopy images with a 3x and 4x resolution increase. This may be because the model is able to pick up and learn more features to reconstruct in natural images than in cellular microscopy images. Natural images typically contain relatively few objects, most of which have large textures, whereas cellular images often contain many small objects that largely disappear when the resolution is reduced too much. This may further support the idea of the importance of preserving information intact in the downsampling step.

[0112] Hardware Configuration FIG. 15 illustrates an exemplary hardware configuration of a computer that can be used to implement at least a portion of the above-described system. For example, the computing device 20 illustrated in FIG. 1 can be realized by the computer 7 illustrated in FIG. 15. The computer 7 illustrated in FIG. 15 includes a central processing unit (CPU) 70, a system memory 72, a network interface 74, a hard disk drive (HDD) interface 76, an external disk drive interface 78, and an input / output (I / O) interface 80. These components of the computer are coupled to each other via a system bus 82. The CPU 70 can perform arithmetic, logical, and / or control operations by accessing the system memory 72. The system memory 72 can store information and / or instructions for use in conjunction with the CPU 70. The system memory 72 can include volatile and nonvolatile memory, such as a random access memory (RAM) 720 and a read-only memory (ROM) 722. A basic input / output system (BIOS), containing the basic routines that help to transfer information between elements within the computer 7, such as during start-up, can be stored in the ROM 722. System bus 82 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures.

[0113] The computer may include a network interface 74 for communicating with other computers and / or devices over a network.

[0114] Additionally, the computer may include a hard disk drive (HDD) 84 for reading from and writing to a hard disk (not shown) and an external disk drive 86 for reading from and writing to a removable disk (not shown). The removable disk may be a magnetic disk for a magnetic disk drive or an optical disk, such as a CD-ROM for an optical disk drive. The HDD 84 and external disk drive 86 are connected to the system bus 82 by an HDD interface 76 and an external disk drive interface 78, respectively. The drives and their associated computer-readable media provide non-volatile storage of computer-readable instructions, data structures, program modules, and other data for the general-purpose computer. The data structures may include associated data for implementing the exemplary methods and variations thereof described herein. The associated data may be organized in a database, for example, a relational or object database.

[0115] Although the exemplary environment described herein uses a hard disk (not shown) and an external disk (not shown), it should be understood by those skilled in the art that other types of computer-readable media capable of storing data accessible by a computer, such as magnetic cassettes, flash memory cards, digital video disks, random access memory, read-only memory, etc., can also be used in the exemplary operating environment.

[0116] A number of program modules may be stored on the hard disk, external disk, ROM 722, or RAM 720, including an operating system (not shown), one or more application programs 7202, other program modules (not shown), and program data 7204. The application programs may include at least some of the functionality described above.

[0117] The computer 7 can be connected to an input device 92, such as a mouse and / or a keyboard, and a display device 94, such as a liquid crystal display, via corresponding I / O interfaces 80a and 80b and a system bus 82. If the computer 7 is implemented as a tablet computer, for example, a touch panel that displays information and accepts input may be connected to the computer 7 via corresponding I / O interfaces and the system bus 82. Furthermore, in some examples, although not shown in FIG. 15 , the computer 7 may further be connected to an imaging device, such as a printer and / or a camera, via corresponding I / O interfaces and the system bus 82.

[0118] In addition to, or as an alternative to, an implementation using a computer 7 such as that shown in Figure 15, some or all of the functionality of the exemplary embodiments described herein may be implemented as one or more hardware circuits. Examples of such hardware circuits may include, but are not limited to, large scale integrated circuits (LSIs), reduced instruction set circuits (RISCs), application specific integrated circuits (ASICs), and field programmable gate arrays (FPGAs).

Claims

1. 1. A computer-implemented method for processing an image, comprising: downsampling a plurality of first images having a first resolution to obtain a plurality of second images having a second resolution, the first resolution being higher than the second resolution, and each of the plurality of second images being a downsampled version of one of the plurality of first images; Training an artificial neural network (ANN) model (40) to process input images and output output images having a higher resolution than the input images, wherein training data for said training includes pairs of images, each pair of images comprising: one of the plurality of second images as an input to the ANN model (40); and training the ANN model (40) to include one of the plurality of first images corresponding to the one of the plurality of second images as a desired output from the ANN model (40) when the one of the plurality of second images is input to the ANN model (40); inputting at least one of the plurality of first images into the trained ANN model (40); obtaining at least one output image from the trained ANN model (40) having a third resolution higher than the first resolution.

2. The method of claim 1 , wherein the ANN model (40) comprises a convolutional neural network.

3. The ANN model (40) a generator (420) comprising a first ANN (420) that processes the input image and outputs the output image at a higher resolution than the input image; a classifier (44) comprising a second ANN (440) for distinguishing images generated by the first ANN (420) from real images that are considered to be desired outputs from the first ANN (420); The ANN model (40) training the generator (42) using the training data to generate, from the input image, the output image having a higher resolution than the input image; image input to the second ANN (440) using images output from the trained generator and at least some of the plurality of first images: an image output from the trained generator (42); or training the classifier (44) to determine whether the image is one of a plurality of first images; Further training the generator (42) and the classifier (44), using the output from the trained classifier (44) to update parameter values of the first ANN (420) to increase the error rate of the second ANN (440); further training the generator (42) and the classifier (44) by repeating the steps of: using the output from the trained generator to update parameter values of the second ANN (440) so as to reduce the error rate of the second ANN (440); The method of claim 1 or 2, wherein the training is performed by

4. the parameter values of the first ANN obtained by the step of training the generator (42); the updated parameter values of the first ANN (420) obtained by the step of further training the generator (42) and the discriminator (44); and further comprising calculating a weighted average between 4. The method of claim 3, wherein the first ANN (420) having the weighted average values as the parameter values is used as the trained ANN model (40) to obtain the at least one output image.

5. applying an image enhancement pre-processing step to the plurality of first images prior to downsampling of the plurality of first images; the image enhancement pre-processing step comprises: Blur, Adding noise to each pixel, Applying out-of-focus distortion, application of motion blur; 5. The method according to any one of claims 1 to 4.

6. 6. The method of claim 1, wherein the first resolution is higher than the second resolution by a scaling factor (S), and the third resolution is higher than the first resolution by the scaling factor (S).

7. The method of claim 1 , wherein the plurality of first images comprises microscopic images of cells.

8. A computer program comprising computer readable instructions which, when loaded and executed on a computer, cause the computer to carry out the method of any one of claims 1 to 7.

9. 1. A system for processing an image, comprising: a storage medium storing a plurality of first images having a first resolution and an artificial neural network (ANN) model (40) that processes the input images and outputs an output image having a higher resolution than the input images; 1. A processor, comprising: downsampling the plurality of first images to obtain a plurality of second images having a second resolution, the first resolution being higher than the second resolution, and each of the plurality of second images being a downsampled version of one of the plurality of first images; training the ANN model (40) using training data comprising pairs of images, each pair of images comprising: one of the plurality of second images as an input to the ANN model (40); and one of the plurality of first images corresponding to said one of the plurality of second images as a desired output from said ANN model (40) when said one of said plurality of second images is input to said ANN model (40); training, including inputting at least one of the plurality of first images into the trained ANN model (40); and obtaining from the trained ANN model (40) at least one output image having a third resolution higher than the first resolution.

10. The system of claim 9 , wherein the ANN model (40) comprises a convolutional neural network.

11. The ANN model (40) a generator (42) comprising a first ANN (420) that processes the input image and outputs the output image at a higher resolution than the input image; a classifier (44) comprising a second ANN (440) for distinguishing images generated by the first ANN (420) from real images that are considered to be desired outputs from the first ANN (420); The ANN model is training the generator (42) using the training data to generate, from the input image, the output image having a higher resolution than the input image; image input to the second ANN (440) using the images output from the trained generator (42) and at least some of the plurality of first images, an image output from the trained generator (42); or training the classifier (44) to determine whether the image is one of a plurality of first images; further training the generator and the classifier, using the output from the trained classifier to update parameter values of the first ANN (420) to increase the error rate of the second ANN (440); further training the generator and the classifier by repeating the steps of: using outputs from the trained generator to update parameter values of the second ANN (440) so as to reduce the error rate of the second ANN (440); 11. The system of claim 9 or 10, trained by:

12. The processor further comprises: the parameter values of the first ANN (420) obtained by the step of training the generator (42); the updated parameter values of the first ANN (420) obtained by the step of further training the generator and the discriminator (44); configured to calculate a weighted average between 12. The system of claim 11, wherein the first ANN (420) having the weighted average values as the parameter values is used as the trained ANN model (40) to obtain the at least one output image.

13. The processor further comprises: configured to apply an image enhancement pre-processing step to the plurality of first images prior to downsampling of the plurality of first images; the image enhancement pre-processing step comprises: Blur, Adding noise to each pixel, Applying out-of-focus distortion, 13. The system of claim 9, wherein the method further comprises one or more of: applying motion blur;

14. 14. The system of claim 9, wherein the first resolution is higher than the second resolution by a scaling factor (S), and the third resolution is higher than the first resolution by the scaling factor (S).

15. The system of claim 9 , wherein the plurality of first images comprises microscopic images of cells.

Images