Computer-implemented method, computer program product, and system for processing images

By downsampling the original resolution image and training the artificial neural network model, using the combined model of generator and discriminator, the problem of high-resolution image training in the prior art is solved, and the problem of no target resolution is achieved high-resolution image generation improves the quality of image resolution extrapolation.

CN115867933BActive Publication Date: 2024-11-01SARTORIUS STEDIM DATA ANALYTICS AB
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202180049669.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2020-07-08
Filing Date
2021-07-08
Publication Date
2024-11-01
Estimated Expiration
2041-07-08

AI Technical Summary

Technical Problem

In the existing super-resolution image processing methods, deep learning-based methods require high-resolution images for training, and it is difficult to generate high-resolution images without target resolution images.

Method used

By downsampling the original resolution image, an artificial neural network model is trained to generate high-resolution images from low-resolution images, iteratively trained using a combined model of the generator and discriminator, and the parameters are updated to improve the quality of the output image.

Benefits of technology

The generation of resolution images higher than the input image without the need for target resolution images is achieved, improving the quality of image resolution extrapolation and avoiding the need for training with high-resolution images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure BDA0004040122780000181
    Figure BDA0004040122780000181
  • Figure BDA0004040122780000191
    Figure BDA0004040122780000191
  • Figure HDA0004040122790000011
    Figure HDA0004040122790000011
Patent Text Reader

Abstract

A computer-implemented method for processing an image is provided. The method comprises: downsampling a plurality of first images having a first resolution to obtain a plurality of second images having a second resolution, wherein the first resolution is higher than the second resolution, and each of the plurality of second images is a downsampled version of a first image in the plurality of first images; training an artificial neural network (ANN) model (40) to process an input image and output an output image having a higher resolution than the input image, wherein training data for training comprises image pairs, each image pair comprising: a second image in the plurality of second images as an input to the ANN model (40); and, when a second image in the plurality of second images is input to the ANN model (40), a first image in the plurality of first images corresponding to the second image in the plurality of second images as a desired output from the ANN model (40); inputting at least one first image in the plurality of first images to the trained ANN model (40); and obtaining at least one output image from the trained ANN model (40), wherein the at least one output image has a third resolution higher than the first resolution.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to a computer-implemented method, a computer program product and a system for processing images. Background Art

[0002] The goal of the field of super-resolution (SR) is to increase the signal resolution above the sensor resolution. In other words, the goal is to approximate the signal using more sampling points, such as a greater number of pixels in the case of 2D images, or more voxels in the case of 3D. The concept of SR can also be applied to 1D or 2D spectra or any type of data for which a spatial resolution is defined.

[0003] Several heuristic methods can be used to achieve higher resolution approximations. For example, methods commonly used for upsampling images may include nearest neighbor interpolation, bilinear interpolation, bicubic interpolation, etc. In such methods, interpolation of values ​​can be obtained from original pixel values ​​by applying an upsampling operation that is independent of context. For example, one such algorithm may include inserting new pixels and assigning the new pixels the average value of the pixels that previously existed around the new pixels. These heuristic methods typically produce approximations that may be perceived as "pixelated" or "blurred" because the actual high-resolution (HR) version of the image may not look like those approximations. For example, sharp lines may not remain sharp because missing pixels along the edges of the lines may not have values ​​that are interpolated between the line and the surroundings, but rather belong to the line or the surroundings.

[0004] Alternative approaches may be provided by the field of deep learning, where deep artificial neural network (ANN) models are trained to approximate an HR version of the input with high fidelity (e.g., see W. Yang, X. Zhang, Y. Tian, ​​W. Wang, and J.-H. Xue, “Deep Learning for Single Image Super-Resolution: A Brief Review,” IEEE Transactions on Multimedia, pp. 1-1, 2019). Using an optimization algorithm, the ANN can be trained to perform this upsampling by updating its parameters in a way that improves the approximation of the target HR image. For training, it may be necessary to provide a dataset of training samples. Training can be supervised or unsupervised. In supervised training, the model has access to the HR version of the input image that the model is being trained to enhance (e.g., see C. Dong, C. C. Loy, K. He, and X. Tang, “Learning a Deep Convolutional Network for Image Super-Resolution,” 2014). In the case of unsupervised training, the model may have access only to original resolution (OR) images (see, e.g., WO2019 / 102476A2), or in most cases to HR images that are actual HR versions of the input images that are not original resolution (see, e.g., Y. Yuan, S. Liu, J. Zhang, Y. Zhang, C. Dong, and L. Lin, “Unsupervised Image Super-Resolution using Cycle-in-Cycle Generative Adversarial Networks,” ArXiv180900437Cs, September 2018).

[0005] Compared to the heuristic upsampling techniques described above, ANN-based upsampling can be domain-aware and context-aware. For example, an ANN model can learn high-complexity upsampled images such as checkerboards and sharp-line regions differently from images of clouds and regions with smooth lines. However, in known ANN-based upsampling methods, training the ANN model usually requires HR images, which are not always easy to obtain. Summary of the invention

[0006] According to one aspect, the problem relates to facilitating image resolution extrapolation with improved quality. This problem is solved by the features disclosed in the independent claims. Furthermore, exemplary embodiments are defined by the dependent claims.

[0007] According to one aspect, a computer-implemented method for processing an image is provided. The method comprises:

[0008] downsampling a plurality of first images having a first resolution to obtain a plurality of second images having a second resolution, wherein the first resolution is higher than the second resolution, and each of the plurality of second images is a downsampled version of one of the plurality of first images;

[0009] An artificial neural network (ANN) model is trained to process an input image and output an output image having a higher resolution than the input image, wherein training data for the training includes image pairs, each image pair including:

[0010] a second image from the plurality of second images as an input to the ANN model; and

[0011] In the case where the one second image among the plurality of second images is input to the ANN model, a first image among the plurality of first images corresponding to the one second image among the plurality of second images as a desired output from the ANN model;

[0012] inputting at least one first image of the plurality of first images to a trained ANN model; and

[0013] At least one output image is obtained from the trained ANN model, the at least one output image having a third resolution higher than the first resolution.

[0014] In the present disclosure, the term "image" may be understood as data for which a spatial resolution is defined. For example, an "image" may be a 2D digital image comprising pixels or a 3D digital image comprising voxels. In such examples, each pixel or voxel in the image may indicate an intensity or grayscale level. In other examples, an "image" may be a 1D or 2D spectrum.

[0015] In various embodiments and examples described herein, an "ANN model" may include an ANN where the operation of the ANN does not depend on the resolution of the input data and the output data. In other words, the ANN model may include an ANN that can process input data with different resolutions and output data with different resolutions.

[0016] The method according to the above aspects can enable the generation of an image having a target resolution (e.g., a third resolution) higher than the image input to the ANN model without using the image having the target resolution to train the ANN model. In addition, using the ANN model can improve the quality of the obtained image having the target resolution compared to using known heuristic methods (such as nearest neighbor interpolation or bicubic interpolation).

[0017] In the method according to the above aspect, the ANN model may include a convolutional neural network.

[0018] Furthermore, in the method according to the above aspect, the ANN model may include:

[0019] a generator comprising a first ANN for processing the input image and outputting the output image having a higher resolution than the input image; and

[0020] a discriminator comprising a second ANN for distinguishing images generated by the first ANN from actual images believed to be desired outputs from the first ANN,

[0021] The ANN model can be trained by the following operations:

[0022] training the generator using the training data to generate the output image from the input image having a higher resolution than the input image;

[0023] The discriminator is trained using images output from the trained generator and at least some of the plurality of first images to determine whether an image input to the second ANN is:

[0024] an image output from the trained generator; or

[0025] a first image among the plurality of first images;

[0026] The generator and the discriminator are further trained by iterating the following steps:

[0027] Using outputs from the trained discriminator, updating parameter values ​​of the first ANN to increase the error rate of the second ANN; and

[0028] Using the output from the trained generator, parameter values ​​of the second ANN are updated to reduce the error rate of the second ANN.

[0029] In the present disclosure, the term "parameter value" of the ANN may be understood as the value of a parameter used in an operation performed at the ANN. For example, the "parameter value" may include the value of the weight of a connection between nodes in the ANN.

[0030] In an exemplary embodiment where the ANN model includes a generator and a discriminator, the first ANN and / or the second ANN may be a convolutional neural network.

[0031] Furthermore, in an exemplary embodiment where the ANN model includes a generator and a discriminator, the method may further include:

[0032] Computes the weighted average between:

[0033] The parameter values ​​of the first ANN obtained by the step of training the generator; and

[0034] updated parameter values ​​of the first ANN obtained by further training the generator and the discriminator,

[0035] Therein, the first ANN having the weighted average as the parameter value may be used as the trained ANN model for obtaining the at least one output image.

[0036] In addition, any of the methods according to the above aspects and various embodiments thereof may further include:

[0037] applying an image enhancement pre-processing step to said plurality of first images prior to said downsampling of said plurality of first images,

[0038] The image enhancement preprocessing step may include one or more of the following:

[0039] Fuzzification;

[0040] Add pixel-by-pixel noise;

[0041] Apply out-of-focus distortion;

[0042] Applies motion blur.

[0043] Performing an image enhancement preprocessing step can increase the ability of the ANN model to robustly increase image resolution.

[0044] Furthermore, in the method according to any of the above aspects and various embodiments thereof, the first resolution may be higher than the second resolution by a scaling factor (S), and the third resolution may be higher than the first resolution by a scaling factor (S).

[0045] In various aspects and embodiments described herein, the scaling factor (S) may be a real number with a value greater than 1.0. In some exemplary embodiments, the scaling factor (S) may be a value between 2.0 and 4.0. In other exemplary embodiments, the scaling factor (S) may be a value greater than 4.0.

[0046] In various aspects and embodiments described herein, the plurality of first images may include microscopic images of cells. Where the plurality of first images include microscopic images of cells, it may be preferred to set the scaling factor (S) to a value less than 3.0, and in some cases, to a value equal to or less than 2.0.

[0047] According to another aspect, a computer program product is provided, which includes computer-readable instructions, and when the computer-readable instructions are loaded on a computer and run on the computer, the computer is caused to execute the method according to any one of the above aspects and various embodiments thereof.

[0048] According to yet another aspect, a system for processing an image is provided. The system comprises:

[0049] A storage medium storing a plurality of first images having a first resolution and an artificial neural network (ANN) model, the ANN model being used to process an input image to output an output image having a higher resolution than the input image; and

[0050] Processor, configured as:

[0051] downsampling the plurality of first images to obtain a plurality of second images having a second resolution, the first resolution being higher than the second resolution, each of the plurality of second images being a downsampled version of one of the plurality of first images;

[0052] The ANN model is trained using training data comprising image pairs, each image pair comprising:

[0053] a second image from the plurality of second images as an input to the ANN model; and

[0054] In the case where the one second image among the plurality of second images is input to the ANN model, a first image among the plurality of first images corresponding to the one second image among the plurality of second images as a desired output from the ANN model;

[0055] inputting at least one first image of the plurality of first images to a trained ANN model; and

[0056] At least one output image is obtained from the trained ANN model, the at least one output image having a third resolution higher than the first resolution.

[0057] In the system according to the above aspect, the ANN model may include a convolutional neural network.

[0058] Furthermore, in the system according to the above aspect, the ANN model may include:

[0059] a generator comprising a first ANN for processing the input image and outputting the output image having a higher resolution than the input image; and

[0060] a discriminator comprising a second ANN for distinguishing images generated by the first ANN from actual images believed to be desired outputs from the first ANN,

[0061] The ANN model can be trained by the following operations:

[0062] training the generator using the training data to generate the output image from the input image having a higher resolution than the input image;

[0063] The discriminator is trained using images output from the trained generator and at least some of the plurality of first images to determine whether an image input to the second ANN is:

[0064] an image output from the trained generator; or

[0065] a first image among the plurality of first images;

[0066] The generator and the discriminator are further trained by iterating the following steps:

[0067] Using outputs from the trained discriminator, updating parameter values ​​of the first ANN to increase the error rate of the second ANN; and

[0068] Using the output from the trained generator, parameter values ​​of the second ANN are updated to reduce the error rate of the second ANN.

[0069] Furthermore, in the case where the ANN model includes a generator and a discriminator, the processor may also be configured to:

[0070] Computes the weighted average between:

[0071] The parameter values ​​of the first ANN obtained by the step of training the generator; and

[0072] updated parameter values ​​of the first ANN obtained by further training the generator and the discriminator,

[0073] Therein, the first ANN having the weighted average as the parameter value may be used as the trained ANN model for obtaining the at least one output image.

[0074] In the system according to any one of the above aspects and various embodiments thereof, the processor may also be configured to:

[0075] applying an image enhancement pre-processing step to said plurality of first images prior to said downsampling of said plurality of first images,

[0076] The image enhancement preprocessing step may include one or more of the following:

[0077] Fuzzification;

[0078] Add pixel-by-pixel noise;

[0079] Apply out-of-focus distortion;

[0080] Apply motion blur.

[0081] Furthermore, in a system according to any of the above aspects and various embodiments thereof, the first resolution may be higher than the second resolution by a scaling factor (S), and the third resolution may be higher than the first resolution by a scaling factor (S).

[0082] In the system according to any of the above aspects and various embodiments thereof, the plurality of first images may include microscopic images of cells.

[0083] The subject matter described in this application may be implemented as a method or system, possibly in the form of one or more computer program products. The subject matter described in this application may be implemented in a data signal or on a machine-readable medium, wherein the medium is embodied in one or more information carriers, such as a CD-ROM, a DVD-ROM, a semiconductor memory, or a hard disk. Such a computer program product may cause a data processing device to perform one or more operations described in this application.

[0084] In addition, the subject matter described in this application can also be implemented as a system, which includes a processor and a memory connected to the processor. The memory can encode one or more programs to cause the processor to perform one or more of the methods described in this application. In some examples, the system can be a general-purpose computer system. In other examples, the system can be a special-purpose computer system including an embedded system.

[0085] In some cases, any of the above aspects and any of the various implementations and examples described herein may provide one or more of the following advantages:

[0086] - achieve improved quality of image resolution extrapolation compared to heuristic methods such as nearest neighbor interpolation or bicubic interpolation;

[0087] - Eliminating the need to use images of a target resolution (e.g., high resolution) during training, as may be the case with known deep learning methods;

[0088] - Facilitating and / or providing improvements in further processing of images, such as cell segmentation, by achieving increased image quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] The details of one or more implementations are described in the exemplary drawings and the following description. Other features will be apparent from the description, drawings, and claims. However, it should be understood that although embodiments are described separately, individual features of different embodiments may be combined into other embodiments.

[0090] Figure 1 A schematic conceptual diagram illustrating a method according to the present disclosure is shown.

[0091] Figure 2 A schematic diagram showing an exemplary system for processing an image.

[0092] Figure 3 Shown by Figure 2 A flow chart of an exemplary process performed by the system shown.

[0093] Figure 4 A flowchart showing an exemplary process for training an ANN model in the present disclosure is shown.

[0094] Figure 5 A schematic diagram showing an exemplary ANN model according to a generative adversarial network (GAN) approach is shown.

[0095] Figure 6 Show demonstration for training Figure 5 Flowchart of an exemplary process for an exemplary ANN model shown.

[0096] Figure 7 Example visualization showing the training and evaluation steps for natural images.

[0097] Figure 8 An example of a natural image for comparing an image obtained by the method according to the present disclosure and by bicubic interpolation is shown.

[0098] Fig. 9Examples of cell images for comparing images obtained by the method according to the present disclosure and by bicubic interpolation are shown.

[0099] Fig.10 An example of a cell image used for training an ANN model according to the present disclosure is shown.

[0100] Fig.11 Shown is a conceptual diagram demonstrating the relationship between magnification, number of cells in the field of view, and image detail in live cell imaging.

[0101] Fig.12 Exemplary images showing neutrophil-like cells undergoing NETosis, including images captured and images processed using methods according to the present disclosure.

[0102] Fig.13 Examples of images for comparing images obtained by the method according to the present disclosure, by nearest neighbor interpolation, and by bicubic interpolation are shown.

[0103] Fig.14 Other examples of images for comparing images obtained by the method according to the present disclosure, by nearest neighbor interpolation and by bicubic interpolation are shown.

[0104] Fig.15 An exemplary hardware configuration of a computer that can be used to implement at least a portion of a system according to the present disclosure is shown. DETAILED DESCRIPTION

[0105] Hereinafter, a detailed description of the examples will be given with reference to the accompanying drawings. It should be understood that various modifications may be made to the examples. In particular, one or more elements of an example may be combined and used in other examples to form new examples.

[0106] Although exemplary embodiments will be described below with respect to processing 2D images, it should be noted that various aspects and exemplary embodiments may be applied to other types of data for which spatial resolution may be defined. Examples of other types of data may include, but are not limited to, 3D images, 1D and / or 2D spectra, etc.

[0107] Signal quality can be important for all analyses. In the context of image analysis, image quality and resolution can define what information is extracted from the image and / or how easy it is to extract the information, both perceptually and algorithmically. For example, it may be easier to see what is in a high-resolution, sharp image than in a low-resolution, blurry image. Therefore, the simplest way to improve the quality of downstream analysis of a given problem may be to enhance the quality of the signal used for analysis. Various aspects and exemplary embodiments described herein can provide enhancement of the input signal in a qualitative manner.

[0108] ANNs are often used to increase the resolution of images (see, for example, W. Yang, X. Zhang, Y. Tian, ​​W. Wang, and J.-H. Xue, “Deep Learning for Single Image Super-Resolution: A Brief Review,” IEEE Transactions on Multimedia, pp. 1-1, 2019; C. Dong, C. Cloy, K. He, and X. Tang, “Learning a Deep Convolutional Network for Image Super-Resolution,” 2014; B. Lim, S. Son, H. Kim, S. Nah, and K. M Lee, “Enhanced Deep Residual Networks for Single Image Super-Resolution,” ArXiv170702921Cs, July 2017). However, methods used today typically rely on high resolution to train the network with low-resolution (LR)-HR image pairs or by providing unpaired LR and HR images of the same domain as in the case of generative adversarial networks (GANs) (e.g., see Y. Yuan, S. Liu, J. Zhang, Y. Zhang, C. Dong, and L. Lin, “Unsupervised Image Super-Resolution using Cycle-in-Cycle Generative Adversarial Networks,” ArXiv180900437Cs, September 2018).

[0109] Various aspects and exemplary embodiments as described herein may increase the resolution of an image without requiring any HR images to be involved in the training process.

[0110] Figure 1 A schematic conceptual diagram showing a method according to the present disclosure is shown. Figure 1 In step A shown in FIG. 1 , in some exemplary embodiments, an image with an original resolution (e.g., a first resolution) may be downsampled to an image with a low resolution (e.g., a second resolution). Hereinafter, the image with the original resolution may also be referred to as an “OR image”, and the image with the low resolution may also be referred to as an “LR image”. For example, downsampling may be performed by bicubic interpolation. Subsequently, referring to FIG. Figure 1In step B shown, the ANN model 40 may be trained to reproduce the OR image from the LR image. The ANN model 40 may include an ANN, for example, a convolutional neural network (CNN) architecture for image processing (e.g., see A. Krizhevsky, I. Sutskever, and G. Hinton, "ImageNet Classification with Deep Convolutional Neural Networks", Neural Information Processing Systems, Vol. 25, January 2012). Once the training is completed, as shown in FIG. Figure 1 As shown in step C of , the OR image can be enhanced to an image with a high resolution (e.g., a third resolution higher than the original resolution) using an ANN model 40 having operations learned from upsampling the LR image to the OR image, in other words, the image resolution is increased from the original resolution to the high resolution. Hereinafter, the image with a high resolution may also be referred to as an "HR image".

[0111] System Configuration

[0112] Figure 2 A schematic diagram showing an exemplary system for image processing. Figure 2 The exemplary system shown includes an imaging system 10 , a computing device 20 , and a data storage device 30 .

[0113] The imaging system 10 may be configured to capture an image and provide the captured image to the computing system 20. The image may be, for example, a natural image of one or more scenes. In some cases, the image may be a microscopic image of a cell, in which case the imaging system 10 may include a microscopic imaging device (not shown), such as an optical microscope, a fluorescence microscope, or an electron microscope.

[0114] The computing device 20 may be a computer connected to the imaging system 10 via a wired and / or wireless communication network. The computing device 10 may receive images captured by the imaging system 10. The computing device 20 may be configured to perform methods according to various embodiments and examples described herein. The data storage device 30 may store information used by the computing device 20 and / or information generated by the computing device 20. For example, the storage device 30 may store an ANN model 40 (e.g., see Figure 1 ).

[0115] It should be noted that the imaging system 10, the computing device 20 and the data storage device 30 may be incorporated into a single device having one main body or implemented with more than one separate device. In addition, the computing device 20 may be implemented with more than one computer connected to each other via a wired and / or wireless communication network.

[0116] In some exemplary embodiments, the computing device 20 does not necessarily receive the image to be processed directly from the imaging system 10 that has captured the image. For example, the computing device 20 may receive the captured image from a database system (not shown) storing images captured by the imaging system 10.

[0117] Unsupervised Image Enhancement

[0118] Figure 3 Shown by Figure 2 Flowchart of an exemplary process performed by the system shown in FIG. Figure 2 The illustrated computing device 20 performs the exemplary process. For example, the exemplary process may be initiated when the computing device 20 receives an instruction from a user via an input device (not shown) to initiate the exemplary process.

[0119] In step S10, the computing device 20 may optionally perform image enhancement preprocessing on the OR image. The image enhancement preprocessing may, for example, include one or more of the following: blurring, adding pixel-by-pixel noise, applying defocus distortion, applying motion blur, etc. The OR image may have been provided (directly or indirectly) from the imaging system 10 and stored in the data storage device 30. The process may proceed to step S20 after step S10. It should be noted that step S10 is an optional step that can be skipped. In the case of skipping step S10, the process may start from step S20.

[0120] In step S20, the computing device 20 may downsample the OR image (e.g., which has been stored in the data storage device 30, as described above) to obtain an LR image. For example, the downsampling may be performed by bicubic interpolation. Each of the LR images obtained in step S20 may be a downsampled version of one of the OR images. The process may proceed to step S30 after step S20.

[0121] In step S30, the computing device 20 may use the LR image and the OR image to train the ANN model, for example, Figure 1 and Figure 2The ANN model 40 shown. The training data used for training in step S30 may include image pairs, wherein each image pair may include one of the LR images as an input to the ANN model 40, and when the one of the LR images is input to the ANN model 40, includes a corresponding one of the OR images as a desired output from the ANN model 40.

[0122] Figure 4 Show Figure 3 A flowchart of an example of a more detailed process of training in step S30 is shown below. Figure 3 In step S30, Figure 4 The exemplary process can begin.

[0123] refer to Figure 4 In step S300, the LR image from the training data may be input to the ANN of the ANN model 40 to obtain an output image with the original resolution. After step S300, the process may proceed to step S302.

[0124] In step S302, the computing device 20 may calculate how far the output image is from the corresponding OR image. This metric may be calculated, for example, using the pixel-by-pixel mean absolute error (MAE) or mean square error (MSE). In the field of deep learning, this metric may be referred to as "loss". After step S302, the process may proceed to step S304.

[0125] In step S304, the computing device 20 may update the weights of the ANN to minimize the difference between the ANN output image and the original OR image. In other words, the weights of the ANN may be updated to minimize the loss calculated in step S302. More specifically, for example, a variation of gradient descent (e.g., random, batch, or mini-batch gradient descent) and a back-propagation algorithm with respect to the current network weights, activations, and calculated losses may be used to perform the update of the weights so as to minimize the difference between the ANN output image and the original OR image.

[0126] After step S304, in step S306, the computing device 20 may determine whether to end the training process. For example, the determination in step S306 may be based on whether the iterations of training steps S300, S302, and S304 have reached a specified number of times. Alternatively or additionally, the determination in step S306 may be based on whether the loss of the ANN has converged. In the case where the specified number of iterations has been reached and / or the loss of the ANN has converged, the computing device 20 may determine to end the training process. If not, the computing device 20 may determine to continue the training process.

[0127] When the computing device 20 determines to continue the training process ("No" in step S306), the process can return to step S300. When the computing device 20 determines to end the training process ("Yes" in step S306), Figure 4 The illustrated process may end.

[0128] exist Figure 4 After the training process shown is completed, Figure 3 Step S30 may end, and the computing device 20 may proceed to execute Figure 3 Step S40.

[0129] Reference again Figure 3 In step S40, the computing device 20 may input at least one of the OR images (e.g., stored in the data storage device 30) to the trained ANN model 40. At least one of the OR images may be enhanced to a target resolution higher than the original resolution by the ANN model 40, and at least one HR image corresponding to at least one of the OR images may be output from the ANN model 40.

[0130] In step S50, the computing device 20 may obtain at least one HR image from the trained ANN model. The HR image output from the trained ANN model may have increased quality (eg, resolution) compared to the OR image input to the ANN model.

[0131] Figure 3 The illustrated process may end after step S50.

[0132] exist Figure 3 In the exemplary process shown, steps S10 and S20 may be Figure 1 The step S30 corresponds to step A of Figure 1 corresponds to step B. In addition, Figure 3 Steps S40 and S50 can be Figure 1 corresponds to step C.

[0133] In some exemplary embodiments, the ANN model 40 may be configured and trained according to a generative adversarial network (GAN) method (e.g., see Goodfellow, Ian; Pouget-Abadie, Jean; Mirza, Mehdi; Xu, Bing; Warde-Farley, David; Ozair, Sherjil; Courville, Aaron; Bengio, Yoshua, "Generative Adversarial Networks", Proceedings of the International Conference on Neural Information Processing Systems (NIPS 2014), pp. 2672-2680, available online at: https: / / arxiv.org / abs / 1406.2661). In such exemplary embodiments, the ANN model 40 may be configured and trained, as will be described below with reference to Figure 5 and Figure 6 describe.

[0134] Figure 5 An exemplary configuration of the ANN model 40 in an exemplary embodiment in which the GAN method is used for the ANN model 40 is shown.

[0135] refer to Figure 5 , the ANN model 40 may include a generator 42 having a first ANN 420 and a discriminator 44 having a second ANN 440 .

[0136] The first ANN 420 of the generator 42 may be used to process an input image and output an output image having a higher resolution than the input image. The first ANN 420 may be a CNN and may be used in a manner similar to Figure 4 The training is performed in the same manner as the exemplary training process shown.

[0137] The second ANN 440 of the discriminator 44 may be used to distinguish images generated by the first ANN 420 from actual images that are considered to be the desired output from the first ANN 420. In other words, the purpose of the discriminator 44 with the second ANN 440 may be to classify whether the input fed to the discriminator 44 is real or synthetic. The second ANN 440 may be a CNN.

[0138] If you can Figure 5 It can be seen that the LR images can be input to the generator 42, processed by the first ANN 420, and then the OR images can be output from the generator 42. These OR images output from the generator 42 and the original OR images (in other words, the OR images from which the LR images have been generated by downsampling) can be used to train the discriminator 44 to determine whether the input to the discriminator 44 is from the output of the generator 42 or the original OR image.

[0139] Figure 6 A flowchart showing an exemplary training process of an ANN model 40 including a generator 42 and a discriminator 44 is shown. For example, Figure 6 The process shown can be used as Figure 3 The portion of the training step S30 shown is performed by the computing device 20. In addition, in some cases, it is possible to start Figure 6 Prior to the exemplary process shown, Figure 4 An exemplary process training generator 42 is shown.

[0140] refer to Figure 6 In step S310, the computing device may pre-train the discriminator 44 to distinguish between the output from the generator 42 and the original OR image. For example, the OR image output from the generator 42 and the original OR image corresponding to the LR image input to the generator 42 for obtaining the OR image may be used as training data for pre-training in step S310 (see also Figure 5 ). After step S310, the process may proceed to step S312.

[0141] In step S312, the computing device 20 may use the output from the trained discriminator 44 to update the parameter values ​​of the first ANN 420 of the generator 42 to increase the error rate of the second ANN 440 of the discriminator 44. In other words, the parameter values ​​of the first ANN 420 may be updated to make the discriminator 44 less correct. After step S312, the process may proceed to step S314.

[0142] In step S314, the computing device 20 may use the output from the trained generator 42 (with the updated parameter values ​​in step S312) to update the parameter values ​​of the second ANN 440 of the discriminator 44 to reduce the error rate of the second ANN 440. In other words, the parameter values ​​of the second ANN 440 may be updated to make it more difficult for the generator 42 to create an output that the discriminator 44 cannot distinguish from the original image. After step S314, the process may proceed to step S316.

[0143] In step S316, the computing device 20 may determine whether to end the training process. For example, the computing device 20 may determine whether one or more conditions for ending the training process are met. The one or more conditions may include, for example, whether the updating steps S312 and S314 have been iterated a specified number of times, whether the metrics used to update steps S312 and / or S314 have converged, etc. Alternatively or additionally, in some cases, one or more conditions may include the convergence of a secondary metric (e.g., a peak signal-to-noise ratio (PSNR)) in addition to the metrics used to update steps S312 and / or S314. If one or more conditions for ending the training process are met, the computing device 20 may determine to end the training process. If not, the computing device 20 may determine to continue the training process.

[0144] When the computing device 20 determines to continue the training process ("No" in step S316), the process can return to step S312, and the update steps S312 and S314 can be iterated. When the computing device 20 determines to end the training process ("Yes" in step S316), the process can proceed to step S318.

[0145] In step S318, after the computing device 20 determines to end the training process in step S316, the computing device 20 may optionally calculate a weighted average between the parameter values ​​of the trained generator 42 used at S310 and the updated parameter values ​​of the generator 42. The first ANN 420 with the calculated weighted average value may be used as the ANN model 40 to generate an HR image from an OR image. It should be noted that step S318 is an optional step that can be skipped.

[0146] After determining "yes" in step S316 or after step S318, Figure 6 The illustrated process may end.

[0147] In use Figure 6 In the exemplary embodiment of the exemplary process for training the ANN model 40, there is a method for executing Figure 6 The first ANN 420 of the generator 42 of parameter values ​​resulting from the training process of the ANN model 40 can be used as the ANN for generating HR images from OR images (see, for example, Figure 3 Step S50).

[0148] In the above reference Figure 5 and Figure 6 In the exemplary embodiment described according to the GAN method, it can be understood that, for example, Figure 4 After training, use Figure 6 The training process shown is used to further fine-tune the first ANN 420 of the generator 420.

[0149] In the sense that HR images with target resolution are not required for training the ANN model 40, as described above with reference to Figures 1 to 6 The described image enhancement process can be considered as an unsupervised and / or self-supervised method.

[0150] experiment

[0151] The following provides the setup and results of experiments performed by the inventors for the method according to the present disclosure.In the experiments, ANN models were trained on different resolution levels (2x, 3x and 4x), and different downsampling methods were tested.

[0152] a) Dataset

[0153] Two different datasets are used for experiments:

[0154] - A natural image dataset consisting of 900 from the DIV2K dataset (e.g., see R. Timofte, S. Gu, L. Van Gool, L. Zhang, and M.-H. Yang, “NTIRE 2018 Challenge on Single Image Super-Resolution: Methods and Results,” 2018 IEEE / CVF Computer Vision and Pattern Recognition Workshops (CVPRW), June 2018, pp. 965–96511, doi: 10.1109 / CVPRW.2018.00130; E. Agustsson and R. Timofte, “NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Research,” Study), 2017 IEEE Computer Vision and Pattern Recognition Workshop (CVPRW), Honolulu, Hawaii, USA, July 2017, pp. 1122–1131, doi: 10.1109 / CVPRW.2017.150); and

[0155] -With S3 Cell microscopy image dataset of 3816 images captured at 10x magnification.

[0156] The DIV2K dataset is divided into 750 training images, 50 validation images, and 100 test images. The cell microscopy image dataset is divided into 2124 training images, 531 validation images, and 1160 test images.

[0157] b) Experimental setup

[0158] The experimental settings for the two datasets (i.e., the natural image dataset and the cell microscopy image dataset) are different. For the general workflow, it is assumed that images of original resolution (OR) are available, and the goal is to improve the OR images to high resolution (HR) without access to the actual HR images. If the magnitude of the resolution improvement from OR to HR is represented by a scaling factor S, then the OR images are equivalently reduced (in other words, downsampled) by the factor S to obtain LR images. The ANN model is trained to recreate the OR images from the LR images, and then used to extrapolate the data in the OR images to HR.

[0159] For the cell microscopy image dataset, three ANN models were trained and the extrapolation step was compared with bicubic interpolation (BI) upscaling. The ANN models were trained with scaling factors of 2x, 3x, and 4x for LR creation by nearest neighbor (NN) downscaling.

[0160] For natural images (DIV2K), a slightly different approach is taken. Instead of using the original images as the OR resolution, the original images of the DIV2K dataset are downscaled using BI to create a downscaled OR dataset. The downscaled OR images are used as OR images for training the ANN model. Once the ANN model is fully trained, the ANN model is used to upscale the downscaled OR test images to HR images. The peak signal-to-noise ratio (PSNR) and mean absolute error (MAE) are calculated between the extrapolated HR image output from the ANN model and the ground truth HR image which is the original image in the DIV2K dataset. This evaluation method can create the possibility of obtaining comparable metrics. However, the information loss of LR images may become significant as the scaling factor S increases, which may make it more difficult to train the ANN model. When the original image is considered to be an image with a target resolution HR, then the low resolution LR used to train the input image can be LR=HR / S 2 Therefore, the original image must be scaled down by a factor of 16 in order to obtain the LR image for the ANN model which increases the resolution fourfold.

[0161] In the experiments, a ResNet network called Enhanced Deep Super-Resolution Network (EDSR) (e.g., see B. Lim, S. Son, H. Kim, S. Nah, and KMLee, “Enhanced Deep Residual Networks for Single Image Super-Resolution,” July 2017, available online at: https: / / arxiv.org / abs / 1707.02921) was used as the architecture of the ANN model.

[0162] Figure 7 An exemplary visualization of different resolutions for an ANN model is shown, with a scaling factor S=3. More specifically, Figure 7 An exemplary visualization showing the training and evaluation steps on natural images with an ANN model that triples the image resolution. The original HR images are downscaled to simulated OR images, which are further downscaled to LR images for training purposes.

[0163] For the DIV2K dataset, six different experiments were performed: one ANN model with BI downscaling and one ANN model with NN downscaling for each of the scaling factors S = 2, 3, and 4. In addition, for each combination of scaling factor and downscaling method, a non-converged version of the ANN model was saved after 50 epochs, resulting in a total of 12 ANN models.

[0164] In all experiments, we used 2e -4 The ANN model was trained with a learning rate of , minimizing the MAE loss using the Adam optimizer (see, e.g., D.P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” ArXiv preprint, ArXiv14126980, 2014). For the cell microscopy dataset, the ANN model used a batch size of five, and for the DIV2K image dataset, the ANN model used a batch size of 12.

[0165] c) Results

[0166] Table 1 shows the results of the peak signal-to-noise ratio (PSNR) of the ANN model according to the present disclosure and a comparative example of the non-convergent model and the known heuristic methods BI and NN. In Table 1, the indications of BI and NN for the ANN model and the non-convergent model represent the reduction method, while the indications of BI and NN in the heuristic method represent the enlargement method. The item "non-convergent model" in Table 1 should be understood as a non-convergent version of the ANN model trained in 50 epochs. The PSNR is calculated between the extrapolated high-resolution image and the corresponding ground truth image.

[0167] Table 1

[0168]

[0169] As can be seen from Table 1, in terms of PSNR (the higher the better), the ANN model according to the present disclosure outperforms the known heuristic methods BI and NN and the non-convergent model. However, due to the use of the reduction method, there seems to be no obvious difference between the fully converged models. The results of the mean absolute error (MAE) metric shown in Table 2 below also follow the same pattern.

[0170] Table 2

[0171]

[0172] Table 2 shows a comparison of MAEs for the ANN model according to the present disclosure and a comparison example of a non-convergent model and known heuristic methods BI and NN. The error is calculated as the average absolute pixel difference between the extrapolated high-resolution image and the corresponding ground truth image. In addition, in Table 2, the indications of BI and NN for the ANN model and the non-convergent model represent the reduction method, while the indications of BI and NN in the heuristic method represent the enlargement method. In addition, the item "non-convergent model" in Table 2 should be understood as a non-convergent version of the ANN model trained in 50 epochs.

[0173] Figure 8 Examples of natural images for comparing images obtained from an ANN model according to the present disclosure and through bicubic interpolation (BI) are shown. Figure 8 Example outputs from ANN models trained at 2x, 3x, and 4x resolution increases are shown (corresponding to Figure 8 ). Figure 8 In the example, the image in column (a) is the input image, column (b) is the output image from the ANN model, column (c) is the ground truth image, and column (d) is the image enlarged using BI. Figure 8The input image shown in column (a) of is taken as the OR input image and the resolution is increased to HR by the corresponding scaling factor. Compared to the BI upscaling, the output image of the ANN model looks sharper and thus better resembles the ground truth image. As the input resolution decreases, the ability to recreate the ground truth HR image also decreases.

[0174] Fig. 9 Examples of cell images for comparing images obtained from an ANN model according to the present disclosure and by bicubic interpolation are shown. Fig. 9 The image rows in correspond to resolution increases of 2x, 3x, and 4x (in other words, scaling factors of 2, 3, and 4). Fig. 9 , the image in column (a) shows the input image, column (b) shows the output image from the ANN model, and column (c) shows the image enlarged with BI enlargement as a comparative example.

[0175] If you can Fig. 9 It can be seen that the cell microscopy image quality is significantly improved with the ANN model when a 2x resolution increase is used compared to the input image and the resolution increase with BI. However, as the resolution increases to 3x and 4x, the quality of the ANN model output does not improve, or even decreases. The ANN model with a resolution increase of 3x and 4x is still able to capture the cell boundaries.

[0176] Fig.10 An example of a cell image for training an ANN model according to the present disclosure is shown. Specifically, Fig.10 The ability of different ANN models to recreate images with higher target resolution from images with lower resolution is shown. Fig.10 The ANN model is trained with the target image (see column (c), the ground truth) and the input image (see Fig.10 Column (a) in the figure) is derived by upscaling via bicubic interpolation (see Fig.10 Compared to the image obtained in column (d) in Figure 1, we can see that the output image from the ANN model (see Fig.10 Column (b) in the figure.

[0177] from Fig.10 , through the training of the ANN model (LR to OR), the LR images seem to suffer from significant information loss in the 3x and 4x settings. A lot of details in the ground truth image are lost in its LR counterpart. In this setting, the reconstruction is still much better than BI.

[0178] Applications for live cell imaging

[0179] In live cell imaging, image quality can be critical to making correct decisions. However, to increase resolution, it is often necessary to reduce the field of view (FOV) by zooming in on the display. Ideally, an instrument would capture high resolution with a large FOV. However, this may not be feasible due to various limitations including instrument throughput, hardware limitations, hard drive storage limitations, etc.

[0180] Therefore, image acquisition at higher magnification can produce clearer and more detailed images, while lower magnification can enable the capture of a larger number of cells within the FOV. As the magnification increases, the number of cells in the FOV decreases, as Fig.11 shown.

[0181] By increasing the effective resolution, more detailed images can be obtained without sacrificing the number of cells in the FOV. Therefore, rare cellular events are more likely to be captured, and the statistical relevance of whole population studies can be increased. Replication in conventional ways (e.g., by acquiring a larger number of high-magnification images and combining them) can be time-consuming and laborious, and may damage cells due to increased phototoxicity.

[0182] Applying methods according to the present disclosure to images acquired at 20x can improve images at the cellular level. The resolution of cell edges can be increased, which can enable more accurate cell segmentation, for example, subcellular features can have enhanced texture and clarity, reflecting biological structures. Although accurate label-free quantification of subcellular processes in 20x images is not yet possible, subcellular organelles such as the nucleus can be clearly visualized by improving image quality. In addition, texture, periphery, and robust features can be more accurately quantified. This is achieved by Fig.12 To show, Fig.12 Enhanced image showing a cell undergoing NETosis, a process in which the cell nucleus expands and changes shape before releasing its contents into the cytoplasm.

[0183] Fig.12 Shown are neutrophil-like cells undergoing NETosis. Fig.12 The top row of images in indicate images acquired at 20x from 0 to 90 minutes (columns (a), (b), (c), and (d)), Fig.12 The bottom row of images of shows the same images processed using an ANN model trained according to the present disclosure. Fig.12It is seen that the processed image at 0 minute has a more visible outline, and the cell contents have an increased texture consistent with this type of cell. In other words, the visible texture, outer cell border and cell shape in the cell are more obvious in the processed image at 0 minute. At 30 minutes, the nucleus has enlarged and this is also more prominent in the processed image. For example, in the processed image at 30 minutes, the outer cell border is clearer, and the nucleus is more prominent, thereby reflecting the biological process of DNA decondensation. At 60 minutes and 90 minutes, the processed image again shows better texture and clearer cell borders in the region where the nucleus has discharged its contents into the cytoplasm. For example, in the processed image at 60 minutes, the texture in the cell is more visible, and the cell outline is clearer. In addition, in the processed image at 90 minutes, the outer cell border is clearer, the texture and cell contents are visible, and the shape of the cell is more obvious.

[0184] Other experiments on live cell imaging

[0185] An exemplary workflow for acquiring images with standard quality and then upscaling in silico using an ANN model according to the present disclosure is provided below.

[0186] In these experiments, a dataset of phase contrast microscopy images of cells in culture at 10X magnification was provided by Essen Instruments under the name Essen Biosciences. The images captured by S3 consist of images of seven different cell types, namely:

[0187] ·A549

[0188] HeLa

[0189] HT1080

[0190] ·HUVEC

[0191] Jurkat

[0192] ·MDAMB231

[0193] ·PC3

[0194] ·SKOV3

[0195] The dataset consists of 1746 images and is split into training, validation, and test datasets. The training data consists of 1037 images from all cell types, and the validation data has 259 images. The test data consists of 450 images from all cell cultures, some of which are from the same wells but isolated from the wells in the training and validation datasets.

[0196] A CNN with a ResNet architecture as used in B. Lim, S. Son, H. Kim, S. Nah, and KMLee, "Enhanced Deep Residual Networks for Single Image Super-Resolution", ArXiv 1707.02921 Cs, July 2017 (available online at https: / / arxiv.org / abs / 1707.02921) was trained by taking patches of the training image and then downsampling the image patches to half the resolution using bicubic interpolation and then assigning the network the task of recreating the original image patches. This was done with the Adam optimizer, batch size 20, and learning rate 0.0001 for 2000 epochs. Pixel-wise mean absolute error (MAE) was used as the loss function, and the network was evaluated on MAE, peak signal-to-noise ratio (PSNR), and visual inspection of the image quality of the validation data.

[0197] When the network is fully trained, the model with the highest MAE and PSNR is selected and further validated on the test data to ensure generalizability. In this final step, the network is modified to take the original images as input instead of the downsampled training and validation images. Therefore, the resolution of the test images is increased from 10x to 20x, and these HR images are then compared to the upsampled images from the test set using bicubic interpolation and nearest neighbor interpolation. Visual inspection shows that the 20x images produced by our model have higher fidelity than those compared.

[0198] Fig.13 and Fig.14 The image enhancement using the ANN model according to the present disclosure is shown (see Fig.13 and Fig.14 (b)) and the original image (see Fig.13 (a) and Fig.14 (d) in), nearest neighbor interpolation (see Fig.13 and Fig.14 (c) in ) and bicubic interpolation (see Fig.13 (d) and Fig.14 Comparison of (a)).

[0199] Possible other implementations

[0200] As can be seen from the above reference Fig. 9 and Fig.10The experimental results described show that, for example, if the information loss of LR is too large, the ANN model according to the present disclosure may not be able to learn how to reconstruct the finer details of the image with the target resolution (e.g., HR). This problem does not occur in the same way when actually accessing LR-HR pairs where the model can directly learn the mapping.

[0201] In this regard, one possible implementation of the present disclosure for achieving a higher resolution increase may be to use the method according to the present disclosure as a pre-training method to later fine-tune the model on actual HR images. An advantage of the possible implementation may be that the amount of HR images required may be less than without any pre-training.

[0202] In some cases, the information loss of LR can also be somewhat mitigated by alternative means of downscaling, such that information of important features that the model expects to learn is preserved. It is not necessarily important to have an exact replica of what the actual LR image looks like, but rather to preserve information of such important features. Thus, for example, instead of downscaling an OR image with a BI or NN as described above, another ANN can be employed and trained to downscale the OR image to an LR image, and then an ANN model according to the present disclosure can be trained to restore the OR image from the LR image. Another ANN for downscaling an OR image can be trained to preserve information used in the upscaling task of the ANN model according to the present disclosure.

[0203] Compared to cell microscopy images with 3x and 4x resolution increase, the ANN model according to the present disclosure appears to perform better on natural images. This may be because the model can pick up and learn to reconstruct more features in natural images than cell microscopy images. Natural images typically contain relatively few objects with mainly large textures, while cell images usually contain many small objects that mostly disappear when the resolution is reduced too much. This may further support the idea of ​​the importance of keeping information intact in the downsampling step.

[0204] Hardware Configuration

[0205] Fig.15 An exemplary hardware configuration of a computer that can be used to implement at least part of the system described above is shown. For example, Figure 1 The computing device 20 shown may be used Fig.15 The computer 7 shown is implemented. Fig.15The computer 7 shown includes a central processing unit (CPU) 70, a system memory 72, a network interface 74, a hard disk drive (HDD) interface 76, an external disk drive interface 78, and an input / output (I / O) interface 80. These components of the computer are connected to each other via a system bus 82. The CPU 70 can perform arithmetic, logic, and / or control operations by accessing the system memory 72. The system memory 72 can store information and / or instructions for use in conjunction with the CPU 70. The system memory 72 can include volatile and non-volatile memory, such as a random access memory (RAM) 720 and a read-only memory (ROM) 722. A basic input / output system (BIOS) containing basic routines that help to transfer information between elements within the computer 7, such as during startup, can be stored in the ROM 722. The system bus 82 can be any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures.

[0206] The computer may include a network interface 74 for communicating with other computers and / or devices via a network.

[0207] In addition, the computer may include a hard disk drive (HDD) 84 for reading and writing from a hard disk (not shown) and an external disk drive 86 for reading or writing from a removable disk (not shown). The removable disk may be a disk for a disk drive or an optical disk for an optical disk drive, such as a CD ROM. The HDD 84 and the external disk drive 86 are connected to the system bus 82 via the HDD interface 76 and the external disk drive interface 78, respectively. The drive and its associated computer-readable medium provide non-volatile storage of computer-readable instructions, data structures, program modules, and other data of a general-purpose computer. The data structure may include relevant data for implementing the exemplary method and its variants as described herein. The relevant data may be organized in a database, such as a relational or object database.

[0208] Although the exemplary environment described herein employs a hard disk (not shown) and an external disk (not shown), those skilled in the art will appreciate that other types of computer-readable media that can store data accessible by a computer, such as magnetic tape cassettes, flash memory cards, digital video disks, random access memory, read-only memory, etc., may also be used in the exemplary operating environment.

[0209] Many program modules may be stored on the hard disk, external disk, ROM 722 or RAM 720, including an operating system (not shown), one or more application programs 7202, other program modules (not shown), and program data 7204. The application programs may include at least part of the functionality described above.

[0210] The computer 7 can be connected to an input device 92 (such as a mouse and / or keyboard) and a display device 94 (such as a liquid crystal display) via corresponding I / O interfaces 80a and 80b and a system bus 82. In the case where the computer 7 is implemented as a tablet computer, for example, a touch panel that displays information and receives input can be connected to the computer 7 via corresponding I / O interfaces and the system bus 82. In addition, in some examples, although Fig.15 Not shown, the computer 7 may also be connected to a printer and / or an imaging device, such as a camera, via corresponding I / O interfaces and the system bus 82 .

[0211] In addition to using Fig.15 In addition to or as an alternative to the implementation of the computer 7 shown, some or all of the functions of the exemplary embodiments described herein may be implemented as one or more hardware circuits. Examples of such hardware circuits may include, but are not limited to: large-scale integration (LSI), reduced instruction set circuits (RISC), application-specific integrated circuits (ASICs), and field programmable gate arrays (FPGAs).

Claims

1. A computer-implemented method for processing an image, the method comprising: downsampling a plurality of first images having a first resolution to obtain a plurality of second images having a second resolution, wherein the first resolution is higher than the second resolution, and each of the plurality of second images is a downsampled version of one of the plurality of first images; as well as An artificial neural network model (40) is trained to process an input image and output an output image having a higher resolution than the input image, wherein training data for the training comprises image pairs, each image pair comprising: a second image from among the plurality of second images as an input to the artificial neural network model (40); and When the one second image among the plurality of second images is input to the artificial neural network model (40), a first image among the plurality of first images corresponding to the one second image among the plurality of second images is used as a desired output from the artificial neural network model (40); Wherein, the artificial neural network model (40) comprises a convolutional neural network; and Wherein, the method further comprises: inputting at least one first image of the plurality of first images into a trained artificial neural network model (40); and Obtaining at least one output image from the trained artificial neural network model (40), the at least one output image having a third resolution higher than the first resolution; Wherein, the artificial neural network model (40) further includes: a generator (42), the generator comprising a first artificial neural network (420), the first artificial neural network being configured to process the input image and output the output image having a higher resolution than the input image; and A discriminator (44) comprising a second artificial neural network (440) for distinguishing images generated by the first artificial neural network (420) from actual images considered to be expected outputs from the first artificial neural network (420), wherein the artificial neural network model (40) is trained by: Using the training data to train the generator (42) to generate the output image from the input image having a higher resolution than the input image; The discriminator (44) is trained using images output from the trained generator and at least some of the plurality of first images to determine whether an image input to the second artificial neural network (440) is: an image output from said trained generator (42); or a first image among the plurality of first images; The generator (42) and the discriminator (44) are further trained by iterating the following steps: Using output from the trained discriminator (44), updating parameter values ​​of the first artificial neural network (420) to increase the error rate of the second artificial neural network (440); and updating parameter values ​​of the second artificial neural network (440) using output from the trained generator to reduce the error rate of the second artificial neural network (440); And wherein the method further comprises: Computes the weighted average between: said parameter values ​​of said first artificial neural network obtained by the step of training said generator (42); and updated parameter values ​​of the first artificial neural network (420) obtained by further training the generator (42) and the discriminator (44), The first artificial neural network (420) having the weighted average value as the parameter value is used as the trained artificial neural network model (40) for obtaining the at least one output image.

2. The method according to claim 1, further comprising: applying an image enhancement pre-processing step to said plurality of first images prior to said downsampling of said plurality of first images, The image enhancement preprocessing step may include one or more of the following: Fuzzification; Add pixel-by-pixel noise; Apply out-of-focus distortion; Apply motion blur.

3. The method according to claim 1 or 2, wherein: The first resolution is higher than the second resolution by a scaling factor, and the third resolution is higher than the first resolution by the scaling factor.

4. The method according to claim 1 or 2, wherein: The plurality of first images include microscopic images of cells.

5. A computer program product comprising computer readable instructions which, when loaded on a computer and executed on the computer, cause the computer to execute the method according to any one of claims 1 to 4.

6. A system for processing an image, the system comprising: A storage medium storing a plurality of first images having a first resolution and an artificial neural network model (40) for processing an input image to output an output image having a higher resolution than the input image; as well as Processor, configured as: downsampling the plurality of first images to obtain a plurality of second images having a second resolution, wherein the first resolution is higher than the second resolution, and each of the plurality of second images is a downsampled version of a first image in the plurality of first images; as well as The artificial neural network model (40) is trained using training data comprising image pairs, each image pair comprising: a second image from among the plurality of second images as an input to the artificial neural network model (40); and When the one second image among the plurality of second images is input to the artificial neural network model (40), a first image among the plurality of first images corresponding to the one second image among the plurality of second images is used as a desired output from the artificial neural network model (40); Wherein, the artificial neural network model (40) comprises a convolutional neural network, Wherein, the processor is further configured to: inputting at least one first image of the plurality of first images into a trained artificial neural network model (40); and Obtaining at least one output image from the trained artificial neural network model (40), the at least one output image having a third resolution higher than the first resolution; Wherein, the artificial neural network model (40) further includes: a generator (42), the generator comprising a first artificial neural network (420), the first artificial neural network being configured to process the input image and output the output image having a higher resolution than the input image; and A discriminator (44) comprising a second artificial neural network (440) for distinguishing an image generated by the first artificial neural network (420) from an actual image considered to be a desired output from the first artificial neural network (420), wherein the artificial neural network model is trained by: Using the training data to train the generator (42) to generate the output image from the input image having a higher resolution than the input image; The discriminator (44) is trained using images output from the trained generator (42) and at least some of the plurality of first images to determine whether an image input to the second artificial neural network (440) is: an image output from said trained generator (42); or a first image among the plurality of first images; The generator and the discriminator are further trained by iterating the following steps: Using outputs from the trained discriminator, updating parameter values ​​of the first artificial neural network (420) to increase the error rate of the second artificial neural network (440); and updating parameter values ​​of the second artificial neural network (440) using output from the trained generator to reduce the error rate of the second artificial neural network (440); And wherein the processor is further configured to: Computes the weighted average between: said parameter values ​​of said first artificial neural network (420) obtained by the step of training said generator (42); and updated parameter values ​​of the first artificial neural network (420) obtained by further training the generator and the discriminator (44), The first artificial neural network (420) having the weighted average value as the parameter value is used as the trained artificial neural network model (40) for obtaining the at least one output image.

7. The system according to claim 6, wherein: The processor is further configured to: applying an image enhancement pre-processing step to said plurality of first images prior to said downsampling of said plurality of first images, The image enhancement preprocessing step may include one or more of the following: Fuzzification; Add pixel-by-pixel noise; Apply out-of-focus distortion; Apply motion blur.

8. The system according to claim 6 or 7, wherein: The first resolution is higher than the second resolution by a scaling factor (S), and the third resolution is higher than the first resolution by the scaling factor (S).

9. The system according to claim 6 or 7, wherein: The plurality of first images include microscopic images of cells.