Method and system for training a convolutional neural network

Abstract

To provide a computer-implemented method, system, and program for training a convolutional neural network.SOLUTION: A method 400 disclosed herein comprises: receiving a captured image (step 402); generating a statistical noise model based on the captured image (step 404); and training a convolutional neural network based on the captured image and the statistical model (step 406).SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a method for training a convolutional neural network. [Background technology]

[0002] Image denoising is used to process images contaminated with additive noise to achieve noise reduction while preserving image features. The inventors have discovered that additive noise can be introduced when users attempt to minimize photobleaching and phototoxicity by minimizing energy input, which can be achieved by shortening the exposure time or reducing the gain, at the expense of increasing the amount of noise. In a typical workflow, images are acquired with short exposure times or small gains, resulting in low-quality images that may contain noise from electronic sources (e.g., Gaussian noise, salt-and-pepper noise, and shot noise) or sample-dependent high-frequency noise due to quantization, sensitivity, or heat from the camera sensor. Summary of the Invention [Problem to be solved by the invention]

[0003] The problem solved by the present invention is how to remove noise from an image and obtain a good signal-to-noise ratio (SNR). [Means for solving the problem]

[0004] In an embodiment of the present invention, a computer-implemented method for training a convolutional neural network is provided, the method including the steps of receiving captured images, generating a statistical noise model based on the captured images, and training a convolutional neural network based on the captured images and the statistical model. Thus, SNR can be increased when training the convolutional neural network using one or more captured images.

[0005] In a possible implementation of the above embodiment, the step of generating the statistical model includes: determining variance or standard deviation values ​​for pixels in the captured image according to ground truth or simulated ground truth; and generating the statistical model based on the variance or standard deviation values. Thus, the convolutional neural network can be trained more efficiently and based on actual captured images, and may not utilize so-called training images.

[0006] In a possible implementation of the above embodiment, the step of training the convolutional neural network includes dividing the captured image into random patches, exchanging a pixel value with an adjacent pixel value in each random patch, and training the convolutional neural network based on the random patches of the captured image and the random patches of the captured image including the exchanged pixel values.

[0007] In a possible implementation of the above embodiment, the step of training the convolutional neural network includes: determining, based on a statistical model, probabilities of pixel values ​​of random patches containing swapped pixel values; and training the convolutional neural network by maximizing the probabilities of each pixel value of the random patches containing swapped pixel values. Thus, the convolutional neural network can be trained and utilized using the first acquired image and the subsequent acquired image without necessarily relying on training data.

[0008] In a possible implementation of the above embodiment, maximizing the probability of each pixel value involves minimizing the negative logarithm of the mean of all pixel probabilities.

[0009] In a possible implementation of the above embodiment, a probability for each pixel value is determined based on a Gaussian distribution having a mean and a variance or standard deviation, where the mean is the average value of each pixel in a random patch of the captured image, and the variance or standard deviation is obtained from a statistical model. Alternative techniques for determining the variance may include determining the noise variance based on the camera counts (i.e., gain) in the image, or modeling the fluorescence to determine the variance of any noise between the expected object and the obtained object in a particular frame with exposure time t. In general, several techniques are available for creating a model that outputs a variance value for each pixel based on camera gain or modeling the fluorescence of the imaged object.

[0010] In a possible implementation of the above embodiment, the method includes applying a trained convolutional neural network to the captured image.

[0011] In a possible implementation of the above embodiment, the method includes correcting each pixel probability of an image generated by the trained convolutional neural network using a statistical model, and averaging each generated pixel probability with each corrected pixel probability.

[0012] In a possible implementation of the above embodiment, the method includes receiving another captured image, updating a statistical model based on the captured image and the other captured image, determining whether a variance or standard deviation value of the updated statistical model has decreased compared to the variance model, and retraining the convolutional neural network based on the other captured image and the updated statistical model if a decrease in the variance or standard deviation value of the updated statistical model has been determined compared to the statistical model. Thus, the performance of the convolutional neural network can be improved each time a new image is acquired.

[0013] In a possible implementation of the above embodiment, the convolutional neural network is an encoder-decoder neural network.

[0014] In an embodiment of the present invention, a system is provided that includes one or more processors and one or more storage devices, the system being configured to perform the methods described above.

[0015] In a possible implementation of the above embodiment, the system includes an imaging device connected to the processor for acquiring microscopic images.

[0016] In an embodiment of the present invention, a trained convolutional neural network is provided, which is trained by receiving a captured image, generating a statistical noise model based on the captured image, and adjusting the convolutional neural network based on the captured image and the statistical model.

[0017] In an embodiment of the present invention, a computer program is provided with program code for performing the above-described method.

[0018] The present disclosure can be understood by reference to the following description of the embodiments in connection with the accompanying drawings. [Brief explanation of the drawings]

[0019] [Figure 1] FIG. 1 illustrates a method for training a convolutional neural network according to an embodiment of the present invention. [Figure 2] FIG. 10 is a diagram illustrating a graph of variance for each value in the grayscale for one image. [Figure 3] FIG. 10 illustrates a graph of variance for each value in the gray scale for 1000 images. [Figure 4] FIG. 1 illustrates a method for training a convolutional neural network according to another embodiment of the present invention. [Figure 5] FIG. 1 shows an acquired image, for example a microscope image, before processing is performed. [Figure 6] FIG. 1 illustrates an image after being processed using a trained convolutional neural network trained in accordance with an embodiment of the present invention. [Figure 7] FIG. 10 is a diagram showing the ground truth of an acquired image. [Figure 8]FIG. 1 illustrates a system for implementing a method for training a convolutional neural network according to another embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0020] The inventors have realised that the way to understand the noise reduction problem is to consider an ideal image (high signal-to-noise ratio, SNR) along with a mathematical distribution that adds a random value to each pixel of the resulting image. This function can be modeled using a Poisson-Gaussian distribution, for example in the case of fluorescence, where Poisson noise (also called shot noise) is the dominant source.

[0021] If the main part of the information remains in the image, the value of a noisy pixel can be statistically estimated from the neighborhood of each pixel. By considering the Gaussian distribution for each pixel, the mean and variance (or standard deviation) can be obtained. The larger the variance (or standard deviation), the larger the noise and the smaller the SNR (i.e., the lower the image quality).

[0022] 1 illustrates a method for training a convolutional neural network in accordance with an embodiment of the present invention. The convolutional neural network may be pre-trained by a manufacturer but forms part of an on-the-fly training system that is trained during customer use. The pre-training performed by the manufacturer may include use of the method of an embodiment of the present invention. A convolutional neural network is a form of machine learning algorithm or model.

[0023] In Figure 1, there are three overall steps: training 100, processing 102, and repeated training 104.

[0024] In step 100, an image 106 (i.e., a microscope image) is acquired, for example, from a microscope. A simulated ground truth image 110 is obtained by applying, for example, a Gaussian blur 108 to the acquired image 106. Alternatives to Gaussian blur include averaging filtering (e.g., average pixel weighting), bilateral filtering, non-local means (NLM) smoothing, and block-matching and 3D filtering (BM3D). If a ground truth image exists (i.e., the same image with a very high peak signal-to-noise ratio, PSNR), obtaining a simulated ground truth image may not be necessary. The acquired image 106 is compared with the simulated ground truth image 110, and the variance of each pixel value is extracted to create a mean variance value for each value on a grayscale of 0 to 255 or a color scale of 0 to 65536. The variance value can be determined based on the acquired image 106 and the simulated ground truth image using the respective values ​​in a two-dimensional histogram. A model 112 is then fitted to the average variance. This model may be a polynomial model, an exponential model, or other suitable statistical model derived by plotting the variance. Alternative techniques for determining the variance may include determining the variance of noise based on the camera counts (i.e., gain) in the image, or modeling the fluorescence to determine the variance of any noise between the expected object and the obtained object in a particular frame having exposure time t. In general, several techniques are available for creating a model that outputs a variance value for each pixel based on camera gain or modeling the fluorescence of the imaged object.

[0025] 2 illustrates graphs of the average variance for each value in the grayscale for one image. In the graph "Variance of 1000 Images (Noise vs. Ground Truth)", the thick solid line represents the ideal statistical model, in the graph "Corresponding Variance of Noise vs. Fake Ground Truth for One Image", the thin solid line represents the value obtained by comparing the acquired image 106 with the simulated ground truth image 110, and in the graph "Fitting of One Image", the dashed line represents the statistical model fitted to "Fake GT for One Image".

[0026] Returning to FIG. 1 , the captured image 106 is cropped or divided into multiple random patches 114. These patches 114 are typically square and represent sections of the captured image 106, and each patch may overlap with adjacent patches. The size of each patch is determined based on the size of the image and the processing power of the image processing device used, and may be, for example, up to 256 x 256 pixels. A typical patch may be 64 x 64 pixels, thereby dividing a 512 x 512 pixel image into 64 patches. Several pixels are selected from each random patch 114 using a uniform random number distribution, and the pixel values ​​of the selected pixels are swapped or replaced with values ​​from neighboring pixels, thereby generating a random patch of the captured image 118 containing swapped pixel values. Typically, 1% to 10% of the pixels may be swapped or replaced within a patch. Image 116 illustrates an enlarged section of one random patch 114, and image 120 illustrates an enlarged section of one random patch of captured image 118 with swapped pixel values.

[0027] The random patches of the captured image 118 containing the swapped pixel values ​​are then used to train a convolutional neural network (CNN) 122. In this example, a U-net is trained, although it will be appreciated that other reduction-enlargement or encoder-decoder CNNs such as Resnet (e.g., Resnet-18) could be used. In particular, a dynamic U-net (Encoder Resnet-18) with sub-pixel convolutions (pixel shuffling) in the merge layer could be used. The CNN 122 is trained based on the random patches of the captured image 18 containing the swapped pixel values ​​and the variance values ​​obtained using the model 112. The CNN 122 maximizes the probability that each pixel in the captured image belongs to the captured image (i.e., the CNN 122 maximizes the probability that each pixel in the captured image is not the result of noise). This is achieved by the following equation 128:

number

[0028] In step 102, a trained CNN 132 is applied to denoise an acquired image 130, which may correspond to acquired image 106, or a similar image, to obtain a denoised image 134.

[0029] In step 104, a new, separate image 136 is acquired, and a new statistical model 138 is generated based on the variance values ​​of the acquired image 130 and the separate acquired image 136. The new statistical model 138 is compared to the previous statistical model 112 generated in step 100, and a determination 140 is made as to whether any variance values ​​have decreased in the statistical model. If the variance values ​​have decreased in the new statistical model 138, the CNN 132 is retrained 142 using the separate acquired image 136, the new statistical model 138, and any previous acquired images, such as acquired image 106. The retrained CNN is then applied to the separate acquired image 136.

[0030] If the variance value is not reduced in the new statistical model 138, denoising 144 another acquired image 136 is performed based on the CNN 132 without further training to obtain a denoised image 146.

[0031] Additionally, the pixel probability of each of the denoised images 134, 146 can be estimated and corrected, averaging it with itself using the current statistical model.

[0032] It may also be desirable to update 148 any previously denoised images based on the latest CNN and statistical models, and retrain the CNN 132 based on previously acquired images and any new statistical models.

[0033] Step 104 is repeated until no decrease is detected in the new statistical model when compared to the previous statistical model. In this regard, step 104 is repeated until the difference between the previous model and the new model is 0 ± a tolerance (e.g., a tolerance of ±0.1% to 1%, ±0.1% to 2%, ±0.1% to 3%, ±0.1% to 4%, ±0.1% to 5%).

[0034] FIG. 3 illustrates graphs of variance for each grayscale value for 1000 images. In the "Variance of 1000 Images (Noise vs. Ground Truth)" graph, the solid line represents the variance data obtained from 1000 images, and in the "Curve Fit of 1000 Images" graph, the dashed line represents the statistical model fitted to the "Variance of 1000 Images (Noise vs. Ground Truth)" line. It is clear that after 1000 images, for example, the variance no longer decreases and the model no longer needs to be updated. However, it will be understood that fewer images, such as 50 images, can also be used.

[0035] A system implementing the above-described method can evaluate the new "noisy" image when a new subsequent image is acquired or its parameters (such as laser power, acquisition speed, exposure time changes, etc.) are updated. If the variance is reduced compared to the previous statistical model, the model can be updated using the methods described herein. Thus, image quality can be iteratively improved, for example, over the course of an experiment.

[0036] Although variance values ​​are referred to herein, it will be apparent that standard deviation values ​​may be used as well.

[0037] 4 illustrates a method 400 for training a convolutional neural network according to another embodiment of the present invention, including receiving a captured image 402, generating a statistical noise model based on the captured image 404, and training a convolutional neural network based on the captured image and the statistical model 406.

[0038] FIG. 5 shows an acquired image, such as a microscope image, before any processing is performed.

[0039] Figure 6 shows the image after processing using a trained convolutional neural network trained in accordance with an embodiment of the present invention. What is evident from the image in Figure 6 is the improved SNR for the captured image.

[0040] Figure 7 shows the ground truth of the acquired image.

[0041] FIG. 8 shows a schematic diagram of a system 800 configured to perform the methods described herein. The system 800 includes a microscope 802 and a computer system 804. The microscope 802 is configured to capture images and is connected to the computer system 804. The computer system 804 is configured to perform at least a portion of the methods described herein. The computer system 804 may be configured to execute machine learning algorithms. The computer system 804 and the microscope 802 may be separate entities or may be integrated into a common housing. The computer system 804 may be part of a central processing system of the microscope 802 and / or may be part of a subordinate component of the microscope 802, such as a sensor, actor, camera, or lighting unit of the microscope 802.

[0042] The computer system 804 may be a local computing device (e.g., a personal computer, laptop, tablet computer, or mobile phone) with one or more processors and one or more storage devices, or may be a distributed computing system (e.g., a cloud computing system 816 with one or more processors and one or more storage devices distributed at various locations, such as local clients and / or one or more remote server farms and / or data centers). The computer system 804 may include any circuit or combination of circuits. In one embodiment, the computer system 804 may include one or more processors 806, which may be of any type. As used herein, a processor may contemplate any type of computing circuit, such as, but not limited to, a microprocessor of a microscope or microscope component (e.g., a camera), a microcontroller, a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a graphics processor 808, a digital signal processor (DSP), a multi-core processor, a field programmable gate array (FPGA), or any other type of processor or processing circuit. Other types of circuits that may be included in computer system 804 may be custom circuits, application specific integrated circuits (ASICs), etc., such as one or more circuits (e.g., communications circuits) used in wireless devices such as cell phones, tablet computers, laptop computers, two-way radios, and similar electronic systems. Computer system 804 may also include one or more storage devices 810, which may include one or more memory elements suitable for a particular application, such as main memory in the form of random access memory (RAM), one or more hard drives and / or one or more drives that handle removable media, such as compact discs (CDs), flash memory cards, digital video discs (DVDs), etc.Computer system 804 may also include a display device 812, one or more speakers and a controller 814 which may include a keyboard and / or a mouse, trackball, touch screen, voice recognition device, or any other device that allows a user of the system to input information to and receive information from computer system 804.

[0043] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items and may be abbreviated as " / ".

[0044] While some aspects have been described in the context of an apparatus, it will be apparent that these aspects also represent a description of a corresponding method, where a block or apparatus corresponds to a step or feature of a step, and similarly, aspects described in the context of a step also represent a description of a corresponding block or item or feature of a corresponding apparatus.

[0045] Some or all of the steps may be performed by (or using) a hardware apparatus, such as, for example, a processor, microprocessor, programmable computer, or electronic circuitry. In some embodiments, any one or more of the critical steps may be performed by such an apparatus.

[0046] Depending on certain implementation requirements, embodiments of the present invention may be implemented in hardware or software. This implementation may be performed by a non-transitory storage medium, such as a digital storage medium, for example, a floppy disk, a DVD, a Blu-ray, a CD, a ROM, a PROM, an EPROM, an EEPROM, or a FLASH memory, on which electronically readable control signals are stored, which cooperate (or can cooperate) with a programmable computer system to implement the respective methods. Therefore, the digital storage medium may be computer-readable.

[0047] Some embodiments of the present invention include a data carrier having electronically readable control signals that can cooperate with a programmable computer system to perform any of the methods described herein.

[0048] Generally, embodiments of the present invention may be implemented as a computer program product comprising program code that is operative to perform any of the methods when the computer program product is run on a computer, and that may be stored, for example, on a machine-readable carrier.

[0049] Further embodiments comprise the computer program for performing any of the methods described herein, stored on a machine readable carrier.

[0050] In other words, an embodiment of the present invention is, therefore, a computer program having a program code for performing any of the methods described herein when the computer program runs on a computer.

[0051] Therefore, another embodiment of the invention is a recording medium (or data carrier or computer readable medium) containing a computer program stored thereon for performing any of the methods described herein when executed by a processor. The data carrier, digital recording medium or recording medium is typically tangible and / or non-transitory. Another embodiment of the invention is an apparatus as described herein, comprising a processor and a recording medium.

[0052] A further embodiment of the present invention is, therefore, a data stream or a sequence of signals representing the computer program for performing any of the methods described herein, the data stream or sequence of signals being for example adapted to be transmitted via a data communication connection, for example the Internet.

[0053] Another embodiment comprises a processing means, for example a computer, or a programmable logic device configured to or adapted to perform any of the methods described herein.

[0054] Another embodiment comprises a computer having installed thereon the computer program for performing any of the methods described herein.

[0055] Another embodiment of the present invention includes an apparatus or system configured to transfer (e.g., electronically or optically) a computer program for implementing any of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a storage device, etc. The apparatus or system may include, for example, a file server for transferring the computer program to the receiver.

[0056] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform any of the methods described herein. In general, the methods are advantageously performed by any hardware apparatus.

[0057] Embodiments may be based on the use of machine learning models or algorithms. Instead of relying on models and inference, machine learning may refer to algorithms and statistical models that a computer system may use to perform a particular task without using explicit instructions. For example, machine learning may use data transformations inferred from an analysis of past data and / or training data instead of rule-based data transformations. For example, image content may be analyzed using a machine learning model or algorithm. For a machine learning model to analyze image content, the machine learning model may be trained using training images as input and training content information as output. By training the machine learning model with a large number of training images and / or training sequences (e.g., words or sentences) and associated training content information (e.g., labels or annotations), the machine learning model "learns" to recognize image content, such that image content not included in the training data can be recognized using the machine learning model. The same principle may be used for other types of sensor data as well: by training the machine learning model with training sensor data and a desired output, the machine learning model "learns" a transformation between sensor data and output, which can be used to provide an output based on the non-training sensor data provided to the machine learning model. The provided data (e.g., sensor data, metadata and / or image data) may be pre-processed to obtain feature vectors that are used as input to machine learning models.

[0058] A machine learning model may be trained using training input data. The above example uses a training method called "supervised learning." In supervised learning, a machine learning model is trained using multiple training samples, where each sample may include multiple input data values ​​and multiple desired output values, i.e., each training sample is associated with a desired output value. By specifying both the training samples and the desired output value, the machine learning model "learns" during training which output value to provide based on input samples similar to the provided sample. In addition to supervised learning, semi-supervised learning may be used. In semi-supervised learning, some of the training samples lack a corresponding desired output value. Supervised learning may be based on a supervised learning algorithm (e.g., a classification algorithm, a regression algorithm, or a similarity learning algorithm). A classification algorithm may be used when the output is restricted to a limited set of values ​​(categorical variables), i.e., the input is classified into one of a limited set of values. A regression algorithm may be used when the output may have any numerical value (within a range). Similarity learning algorithms may be similar to both classification and regression algorithms, but are based on learning from examples using a similarity function that measures how similar or related two objects are. In addition to supervised or semi-supervised learning, unsupervised learning may also be used to train machine learning models. In unsupervised learning, input data may be provided (only), and unsupervised learning algorithms may be used to find structure in the input data (e.g., by grouping or clustering the input data, finding commonalities in the data). Clustering is the assignment of input data containing multiple input values ​​into multiple subsets (clusters) such that input values ​​within the same cluster are similar according to one or more (predefined) similarity criteria, but dissimilar to input values ​​contained in another cluster.

[0059] Reinforcement learning is a third group of machine learning algorithms. In other words, reinforcement learning may be used to train machine learning models. In reinforcement learning, one or more software actors (referred to as "software agents") are trained to take actions in their surroundings. Based on the actions taken, a reward is calculated. Reinforcement learning is based on training one or more software agents to select actions that increase cumulative rewards (as manifested by increasing rewards), resulting in the software agent becoming better at a given task.

[0060] Furthermore, some techniques may be applied to parts of the machine learning algorithm. For example, feature representation learning may be used. In other words, a machine learning model may be trained at least in part using feature representation learning, and / or a machine learning algorithm may include a feature representation learning component. A feature representation learning algorithm, which may be referred to as a representation learning algorithm, may not only preserve information in its input, but may also transform the information to make it useful, often as a preprocessing step before performing classification or prediction. Feature representation learning may be based on, for example, principal component analysis or cluster analysis.

[0061] In some examples, anomaly detection (i.e., outlier detection) may be used, which aims to provide identification of input values ​​that raise suspicion by differing significantly from the majority of the input or training data. In other words, a machine learning model may be trained at least in part with anomaly detection and / or a machine learning algorithm may include an anomaly detection component.

[0062] In some examples, a machine learning algorithm may use a decision tree as a predictive model. In other words, the machine learning model may be based on a decision tree. In a decision tree, an observation about an item (e.g., a set of input values) may be represented by a branch of the decision tree, and an output value corresponding to the item may be represented by a leaf of the decision tree. The decision tree may support both discrete and continuous values ​​as output values. If discrete values ​​are used, the decision tree may be represented as a classification tree, and if continuous values ​​are used, the decision tree may be represented as a regression tree.

[0063] Association rules are another technique that can be used in machine learning algorithms. In other words, a machine learning model may be based on one or more association rules. Association rules are created by identifying relationships between variables in large amounts of data. A machine learning algorithm may identify and / or utilize one or more association rules that represent knowledge derived from the data. These rules may be used, for example, to store, manipulate, or apply the knowledge.

[0064] Machine learning algorithms are typically based on machine learning models. In other words, the term "machine learning algorithm" may refer to a set of instructions that can be used to create, train, or use a machine learning model. The term "machine learning model" may refer to a data structure and / or a set of rules that represent learned knowledge (e.g., based on training performed by a machine learning algorithm). In embodiments, the use of a machine learning algorithm may refer to the use of an underlying machine learning model (or underlying machine learning models). The use of a machine learning model may refer to the machine learning model and / or the set of data structures / rules that are the machine learning model being trained by a machine learning algorithm.

[0065] For example, the machine learning model may be an artificial neural network (ANN). An ANN is a system influenced by biological neural networks, such as those found in the retina or brain. An ANN includes multiple interconnected nodes and multiple junctions, or edges, between the nodes. Typically, there are three types of nodes: input nodes that receive input values, hidden nodes that are (only) connected to other nodes, and output nodes that provide output values. Each node may represent an artificial neuron. Each edge may transmit information from one node to another. The output of a node may be defined as a (nonlinear) function of its input (e.g., the sum of its inputs). The input of a node may be used in a function based on the "weights" of the edges or nodes that provide the input. The weights of the nodes and / or edges may be adjusted during the learning process. In other words, training an artificial neural network may involve adjusting the weights of the nodes and / or edges of the artificial neural network to obtain a desired output for a given input.

[0066] Alternatively, the machine learning model may be a support vector machine, a random forest model, or a gradient boosting model. A support vector machine (i.e., a support vector network) is a supervised learning model with an associated learning algorithm that can be used to analyze data (e.g., in classification or regression analysis). A support vector machine may be trained by providing input with multiple training input values ​​that belong to one of two categories. A support vector machine may be trained to assign new input values ​​to one of two categories. Alternatively, the machine learning model may be a Bayesian network, which is a probabilistic directed acyclic graphical model. A Bayesian network may use a directed acyclic graph to represent a set of random variables and their conditional dependencies. Alternatively, the machine learning model may be based on a genetic algorithm, a search algorithm and a heuristic method that mimics the process of natural selection. [Explanation of symbols]

[0067] 100 training steps 102 Processing Steps 104 repeated training steps 106 Acquired Images 108 Gaussian Blur 110 simulated ground truth images 112 Statistical Models 114 Random Patch 116 Random Patch Enlarged Section 118 Captured image with swapped pixel values 120 Enlarged sections of random patches containing swapped pixel values 122 Convolutional Neural Network (CNN) 124 x m 126s m 128 Typical mathematical expressions 130 acquired images 132 trained CNNs 134 denoised image 136 different acquired images 138 New statistical models 140 Deciding Whether to Retrain a CNN 142 Retraining CNNs 144 Denoising of another acquired image 146 denoised images 148 Updating any previously denoised image and retraining the CNN 800 System 802 Microscope 804 Computer Systems 806 one or more processors 808 graphics processor 810 one or more storage devices 812 Display device 814 keyboard and / or controller 816 Cloud Computing System

Claims

1. 1. A computer-implemented method (400) for training a convolutional neural network, the method comprising: receiving (402) a captured image (106); generating (404) a statistical model (112) based on the captured images; training (406) a convolutional neural network (122) based on the captured images and the statistical model; Including, The step of generating the statistical model comprises: determining a variance or standard deviation value for pixels in the captured image according to a ground truth or a simulated ground truth (110); generating the statistical model based on the variance or the standard deviation; Including, The step of training the convolutional neural network includes: Dividing the captured image into random patches (114); swapping pixel values ​​with adjacent pixel values ​​in each random patch; training the convolutional neural network based on the random patch of the captured image and the random patch of the captured image (118) including swapped pixel values; Including, The step of training the convolutional neural network includes: determining a probability of pixel values ​​of the random patch including swapped pixel values ​​based on the statistical model; training the convolutional neural network by maximizing the probability of each pixel value of the random patch containing swapped pixel values; Including, method.

2. maximizing the probability of each pixel value includes minimizing the negative logarithm of the mean of all pixel probabilities. The method of claim 1.

3. the probability for each pixel value is determined based on a Gaussian distribution having a mean value and a variance or standard deviation value, the mean value being the average value of each pixel in the random patch of the captured image, and the variance or standard deviation value being obtained from the statistical model; 3. The method according to claim 1 or 2.

4. The method includes applying the trained convolutional neural network to the captured image.

4. The method according to any one of claims 1 to 3.

5. The method comprises: receiving another captured image; updating the statistical model based on the captured image and the other captured image; comparing the updated statistical model to determine whether the variance or standard deviation of the updated statistical model has decreased; retraining the convolutional neural network based on the further captured image and the updated statistical model when a decrease in variance or standard deviation value of the updated statistical model is identified compared to the statistical model; Contains, 5. The method according to any one of claims 1 to 4.

6. The convolutional neural network is an encoder-decoder neural network.

6. The method according to any one of claims 1 to 5.

7. A system including one or more processors and one or more storage devices (810), configured to perform the method of any one of claims 1 to 6. system.

8. the system includes an imaging device coupled to the processor for acquiring a microscopic image. The system of claim 7.

9. A computer program comprising program code for implementing the method according to any one of claims 1 to 6.

Images