Real-time denoising method for optical microscopic imaging, optical microscopic imaging system, device and medium
By improving the spatiotemporal redundancy sampling and lightweight design based on the U-Net network, the problems of high real-time performance and high computational resource requirements in existing methods are solved. This enables real-time denoising processing on consumer-grade graphics cards that can adapt to various imaging scenarios, improving both denoising effect and efficiency.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- FUDAN UNIVERSITY
- Filing Date
- 2025-11-07
- Publication Date
- 2026-05-15
AI Technical Summary
Existing self-supervised denoising methods have strict requirements for high temporal or spatial resolution in optical microscopy, resulting in high real-time and computational resource requirements, making them unsuitable for flexible imaging scenarios. Furthermore, the computational consumption of 3D U-shaped networks and Transformer networks is enormous, limiting their application potential.
A denoising model based on the U-Net network is adopted, which combines a spatiotemporal redundancy sampling strategy and self-supervised training. By using a lightweight U-Net network and sparse convolution computation, training pairs are generated for denoising. Skip connections are removed to reduce computational cost, making it suitable for various imaging scenarios.
It enables real-time noise reduction processing on consumer-grade graphics cards, adapts to various imaging scenarios, improves noise reduction effect and computing efficiency, reduces dependence on high-performance hardware, and supports imaging requirements with high temporal and spatial resolution.
Smart Images
Figure CN2025133314_15052026_PF_FP_ABST
Abstract
Description
Real-time noise reduction methods, optical microscopy imaging systems, equipment, and media for optical microscopy imaging. Technical Field
[0001] This invention relates to the field of optical microscopy imaging, and in particular to a real-time noise reduction method, optical microscopy imaging system, device and medium for optical microscopy imaging. Background Technology
[0002] Optical microscopy is highly diverse due to the variations in microscopy techniques, tissue types, cell types, and probes, with different imaging scenarios having different requirements. For example, structural imaging typically aims to maintain higher spatial resolution, while functional imaging seeks higher temporal resolution and imaging depth. Furthermore, the real-time nature of imaging experiments is crucial, as scientists and researchers require high signal-to-noise ratio images for timely image quality assessment or to achieve closed-loop feedback. The diversity of observation methods and objects, along with the need to acquire high signal-to-noise ratio images in real time, presents significant challenges to denoising methods.
[0003] Since the introduction of deep learning denoising methods, self-supervised methods have become a crucial image enhancement step due to their superior denoising performance compared to traditional methods and the fact that they do not require a high signal-to-noise ratio ground truth. These methods utilize the significant spatiotemporal redundancy in fluorescence time-lapse imaging, i.e., repetitive information between temporally adjacent frames and between spatially adjacent pixels, and use neural networks to extract clean image features from the redundant information for denoising.
[0004] Existing self-supervised denoising methods can be divided into two categories: neighbor-based methods and blind-spot-based methods. Among neighbor-based methods, one approach utilizes temporal redundancy to construct a training framework by sampling the time intervals between frames in a series of acquired raw data; the other utilizes spatial redundancy by sampling the spatial pixel intervals within a single frame. However, these methods have strict requirements for high temporal or spatial resolution in imaging; otherwise, oversmoothing and artifacts may occur. Blind-spot-based methods rely on blind-spot degradation in the center frame, using unoccluded pixels from the center and adjacent frames to reconstruct the blind spot. This pixel-level reconstruction task requires significant training time. Furthermore, since this method only yields the denoising result of the center frame after inputting multiple frames, processing the complete imaging data also requires even more time, failing to meet real-time requirements.
[0005] Both of these self-supervised learning denoising methods have improved the signal-to-noise ratio of optical microscopy. However, many challenges remain, including dependence on high temporal or spatial resolution and the need for high computational resources. These challenges severely hinder the realization of real-time denoising in different optical microscopy imaging scenarios. Most existing image enhancement self-supervised learning methods employ networks with large parameter counts or use complex training strategies, leading to significant space and time consumption. Three-dimensional U-shaped networks receive large amounts of three-dimensional data and perform complex three-dimensional convolution operations, resulting in enormous space consumption. Transformer networks, due to their self-attention mechanism, require substantial computation and thus have significant time consumption. Complex training strategies proposed for specific imaging scenarios prevent them from adapting to flexible scene changes, further limiting their application potential. Summary of the Invention
[0006] The purpose of this invention is to overcome the shortcomings of the prior art by providing a real-time denoising method, optical microscopy imaging system, device, and medium with good real-time performance and scene generalization. The objective of this invention can be achieved through the following technical solutions:
[0007] According to a first aspect of the present invention, a real-time denoising method for optical microscopy imaging is provided, comprising:
[0008] Acquire image data output by an optical microscopy imaging system;
[0009] A denoising model based on the U-Net network is used to denoise the image data to obtain denoised optical microscopy imaging results. In this process, a spatiotemporal redundancy sampling strategy is used to generate training pairs, and the denoising model is trained through self-supervised training.
[0010] Preferably, the generation of training pairs using a spatiotemporal redundancy sampling strategy includes the following temporal sampling process:
[0011] For a training stack with H × W × T pixels, where H, W, and T are the height, width, and length of the input image stack, respectively;
[0012] In terms of time, the training stack is divided into an input stack Xi and a target stack Yi using a sliding window method. The size of both the input stack and the target stack is H×W×C. The initial frames differ by S frames, where C and S are the window width and step size of the sliding window, respectively, and S≤C.
[0013] Preferably, the generation of training pairs using a spatiotemporal redundancy sampling strategy includes the following spatial sampling process:
[0014] For the input stack Xi and the target stack Yi, spatial sampling is performed on each image using sparse convolution calculation to obtain two sub-stacks G1(Xi) and G2(Xi) of the input stack Xi, and two sub-stacks G1(Yi) and G2(Yi) of the target stack Yi.
[0015] The same window of the input stack Xi and the target stack Yi uses the same random seed, so the original pixel positions of the sub-stacks G1(Xi) and G1(Yi) are the same, the original pixel positions of G2(Xi) and G2(Yi) are the same, G1(Xi) and G2(Yi) satisfy the spatiotemporal adjacency relationship, and G2(Xi) and G1(Yi) satisfy the spatiotemporal adjacency relationship.
[0016] Xi and G1(Xi) are selected as input stacks, and Yi and G2(Yi) are selected as target stacks to generate training pairs for self-supervised training of the denoising model.
[0017] Preferably, sparse convolution is used for spatial sampling to select neighboring pixels. Specific parameter settings include:
[0018] The sparse convolution calculation uses a window size of a×a pixels and a stride of b. The convolution kernel for each window is randomly selected from a list of mask units. The mask unit is an a×a binary matrix with two elements having a value of 1, used to select adjacent pixel positions. Here, a and b are customized according to the processing speed requirements and the spatial resolution of the image.
[0019] Preferably, the U-Net network model is a lightweight U-Net network model, which reduces the number of downsampling operations to two and removes the top-level skip connections from the original U-Net network model.
[0020] Preferably, the lightweight U-Net network model includes two encoding blocks and two decoding blocks, specifically:
[0021] Two coding blocks are used to reduce dimensionality, each consisting of a convolutional layer followed by a BatchNorm, a ReLU, and a max pooling layer.
[0022] Two decoding blocks are used to recover the input dimension. Each decoding block contains a nearest neighbor interpolation, followed by a convolutional layer, a BatchNorm, and a ReLU.
[0023] At each resolution depth, skip connections are used to connect the output of the second coded block to the input of the sibling coded block.
[0024] According to a second aspect of the present invention, an optical microscopy imaging system is provided, wherein the graphics processor of the system is equipped with a noise reduction module, and the noise reduction module performs noise reduction using any of the methods described in the present invention.
[0025] Preferably, the system displays images before and after noise reduction synchronously on its display.
[0026] According to a third aspect of the present invention, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement any of the methods described above.
[0027] According to a fourth aspect of the invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements any of the methods described herein. Beneficial effects
[0028] (1) Based on the characteristics that signals between temporally adjacent frames and signals between spatially adjacent pixels are similar while noise is independent, this invention utilizes the fact that sub-images obtained by temporally and spatially adjacent downsampling in the original imaging data contain similar signals and independent noise, which can be used as training pairs for neural networks to optimize the parameters of the denoising network. This training method does not have strict requirements on the temporal and spatial resolution of the original image, so it can be applied to various imaging scenarios.
[0029] (2) This invention treats spatial sampling as a downsampling and makes a lightweight improvement to the U-Net network by reducing the number of downsampling times to two, thereby reducing the number of parameters and computation of the denoising network. It can achieve a denoising processing speed that exceeds the imaging speed. The real-time capability does not depend on high-performance hardware and can run on mainstream consumer-grade graphics cards. At the same time, the top-level skip connection in the U-Net network is removed, which can effectively suppress the noise characteristics of the upper layers and further improve the denoising effect.
[0030] (3) The microscopic imaging system of the present invention achieves continuous data acquisition, multi-threaded parallel processing and real-time synchronous display of images before and after denoising by using a dual-queue data inflow and outflow method including data acquisition, multi-threaded parallel processing and real-time synchronous display. Real-time denoising of microscopic imaging can be achieved on mainstream consumer-grade graphics cards without relying on high-performance hardware. Attached Figure Description
[0031] Figure 1 is a flowchart of the method of the present invention;
[0032] Figure 2 is an architecture diagram of the real-time denoising method for optical microscopy proposed in this invention; where (a) is the input and output of the network after temporal and spatial sampling of the acquired image, and the network parameters are optimized by scale constraints and spatiotemporal self-supervision; (b) is a schematic diagram of the spatial sampling strategy.
[0033] Figure 3 shows the network structure diagram; where (a) is the classic two-dimensional U-Net, (b) is the three-dimensional U-Net, and (c) is the ultralight two-dimensional U-Net proposed in this invention;
[0034] Figure 4 shows the enhanced neuron segmentation effect in the embodiment; where (a)~(f) are comparisons of image denoising and neuron segmentation results of the method of the present invention and other methods, (g) is the quantitative evaluation index of neuron segmentation, and (h) is a comparison of denoising and segmentation results on three specific neurons.
[0035] Figure 5 is a schematic diagram of the recovery of high temporal precision neuronal activity using the present invention in the embodiment; where (a) is the difference between the image and signal of voltage imaging data before and after processing by the method of the present invention, (b) is the detailed presentation of the time range marked by the red dashed box in (a) after magnification, and (c) is the Pearson correlation coefficient between the signal obtained after processing by the method of the present invention and the true value of the electrophysiological signal.
[0036] Figure 6 is a schematic diagram of the volumetric structure imaging results in the embodiment; where (a) is a comparison before and after using the method of the present invention to enhance the volumetric imaging of skin optical coherence tomography, and (b) is a single slice of the result in (a). Embodiments of the present invention
[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0038] Example
[0039] As shown in Figure 1, this embodiment provides a real-time denoising method for optical microscopy imaging, including:
[0040] Acquire image data output by an optical microscopy imaging system;
[0041] A denoising model based on the U-Net network is used to denoise the image data to obtain denoised optical microscopy imaging results. In this process, a spatiotemporal redundancy sampling strategy is used to generate training pairs, and the denoising model is trained through self-supervised training.
[0042] The method of this embodiment will now be described in detail.
[0043] (1) Spatiotemporal redundancy sampling
[0044] The acquired images are temporally and spatially sampled and then used as the input and output of a denoising model based on the U-Net network. The parameters of the denoising model are optimized through scale constraints and spatiotemporal self-supervision, as follows:
[0045] A spatiotemporal redundancy sampling strategy is used to generate training pairs for a training stack with H × W × T pixels, where H, W and T are the height, width and length of the input image stack, respectively.
[0046] First, time sampling is performed. The training stack is divided into an input stack Xi and a target stack Yi in time using a sliding window method. The size of the input stack and the target stack is H×W×C. The initial frames differ by S frames, where C and S are the window width and stride of the sliding window, respectively, and S≤C.
[0047] For the input stack Xi and the target stack Yi, spatial sampling is performed on each image using a method similar to sparse convolution in neural networks, resulting in two sub-stacks G1(Xi) and G2(Xi) for the input stack Xi, and two sub-stacks G1(Yi) and G2(Yi) for the target stack Yi.
[0048] The same window of the input stack Xi and the target stack Yi uses the same random seed, so the original pixel positions of the sub-stacks G1(Xi) and G1(Yi) are the same, the original pixel positions of G2(Xi) and G2(Yi) are the same, G1(Xi) and G2(Yi) satisfy the spatiotemporal adjacency relationship, and G2(Xi) and G1(Yi) satisfy the spatiotemporal adjacency relationship.
[0049] Xi and G1(Xi) are selected as input stacks, and Yi and G2(Yi) are selected as target stacks for self-supervised training of the denoising network.
[0050] Then, spatial sampling is performed to select neighboring pixels. For spatial sampling, a mask is randomly selected from a list of mask cells, and the complete image is split into two sub-images according to the mask markings.
[0051] Spatial sampling can be achieved through sparse convolution or other methods. To accelerate the sampling speed, this embodiment uses sparse convolution, with the following parameter settings: the window size for sparse convolution calculation is a×a pixels, the stride is b, and the convolution kernel for each window is randomly selected from a list of mask units. The mask unit is an a×a binary matrix, where two elements have a value of 1, used to select adjacent pixel positions. Parameters a and b can be customized according to processing speed requirements and the spatial resolution of the image.
[0052] In this embodiment, the window size for sparse convolution calculation is 2×2 pixels with a stride of 2. The convolution kernel of each window is randomly selected from a list of 16 mask units. The mask unit is a 2×2 binary matrix with two elements having a value of 1, which is used to select the positions of adjacent pixels.
[0053] (2) Denoising model based on U-Net network
[0054] In this embodiment, a lightweight U-Net network is selected.
[0055] Similar to the original U-Net model, the lightweight U-Net network in this embodiment also uses an encoder-decoder architecture and skip connections. Two encoding blocks are used to reduce dimensionality; each block consists of a 3×3 convolutional layer followed by a BatchNorm, a ReLU, and a 3×3 max-pooling layer. Two decoding blocks are used to restore the input dimension; each decoding block contains a nearest-neighbor interpolation, followed by a 3×3 convolutional layer, a BatchNorm, and a ReLU. At each resolution depth, the number of channels in each feature map is 64, 64, and 64, respectively. Skip connections are used to connect the output of the second encoding block to the input of its sibling encoding block.
[0056] Compared to the original U-Net network, the lightweight U-Net network of this invention has the following two modifications:
[0057] First, the downsampling times are reduced to two. Downsampling can increase the receptive field, enabling the convolutional kernel to extract features over a larger image area. However, as the number of downsampling times increases, the number of network parameters also increases dramatically. The spatial subsampler of this invention can be regarded as a single downsampling step, which has the function of expanding the receptive field. Therefore, the number of downsampling times of U-net can be reduced to shrink the network.
[0058] Second, the top-level skip connections are removed. Skip connections can fuse low-level and high-level features; however, in denoising tasks, high-level features are noise and need to be suppressed. Therefore, removing the top-level skip connections helps improve denoising performance. By minimizing network depth, the number of channels per feature map, and the number of skip connections, in this embodiment, the network is trained with only 0.13M parameters.
[0059] This embodiment also provides an optical microscopic imaging system, in which a denoising module is installed in the graphics processor. The denoising module performs denoising using the method described above, and the system display shows the images before and after denoising simultaneously.
[0060] The above-mentioned denoising model is pre-trained using the self-supervised training method described above, and the trained denoising model is loaded into the graphics processing unit (GPU) as a denoising module.
[0061] The raw output data stream of the imaging system is used as input and enters the processing queue. The images in the processing queue are continuously fed into the graphics processor with the denoising model loaded. The denoised data enters the completion queue, and finally the images before and after denoising are synchronously displayed on the monitor.
[0062] The microscopic imaging system in this embodiment achieves continuous data acquisition, multi-threaded parallel processing, and real-time synchronous display of images before and after denoising through a dual-queue data inflow and outflow method that includes data acquisition, multi-threaded parallel processing, and real-time synchronous display. Real-time denoising of microscopic imaging can be achieved on mainstream consumer-grade graphics cards without relying on high-performance hardware.
[0063] Next, the effectiveness of the noise reduction method of the present invention will be experimentally verified.
[0064] Experiment 1: Enhancing Neuron Segmentation Results
[0065] Background: Fluorescent genetically encoded calcium indicators and two-photon microscopy can simultaneously measure the activity of hundreds to thousands of neurons to study the correlation between neural activity and neural function. Neuron segmentation is a crucial step in analyzing neuronal population calcium imaging, but noise can reduce the number and accuracy of segments, and the loss of neurons overwhelmed by noise can adversely affect the analysis.
[0066] Data preparation: Validation was performed using the Neurofinder dataset, which contains manually labeled segmentation benchmarks. The raw data was contaminated with noise and shot noise, as shown in part 4(a) in Figure 4, which severely damaged the morphology and contours of neurons, which are key features for segmentation.
[0067] Noise Reduction: Two-photon calcium imaging data were processed using the present invention and four other denoising methods, as shown in Figures 4(b) to (f). After processing with the present invention, the morphology of neurons was restored from the noise, cell boundaries were clear, neuronal synapses were visible in the magnified area, and they emitted fluorescence along with the neurons. The DeepCAD-RT results showed smooth transitions, with synaptic structures almost indistinguishable from the background. The DeepVid and SUPPORT results not only failed to observe image details but also contained some artifacts. The SRDTrans results were clear and sharp, but due to the complex Transformer network, processing consumed a significant amount of time and memory.
[0068] Segmentation Analysis: Cellpose was used to segment neurons in the five denoising results. SRDTrans segmented the most extra neurons (29), while SUPPORT lost the most neurons (57). The image enhanced by this invention segmented the most neurons (125) and had the fewest extra neurons (19). The segmented magnified images from three denoising methods were selected, as shown in part (g) of Figure 4. Only this invention could completely identify neurons, with a deviation of only 1-2 pixels from the baseline.
[0069] Performance evaluation: The denoising method of the present invention achieved the highest F1 score (0.64), indicating that the denoising method of the present invention can enhance calcium imaging of neuronal populations to facilitate accurate and comprehensive neuronal segmentation.
[0070] Results: The processed neurons showed significant morphological restoration, with clear cell boundaries and visible synapses. This invention demonstrated the best performance in terms of the number and accuracy of segmented neurons, achieving the highest F1 score.
[0071] Conclusion: The denoising method of the present invention significantly improves the accuracy and comprehensiveness of neuron segmentation, providing reliable data support for the study of neural activity and neural function.
[0072] Experiment 2: Recovering high-temporal-precision neuronal activity
[0073] Background: High-temporal-precision recording of neuronal activity is crucial for studying patterns of neuronal activity. The development of voltage indicators necessitates higher imaging speeds, requiring voltage imaging rates of up to 1 kHz. However, due to the inherent trade-off between imaging speed and the maximum number of photons collected, a decrease in signal-to-noise ratio (SNR) is unavoidable. This decrease impairs the recorded neuronal morphology and signal; neuronal activity signals may be masked or even obscured by noise. Denoising can recover these signals, enabling accurate, high-precision recording of neuronal activity.
[0074] Data preparation: The activity of individual QuasAr6a-expressing neurons in the second and third layers of the mouse cortex was recorded using wide-field microscopy (imaging speed 1 kHz) and synchronized patch-clamp electrophysiological recordings were performed. The raw data were significantly contaminated with noise, which severely disrupted the spatial footprint and temporal trajectory of neurons, as shown in part 5(a) of Figure 5.
[0075] Noise reduction: By enhancing these data using the present invention, cell boundaries become identifiable, and voltage trajectories emerge from the noise. Fluorescence was randomly extracted from 100 cytoplasmic pixels, and it was found that after enhancement using the present invention, voltage fluctuations could be displayed at the single-pixel level, as shown in part (b) of Figure 5.
[0076] Performance evaluation: The mean Pearson correlation coefficient between voltage trajectories and electrophysiological recordings increased from 0.13 to 0.29, as shown in part (c) of Figure 5.
[0077] Results: The neuronal morphology and voltage trajectories were significantly restored after processing with this invention, cell boundaries were clear, and voltage fluctuations were visible. This invention significantly improved the correlation between voltage trajectories and electrophysiological recordings, increasing it from 0.13 to 0.29. Denoising results of in vivo population neuronal voltage imaging data showed that neuronal morphology and structure were restored, and voltage transients were preserved.
[0078] Conclusion: This invention can effectively denoise high temporal resolution single neuron and group neuron voltage imaging data, while faithfully recovering voltage transients, providing reliable technical support for high-precision recording of neuronal activity.
[0079] Experiment 3: Enhanced Volumetric Structure Imaging
[0080] Background: Optical coherence tomography (OCT) is a non-invasive imaging technique widely used in the diagnosis and research of skin diseases. OCT can provide high-resolution two-dimensional and three-dimensional images, revealing the microscopic structure of skin tissue. However, image quality is severely affected by noise interference during OCT imaging, especially in volume imaging, where noise is a more prominent issue. Noise not only reduces image clarity but can also lead to misinterpretations of skin tissue structure and pathological changes. Therefore, developing an effective denoising method is crucial for improving the quality of OCT volume imaging. In skin OCT volume imaging, noise mainly originates from several aspects: First, hardware limitations of the OCT system itself, such as the stability of the light source and the sensitivity of the detector, introduce noise; second, the influence of ambient light, sample scattering and absorption during imaging also increase noise; finally, algorithm errors and computational accuracy issues during data processing also lead to noise accumulation.
[0081] Conclusion: Figure 6 shows a schematic diagram of the volumetric imaging results using the method of this invention, where (a) is a before-and-after comparison of enhancing skin optical coherence tomography volumetric imaging using the method of this invention, and (b) is a single slice of the result in (a). It can be seen that the method of this invention can effectively remove noise in volumetric imaging, improving image clarity and resolution. Experimental results show that this method performs excellently in both image quality improvement and noise reduction, and has significant potential application value. For example, it helps improve the resolution of skin OCT volumetric imaging, clearly displaying the fine structures of skin tissue, such as the boundaries between the epidermis, dermis, and subcutaneous tissue, making the detection of lesions more accurate and complete. The denoised image can better reflect the location and extent of the lesion, providing a more reliable basis for the early diagnosis and treatment of skin diseases.
[0082] Conclusion: The application of this invention in the volumetric time-series structural and functional imaging of astrocytes demonstrates its scalability across different fluorescence and dimensions, providing reliable technical support for the observation of heterogeneous activity.
[0083] The electronic device of this invention includes a central processing unit (CPU) and a graphics processing unit (GPU), which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) or loaded from storage units into random access memory (RAM). The RAM can also store various programs and data required for device operation. The CPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.
[0084] Multiple components in the device are connected to the I / O interface, including: input units such as keyboards and mice; output units such as various types of displays and speakers; storage units such as disks and optical discs; and communication units such as network interface cards (NICs), modems, and wireless transceivers. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0085] The processing unit performs the various methods and processes described above. For example, in some embodiments, the methods may be implemented as computer software programs tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via ROM and / or a communication unit. When the computer program is loaded into RAM and executed by the central processing unit (CPU) and graphics processing unit (GPU), one or more steps of the methods described above may be performed. Alternatively, in other embodiments, the central processing unit (CPU) and graphics processing unit (GPU) may be configured to execute the methods by any other suitable means (e.g., by means of firmware).
[0086] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload programmable logic devices (CPLDs), and so on.
[0087] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.
[0088] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0089] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A real-time denoising method for optical microscopy imaging, characterized in that, include: Acquire image data output by an optical microscopy imaging system; A denoising model based on the U-Net network is used to denoise the image data to obtain denoised optical microscopy imaging results. In this process, a spatiotemporal redundancy sampling strategy is used to generate training pairs, and the denoising model is trained through self-supervised training.
2. The real-time noise reduction method for optical microscopy imaging according to claim 1, characterized in that, The training pairs are generated using a spatiotemporal redundancy sampling strategy, wherein the time sampling process is specifically as follows: For a training stack with H × W × T pixels, where H, W, and T are the height, width, and length of the input image stack, respectively; In terms of time, the training stack is divided into an input stack Xi and a target stack Yi using a sliding window method. The size of both the input stack and the target stack is H×W×C. The initial frames differ by S frames, where C and S are the window width and step size of the sliding window, respectively, and S≤C.
3. The real-time noise reduction method for optical microscopy imaging according to claim 2, characterized in that, The training pairs are generated using a spatiotemporal redundancy sampling strategy, wherein the spatial sampling process is specifically as follows: For the input stack Xi and the target stack Yi, spatial sampling is performed on each image using sparse convolution calculation to obtain two sub-stacks G1(Xi) and G2(Xi) of the input stack Xi, and two sub-stacks G1(Yi) and G2(Yi) of the target stack Yi. The same window of the input stack Xi and the target stack Yi uses the same random seed, so the original pixel positions of the sub-stacks G1(Xi) and G1(Yi) are the same, the original pixel positions of G2(Xi) and G2(Yi) are the same, G1(Xi) and G2(Yi) satisfy the spatiotemporal adjacency relationship, and G2(Xi) and G1(Yi) satisfy the spatiotemporal adjacency relationship. Xi and G1(Xi) are selected as input stacks, and Yi and G2(Yi) are selected as target stacks to generate training pairs for self-supervised training of the denoising model.
4. The real-time denoising method for optical microscopy imaging according to claim 3, characterized in that, Spatial sampling is performed using sparse convolution computation. Neighboring pixels are selected through spatial sampling. Specific parameter settings include: The sparse convolution calculation uses a window size of a×a pixels and a stride of b. The convolution kernel for each window is randomly selected from a list of mask units. The mask unit is an a×a binary matrix with two elements having a value of 1, used to select adjacent pixel positions. Here, a and b are customized according to the processing speed requirements and the spatial resolution of the image.
5. A real-time noise reduction method for optical microscopy imaging according to claim 1, characterized in that, The U-Net network model is a lightweight U-Net network model, which reduces the number of downsampling operations to two and removes the top-level skip connections from the original U-Net network model.
6. A real-time denoising method for optical microscopy imaging according to claim 5, characterized in that, The lightweight U-Net network model comprises two encoding blocks and two decoding blocks, specifically: Two coding blocks are used to reduce dimensionality, each consisting of a convolutional layer followed by a BatchNorm, a ReLU, and a max pooling layer. Two decoding blocks are used to recover the input dimension. Each decoding block contains a nearest neighbor interpolation, followed by a convolutional layer, a BatchNorm, and a ReLU. At each resolution depth, skip connections are used to connect the output of the second coded block to the input of the sibling coded block.
7. An optical microscopic imaging system, characterized in that, The system's graphics processor is equipped with a noise reduction module, which performs noise reduction using the method described in any one of claims 1 to 6.
8. The system according to claim 7, characterized in that, The system's display shows the images before and after noise reduction synchronously.
9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 6.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 6.