A method for enhancing the image processing of fluorescent labeled cell imaging
By combining the two-layer residual polymerization structure and the extended dual attention mechanism of the receptive field, the problem of low resolution and signal-to-noise ratio in fluorescence microscopy imaging is solved, and high-precision single-image super-resolution reconstruction is achieved. The quality of detail retention and reconstruction of fluorescently labeled cell imaging images has been significantly improved.
Patent Information
- Application Number
- CN202410977044.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-22
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-07-22
AI Technical Summary
The prior art has limited resolution, low signal-to-noise ratio and blurred image in fluorescence microscopy imaging, making it difficult to take into account high resolution and deep penetration, and the residual network has shortcomings in detail preservation.
Using a deep neural network model combining two-layer residual aggregation structure and a receptive field extended dual attention mechanism, the imaging images of fluorescent labeled cells are processed, and the loss of residual information is reduced through jump connections. Multi-scale convolutional kernels capture context information, space -Channel dual attention mechanism enhances discriminative learning.
The accuracy and quality of image reconstruction are significantly improved, and the details are retained better. The experimental results are better than existing models on the Set5, Set14, BSD100 and Urban100 datasets, showing significant performance advantages.
Smart Images

Figure CN118521497B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to image processing technology, and in particular to a deep learning network structure for image enhancement processing of fluorescent labeled cell imaging and an application method thereof. Background Art
[0002] Fluorescence microscopy is widely used in biomedical research, but it has problems such as limited resolution, low signal-to-noise ratio and blurred images. Traditional methods such as three-dimensional structured illumination microscopy and high numerical aperture light sheet microscopy have their own advantages and disadvantages, and it is difficult to balance high resolution and deep penetration. In recent years, deep learning methods have made significant progress in the field of image processing, but the existing residual network has shortcomings in utilizing residual features and preserving details. Summary of the invention
[0003] The purpose of the present invention is to provide an image enhancement processing method for fluorescent labeled cell imaging, which uses a deep neural network model that combines a two-layer residual aggregation structure and a receptive field expansion dual attention mechanism to achieve high-precision single image super-resolution reconstruction and improve the detail retention and reconstruction quality of fluorescent labeled cell imaging images.
[0004] To achieve the above purpose, the technical solution adopted by the present invention is as follows:
[0005] A method for enhancing the image processing of fluorescent labeled cell imaging, characterized in that the method comprises the following steps:
[0006] Acquire low signal-to-noise ratio image data of fluorescently labeled cell imaging; use a deep neural network model to process the low signal-to-noise ratio image data, the deep neural network model includes a two-layer residual aggregation structure, a multi-scale receptive field expansion module and a space-channel dual attention mechanism; the two-layer residual aggregation structure forms a nested residual aggregation network through jump connections, and hierarchically aggregates and fuses the residual information extracted from each layer of the network to reduce the loss of residual information containing image details; the multi-scale receptive field expansion module captures contextual information of different scales through convolution kernels of different scales; the space-channel dual attention mechanism enhances the discriminative learning ability of the residual network and improves the quality of the reconstructed image.
[0007] Furthermore, preferably, the deep neural network model further includes a plurality of residual aggregation blocks, each residual aggregation block includes a plurality of residual blocks, and forms a nested structure through jump connections.
[0008] Further, preferably, the residual block includes the following steps: performing a convolution operation on the input image to obtain an initial residual feature; performing a nonlinear transformation on the initial residual feature through a ReLU activation function; performing a convolution operation again to obtain an enhanced residual feature; and performing a jump connection between the enhanced residual feature and the input image to obtain an output residual feature.
[0009] Furthermore, preferably, the multi-scale receptive field expansion module includes multiple convolution kernels of different scales for capturing contextual information of different scales, and is implemented by the following steps: performing a multi-scale convolution operation on the input image to obtain feature maps of different scales; splicing and fusing the feature maps of different scales to obtain multi-scale receptive field features; and inputting the multi-scale receptive field features into a subsequent residual aggregation network for processing.
[0010] Further, preferably, the spatial-channel dual attention mechanism includes the following steps: performing a channel attention operation on the input feature map to obtain a channel weight; applying the channel weight to the input feature map to obtain a channel weighted feature map; performing a spatial attention operation on the channel weighted feature map to obtain a spatial weight; applying the spatial weight to the channel weighted feature map to obtain a final attention-enhanced feature map.
[0011] Further, preferably, the channel attention operation includes the following steps: performing global average pooling on the input feature map to obtain a channel description vector; performing nonlinear transformation on the channel description vector through two fully connected layers to obtain channel weights; and normalizing the channel weights to ensure the stability of the weights.
[0012] Further, preferably, the spatial attention operation includes the following steps: performing average pooling and maximum pooling on the channel weighted feature map to obtain two sets of spatial description vectors; fusing the two sets of spatial description vectors through a convolutional layer to obtain spatial weights; and normalizing the spatial weights to ensure the stability of the weights.
[0013] Furthermore, preferably, the training process of the deep neural network model includes the following steps: using high signal-to-noise ratio fluorescent labeled cell imaging data as a supervisory signal to train low signal-to-noise ratio data; optimizing model parameters using a mean square error loss function and an Adam optimizer; and avoiding model overfitting through cross-validation and early stopping strategies.
[0014] Furthermore, preferably, the training data includes labeled fluorescence microscopic image data of different organelles to ensure the diversity and coverage of the training data.
[0015] Furthermore, preferably, the deep neural network model is further optimized to adapt to the memory limitations of the graphics processing unit, thereby improving the efficiency of the model by reducing the number of parameters and computational complexity.
[0016] Compared with the prior art, the present invention has the following beneficial effects:
[0017] Compared with the prior art, the innovations and advantages of the present invention are as follows: the two-layer residual aggregation structure forms a nested residual aggregation network through jump connections, which can more effectively aggregate and fuse residual information at different levels, reduce the loss of detail information, and improve the accuracy of image reconstruction; the multi-scale receptive field expansion module designs a multi-scale convolution kernel structure, which can capture contextual information at different scales, enhance the model's ability to extract multi-scale features, and thus improve the quality of image reconstruction; the spatial-channel dual attention mechanism enhances the model's ability to discriminate features by introducing attention mechanisms at two levels, channel and space, so that the reconstructed image quality is higher and the details are clearer; the experiment proves that the effect is superior: the experimental results on Set5, Set14, BSD100 and Urban100 data sets show that the model of the present invention is superior to the existing mainstream models in both objective indicators and subjective visual effects, and has significant performance advantages. Through these innovations and improvements, the present invention provides an efficient and accurate fluorescent labeled cell imaging image enhancement processing method, which can significantly improve image quality in practical applications and meet the high-demand scientific research and medical diagnosis needs. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0019] Figure 1 A flowchart of a method for enhancing the image processing of fluorescently labeled cell imaging provided by one embodiment of the present invention;
[0020] Figure 2 A residual block flow chart of an image enhancement processing method for fluorescent labeled cell imaging provided by one embodiment of the present invention;
[0021] Figure 3 A multi-scale receptive field expansion module flow chart of a fluorescent labeled cell imaging image enhancement processing method provided by an embodiment of the present invention;
[0022] Figure 4 A flow chart of a spatial-channel dual attention mechanism of an image enhancement processing method for fluorescent labeled cell imaging provided by one embodiment of the present invention;
[0023] Figure 5 This is a diagram of experimental results of an image enhancement processing method for fluorescent labeled cell imaging provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0024] The present invention is further described in detail below in conjunction with embodiments.
[0025] Those skilled in the art will appreciate that the following examples are only used to illustrate the present invention and should not be considered to limit the scope of the present invention. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art of the present invention.
[0026] The specific scheme of the image enhancement processing method for fluorescent labeled cell imaging provided by the present invention is described in detail below with reference to the accompanying drawings.
[0027] See also Figures 1 to 4 , which shows a step flow chart of a fluorescent labeled cell imaging image enhancement processing method provided by an embodiment of the present invention, a residual block flow chart, a multi-scale receptive field expansion module flow chart, and a space-channel dual attention mechanism flow chart, the method comprising the following steps:
[0028] First, obtain low signal-to-noise ratio and high signal-to-noise ratio image data of fluorescently labeled cell imaging. In order to ensure the diversity and coverage of the data, labeled fluorescent microscopy image data of different organelles were selected and divided into training sets and test sets. Each data set is carefully annotated to ensure that high signal-to-noise ratio images can be used as supervisory signals to help the model better learn the enhanced characteristics of low signal-to-noise ratio images. In order to ensure the quality and reliability of the data, these image data need to undergo a rigorous acquisition and processing process. When acquiring images, high-precision microscopes and advanced imaging technologies such as OlympusIX83 microscopes and Andor Zyla 4.2 PLUS sCMOS cameras are used. The resolution of the microscope is set to 2048x2048 pixels to ensure the resolution and clarity of the image, the objective lens is 40x, the exposure time is 100ms, and the gain is set to 1.
[0029] Data preprocessing includes three parts: background noise removal, normalization, and image segmentation. First, use a Gaussian filter to smooth the image and remove high-frequency noise. This step can effectively reduce the random noise in the image, making subsequent processing more stable. This step can be implemented through programming, such as using OpenCV and NumPy libraries in Python. Removing background noise is the first step. A Gaussian filter is used to smooth the image to reduce the impact of high-frequency noise while retaining the main details of the image. In the Gaussian filtering process, the filter size can be set to 5x5 with a standard deviation of 1.0. These parameters need to be adjusted according to the experimental results to achieve the best denoising effect. The following is the code to implement Gaussian filtering:
[0030] import cv2
[0031] import numpy as np
[0032] def gaussian_filter(image):
[0033] return cv2.GaussianBlur(image, (5, 5), 1.0)
[0034] image = cv2.imread('low_snr_image.png', cv2.IMREAD_GRAYSCALE)
[0035] filtered_image = gaussian_filter(image)
[0036] cv2.imwrite('filtered_image.png', filtered_image)
[0037] Then, the image pixel values are normalized to the interval [0, 1], which can avoid the problem of computational instability caused by too large or too small values, and is also conducive to the training of the neural network. Normalization can eliminate the brightness differences between different images and enhance the generalization ability of the model. Finally, the preprocessed image is cut into small blocks suitable for the input size of the neural network, such as 64x64 pixels. This not only increases the amount of data samples, but also ensures that each small block contains enough feature information. The code for normalization is as follows:
[0038] def normalize_image(image):
[0039] return image / 255.0
[0040] def split_image(image, block_size):
[0041] h, w = image.shape
[0042] return [image[i:i+block_size, j:j+block_size] for i inrange(0, h,block_size) for j in range(0, w, block_size)]
[0043] normalized_image = normalize_image(filtered_image)
[0044] image_blocks = split_image(normalized_image, 64)
[0045] After normalization and segmentation, the image blocks also need to undergo data enhancement processing, such as rotation, flipping, and scaling, to increase the diversity of the data. This can enhance the model's ability to adapt to different scenarios. The data enhancement code is as follows:
[0046] def augment_image(image):
[0047] augmented_images = [image, np.rot90(image), np.fliplr(image),np.flipud(image)]
[0048] return augmented_images
[0049] augmented_blocks = [augment_image(block) for block in image_blocks]
[0050] In order to build a high-quality training dataset, each image block needs to be labeled to ensure the correspondence between low signal-to-noise ratio images and high signal-to-noise ratio images. The labeling work requires the use of professional image labeling tools to ensure the accuracy and consistency of the labeling. After the labeling is completed, the image blocks and the corresponding labels can be stored in the dataset file. During the dataset construction process, the HDF5 format can be used to store large-scale image data. HDF5 is an efficient data storage format that supports efficient random access and data compression. The following is the code for storing training data using HDF5:
[0051] import h5py
[0052] def save_to_hdf5(data, labels, filename='dataset.h5'):
[0053] with h5py.File(filename, 'w') as f:
[0054] f.create_dataset('data', data=data)
[0055] f.create_dataset('labels', data=labels)
[0056] data = np.array(augmented_blocks)
[0057] labels = np.array(augmented_blocks)
[0058] save_to_hdf5(data, labels)
[0059] After completing the dataset construction, you can use deep learning frameworks such as Keras or PyTorch to load the dataset and perform data preprocessing and batch processing to facilitate subsequent model training. The code for loading the HDF5 dataset using PyTorch is as follows:
[0060] import torch
[0061] from torch.utils.data import Dataset, DataLoader
[0062] class HDF5Dataset(Dataset):
[0063] def __init__(self, hdf5_file):
[0064] self.file = hdf5_file
[0065] self.data = h5py.File(self.file, 'r')['data']
[0066] self.labels = h5py.File(self.file, 'r')['labels']
[0067] def __len__(self):
[0068] return len(self.data)
[0069] def __getitem__(self, idx):
[0070] data = self.data[idx]
[0071] label = self.labels[idx]
[0072] return torch.tensor(data, dtype=torch.float32), torch.tensor(label, dtype=torch.float32)
[0073] dataset = HDF5Dataset('dataset.h5')
[0074] dataloader = DataLoader(dataset, batch_size=32, shuffle=True)
[0075] Then, a deep neural network model combining a two-layer residual aggregation structure and a receptive field expansion dual attention mechanism is designed. The model consists of multiple modules, including an input convolutional layer, a residual aggregation block, a multi-scale receptive field expansion module, and a spatial-channel dual attention mechanism. The model specifically includes the following components:
[0076] Two-layer residual aggregation structure: A two-layer nested residual aggregation network is formed through jump connections, and a large amount of residual information extracted from each layer of the network is hierarchically aggregated and fused to reduce the loss of residual information containing image details.
[0077] Multi-scale receptive field expansion module: It captures contextual information of different scales through convolution kernels of different scales, and is implemented through the following steps: performing multi-scale convolution operations on the input image to obtain feature maps of different scales; splicing and fusing feature maps of different scales to obtain multi-scale receptive field features; and inputting the multi-scale receptive field features into the subsequent residual aggregation network for processing.
[0078] Spatial-channel dual attention mechanism: Enhance the discriminative learning ability of the residual network and improve the quality of the reconstructed image. The specific steps are as follows: perform channel attention operation on the input feature map to obtain channel weights; apply the channel weights to the input feature map to obtain channel weighted feature maps; perform spatial attention operation on the channel weighted feature map to obtain spatial weights; apply the spatial weights to the channel weighted feature map to obtain the final attention-enhanced feature map.
[0079] Through the collaborative work of these modules, the model can effectively extract and enhance image features, thereby achieving high-quality image reconstruction. The two-layer residual aggregation structure forms a nested residual aggregation network through jump connections, and hierarchically aggregates and fuses the residual information extracted from each layer of the network. This can reduce the loss of residual information containing image details and improve the accuracy of image reconstruction. The number of residual blocks is set to 16, and the specific structure of each residual block includes two 3x3 convolutional layers and a ReLU activation function. In the residual block, the output of the first convolutional layer is passed to the second convolutional layer after the ReLU activation function. The output of the second convolutional layer is directly added to the input to realize the learning and transmission of the residual. The code implementation of the residual block is as follows:
[0080] import torch
[0081] import torch.nn as nn
[0082] class ResidualBlock(nn.Module):
[0083] def __init__(self, in_channels, out_channels):
[0084] super(ResidualBlock, self).__init__()
[0085] self.conv1 = nn.Conv2d(in_channels, out_channels, kernel_size=3, stride=1, padding=1)
[0086] self.relu = nn.ReLU(inplace=True)
[0087] self.conv2 = nn.Conv2d(out_channels, out_channels, kernel_size=3, stride=1, padding=1)
[0088] def forward(self, x):
[0089] residual = x
[0090] out = self.conv1(x)
[0091] out = self.relu(out)
[0092] out = self.conv2(out)
[0093] out += residual
[0094] return out
[0095] The residual aggregation structure realizes the extraction and fusion of deep features by stacking multiple residual blocks. Each residual block can effectively alleviate the gradient vanishing problem in deep neural network training, allowing the network to learn and extract feature information at a deeper level. The entire model structure is as follows:
[0096] class SuperResolutionNet(nn.Module):
[0097] def __init__(self, in_channels=3, num_residual_blocks=16):
[0098] super(SuperResolutionNet, self).__init__()
[0099] self.conv1 = nn.Conv2d(in_channels, 64, kernel_size=3, stride=1, padding=1)
[0100] self.relu = nn.ReLU(inplace=True)
[0101] self.residual_blocks = nn.Sequential(
[0102] *[ResidualBlock(64, 64) for _ in range(num_residual_blocks)] )
[0104] self.conv2 = nn.Conv2d(64, in_channels, kernel_size=3, stride=1, padding=1)
[0105] def forward(self, x):
[0106] out = self.conv1(x)
[0107] out = self.relu(out)
[0108] out = self.residual_blocks(out)
[0109] out = self.conv2(out)
[0110] return out
[0111] During the training process, the mean square error (MSE) can be used as the loss function, the optimizer selects the Adam optimizer, and the learning rate is set to 0.001. The training code is as follows:
[0112] model = SuperResolutionNet()
[0113] criterion = nn.MSELoss()
[0114] optimizer = torch.optim.Adam(model.parameters(), lr=0.001)
[0115] for epoch in range(num_epochs):
[0116] for data in dataloader:
[0117] inputs, targets = data
[0118] optimizer.zero_grad()
[0119] outputs = model(inputs)
[0120] loss = criterion(outputs, targets)
[0121] loss.backward()
[0122] optimizer.step()
[0123] print("Training completed")
[0124] The multi-scale receptive field expansion module captures contextual information of different scales through convolution kernels of different scales. Use 3x3, 5x5 and 7x7 convolution kernels to perform convolution operations on the input image to obtain feature maps of different scales. This can effectively improve the performance of the model when processing multi-scale information, improve the accuracy of image reconstruction and the ability to retain details. The code is as follows:
[0125] class MultiScaleConv(nn.Module):
[0126] def __init__(self, in_channels, out_channels):
[0127] super(MultiScaleConv, self).__init__()
[0128] self.conv3x3 = nn.Conv2d(in_channels, out_channels, kernel_size=3, padding=1)
[0129] self.conv5x5 = nn.Conv2d(in_channels, out_channels, kernel_size=5, padding=2)
[0130] self.conv7x7 = nn.Conv2d(in_channels, out_channels, kernel_size=7, padding=3)
[0131] def forward(self, x):
[0132] out3x3 = self.conv3x3(x)
[0133] out5x5 = self.conv5x5(x)
[0134] out7x7 = self.conv7x7(x)
[0135] out = torch.cat([out3x3, out5x5, out7x7], dim=1)
[0136] return out
[0137] The output feature maps of the multi-scale convolution module are concatenated in the channel dimension to form a comprehensive feature map containing multi-scale information. This multi-scale feature can better adapt to objects of different sizes and shapes and improve the performance of the model in complex scenes. The following is the fusion code of the multi-scale receptive field feature:
[0138] class SuperResolutionNet(nn.Module):
[0139] def __init__(self, in_channels=3, num_residual_blocks=16):
[0140] super(SuperResolutionNet, self).__init__()
[0141] self.conv1 = nn.Conv2d(in_channels, 64, kernel_size=3, stride=1, padding=1)
[0142] self.relu = nn.ReLU(inplace=True)
[0143] self.residual_blocks = nn.Sequential(
[0144] *[ResidualBlock(64, 64) for _ in range(num_residual_blocks)] )
[0146] self.multi_scale_conv = MultiScaleConv(64, 64)
[0147] self.conv2 = nn.Conv2d(64 * 3, in_channels, kernel_size=3,stride=1, padding=1)
[0148] def forward(self, x):
[0149] out = self.conv1(x)
[0150] out = self.relu(out)
[0151] out = self.residual_blocks(out)
[0152] out = self.multi_scale_conv(out)
[0153] out = self.conv2(out)
[0154] return out
[0155] In the design of the multi-scale receptive field expansion module, the number of convolution kernels of different scales can be adjusted according to the specific application. For example, a 3x3 convolution kernel can capture local details, while a 5x5 and 7x7 convolution kernel can capture a wider range of contextual information. By properly designing the number and size of convolution kernels, the performance and robustness of the model can be improved.
[0156] The spatial-channel dual attention mechanism further improves the model's ability to focus on important features by introducing channel attention and spatial attention. Channel attention is implemented through global average pooling and fully connected layers to aggregate and weight the global information of each channel. The code implementation is as follows:
[0157] class ChannelAttention(nn.Module):
[0158] def __init__(self, in_channels, reduction=8):
[0159] super(ChannelAttention, self).__init__()
[0160] self.avg_pool = nn.AdaptiveAvgPool2d(1)
[0161] self.fc = nn.Sequential(
[0162] nn.Conv2d(in_channels, in_channels / / reduction, 1,padding=0),
[0163] nn.ReLU(inplace=True),
[0164] nn.Conv2d(in_channels / / reduction, in_channels, 1,padding=0),
[0165] nn.Sigmoid() )
[0167] def forward(self, x):
[0168] out = self.avg_pool(x)
[0169] out = self.fc(out)
[0170] return x * out
[0171] Spatial attention is implemented through convolutional layers and activation functions to aggregate and weight the local information of each pixel. This can enhance the model's attention to local important features and improve the accuracy of image reconstruction and detail retention. The code for the spatial attention mechanism is as follows:
[0172] class SpatialAttention(nn.Module):
[0173] def __init__(self, kernel_size=7):
[0174] super(SpatialAttention, self).__init__()
[0175] self.conv = nn.Conv2d(2, 1, kernel_size, padding=(kernel_size- 1) / / 2, bias=False)
[0176] self.sigmoid = nn.Sigmoid()
[0177] def forward(self, x):
[0178] avg_out = torch.mean(x, dim=1, keepdim=True)
[0179] max_out, _ = torch.max(x, dim=1, keepdim=True)
[0180] out = torch.cat([avg_out, max_out], dim=1)
[0181] out = self.conv(out)
[0182] return x * self.sigmoid(out)
[0183] Combining channel attention and spatial attention mechanisms can significantly improve the model's ability to focus on important features and enhance the accuracy of image reconstruction. The code for integrating the spatial-channel dual attention mechanism into the overall model is as follows:
[0184] class SuperResolutionNet(nn.Module):
[0185] def __init__(self, in_channels=3, num_residual_blocks=16):
[0186] super(SuperResolutionNet, self).__init__()
[0187] self.conv1 = nn.Conv2d(in_channels, 64, kernel_size=3, stride=1, padding=1)
[0188] self.relu = nn.ReLU(inplace=True)
[0189] self.residual_blocks = nn.Sequential(
[0190] *[ResidualBlock(64, 64) for _ in range(num_residual_blocks)] )
[0192] self.multi_scale_conv = MultiScaleConv(64, 64)
[0193] self.channel_attention = ChannelAttention(64)
[0194] self.spatial_attention = SpatialAttention()
[0195] self.conv2 = nn.Conv2d(64 * 3, in_channels, kernel_size=3,stride=1, padding=1)
[0196] def forward(self, x):
[0197] out = self.conv1(x)
[0198] out = self.relu(out)
[0199] out = self.residual_blocks(out)
[0200] out = self.multi_scale_conv(out)
[0201] out = self.channel_attention(out)
[0202] out = self.spatial_attention(out)
[0203] out = self.conv2(out)
[0204] return out
[0205] After the model training is completed, it is applied to the enhancement processing of low signal-to-noise ratio fluorescent labeled cell imaging images. The specific steps are as follows: Image input: Input the low signal-to-noise ratio fluorescent labeled cell imaging image into the trained deep neural network model; Image processing: Use the deep neural network model to process the input image to obtain an enhanced high signal-to-noise ratio image; Result output: Post-process the processed high signal-to-noise ratio image, including denoising, sharpening and other operations, to obtain the final high-quality image.
[0206] The code for running the model is as follows
[0207] def enhance_image(model, image):
[0208] model.eval()
[0209] with torch.no_grad():
[0210] input_tensor = torch.from_numpy(image).unsqueeze(0).unsqueeze(0).float()
[0211] enhanced_tensor = model(input_tensor)
[0212] enhanced_image = enhanced_tensor.squeeze(0).squeeze(0).numpy()
[0213] return enhanced_image
[0214] low_snr_image = cv2.imread('low_snr_image.png', cv2.IMREAD_GRAYSCALE)
[0215] enhanced_image = enhance_image(trained_model, low_snr_image)
[0216] cv2.imwrite('enhanced_image.png', enhanced_image * 255)
[0217] See also Figure 5 , which shows an experimental result diagram of an image enhancement processing method for fluorescent labeled cell imaging provided by an embodiment of the present invention. From the image result analysis, it can be seen that the changes before and after the image processing are obvious. Figure 5 (a) and (b) are images of fluorescently labeled cells before processing. It can be observed that there is a lot of noise and detail loss in the images. Figure 5 (c) and (d) are processed images. After being processed by the method of the present invention, the details of the images are significantly improved and the noise is effectively suppressed. By combining the two-layer residual aggregation structure and the receptive field expansion dual attention mechanism, the model can effectively suppress the noise in the image and improve the signal-to-noise ratio. The processed image is visually clearer and the cell edges are sharper. The introduction of the multi-scale receptive field expansion module and the space-channel dual attention mechanism enables the model to capture more detailed information. In the processed image, the cell structure and boundary details are better preserved and reproduced. After the image is processed, the overall quality is significantly improved, the cell contours are clearer, and the background noise is greatly reduced. The experimental results show that the method of the present invention has significant advantages in improving the reconstruction quality of fluorescently labeled cell imaging images.
[0218] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for enhancing the image processing of fluorescent labeled cell imaging, characterized in that: The method comprises the following steps: Acquire low signal-to-noise ratio image data for imaging fluorescently labeled cells; Processing the low signal-to-noise ratio image data using a deep neural network model, wherein the deep neural network model includes a two-layer residual aggregation structure, a multi-scale receptive field expansion module, and a space-channel dual attention mechanism; The two-layer residual aggregation structure forms a nested residual aggregation network through skip connections, hierarchically aggregating and fusing the residual information extracted from each layer of the network to reduce the loss of residual information containing image details; The multi-scale receptive field expansion module captures context information of different scales through convolution kernels of different scales; The spatial-channel dual attention mechanism enhances the discriminative learning ability of the residual network and improves the quality of the reconstructed image; The deep neural network model further includes a plurality of residual aggregation blocks, each residual aggregation block includes a plurality of residual blocks, and a nested structure is formed through skip connections; The residual block comprises the following steps: Perform convolution operation on the input image to obtain the initial residual features; The initial residual features are nonlinearly transformed through the ReLU activation function; Perform convolution operation again to obtain enhanced residual features; The enhanced residual features are skip-connected with the input image to obtain the output residual features; The multi-scale receptive field expansion module includes a plurality of convolution kernels of different scales, which are used to capture context information of different scales, and is implemented by the following steps: Perform multi-scale convolution operations on the input image to obtain feature maps of different scales; The feature maps of different scales are spliced and fused to obtain multi-scale receptive field features; The multi-scale receptive field features are input into the subsequent residual aggregation network for processing; The spatial-channel dual attention mechanism includes the following steps: Perform channel attention operation on the input feature map to obtain channel weights; Apply the channel weights to the input feature map to obtain a channel-weighted feature map; Perform spatial attention operation on the channel weighted feature map to obtain the spatial weight; Apply the spatial weights to the channel-weighted feature map to obtain the final attention-enhanced feature map; The channel attention operation includes the following steps: Perform global average pooling on the input feature map to obtain the channel description vector; The channel description vector is transformed nonlinearly through two fully connected layers to obtain the channel weight; Normalize the channel weights to ensure the stability of the weights; The spatial attention operation includes the following steps: Perform average pooling and maximum pooling on the channel weighted feature map to obtain two sets of spatial description vectors; The two sets of spatial description vectors are fused through the convolution layer to obtain the spatial weights; The spatial weights are normalized to ensure the stability of the weights.
2. The method according to claim 1, wherein: The training process of the deep neural network model includes the following steps: Use high signal-to-noise ratio fluorescent labeled cell imaging data as a supervisory signal to train low signal-to-noise ratio data; Use mean square error loss function and Adam optimizer to optimize model parameters; Avoid model overfitting through cross-validation and early stopping strategies.
3. The method according to claim 2, wherein: The training data includes labeled fluorescence microscopy image data of different organelles to ensure the diversity and coverage of the training data.
4. The method according to claim 1, wherein: The deep neural network model is further optimized to adapt to the memory limitations of the graphics processing unit, improving the efficiency of the model by reducing the number of parameters and computational complexity.
