A method for enhancing and processing microbial images

Abstract

The present invention discloses a method for enhancing and processing microbial images, belonging to the field of image processing; it is applicable to the optimization of low-light and blurred images. This method significantly improves image details and global contrast through a hybrid network architecture that combines spatial and frequency-domain features. Its main steps include: making a low-light blurred image dataset, constructing an attention module, a frequency-domain feedforward network, a downsampling encoding module, an extended spatial attention module, a feedforward network module, and an upsampling module; finally, completing image optimization output through a microbial image enhancement processing network model. The model innovatively combines spatial and frequency-domain feature extraction, convolution operations, and Fourier transform, effectively retaining the global information of the image while optimizing details, and having excellent performance especially in complex backgrounds. Compared with traditional methods, the present invention has higher robustness and efficiency in the field of microbial image enhancement, and is applicable to fields such as medical diagnosis and environmental monitoring.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and particularly relates to a method for enhancing microbial images. Background Art

[0002] The quality of microbial images is crucial in microbial research, medical diagnosis, and environmental monitoring. However, images captured by traditional electron microscopy imaging techniques under low-light conditions usually suffer from problems such as blurring and loss of details, making it difficult to meet the requirements of precise analysis and high-resolution display. Although some post-processing algorithms can enhance the image quality to a certain extent, these methods often rely on global adjustment strategies and have limited effects on restoring details, especially in complex backgrounds, it is difficult to ensure the stability of the enhancement results. Therefore, developing a microbial image enhancement method that focuses on detail optimization, can effectively retain the original image information, and improve the clarity of image details is an important direction of current research.

[0003] In recent years, deep learning techniques have made remarkable progress in the field of image processing, especially showing excellent performance in applications such as image deblurring, super-resolution, and enhancement. However, existing deep learning-based image enhancement methods are usually optimized for natural images and lack specialized designs for the low-light blurring characteristics of microbial images. When processing microbial images, these methods often suffer from insufficient extraction of detail information due to the limitations of the feature extraction module. In addition, traditional convolutional neural networks lack joint spatial and frequency domain attention modeling of microbial structural features and cannot comprehensively improve the visual quality of images.

[0004] With the development of artificial intelligence technology, hybrid network architectures that combine spatial and frequency domain feature processing have gradually received attention. Such methods can effectively make up for the deficiencies of single processing methods by enhancing local attention in the spatial domain and retaining global image information in the frequency domain. However, there is currently no dedicated network structure for microbial image enhancement, and there is also a lack of feature extraction and processing methods for low-light blurry images. Therefore, a method for enhancing microbial images that comprehensively considers spatial and frequency domain features is proposed, which can achieve efficient detail optimization and global contrast enhancement on low-light blurry images, while improving the training and detection efficiency of the model. Summary of the Invention

[0005] The present invention provides a method for enhancing microbial images, aiming to effectively solve the problem of enhancing low-light blurry microbial images through an innovative method that combines spatial and frequency domain features, and achieve efficient processing of detail optimization and global information retention.

[0006] A method for enhancing microbial images proposed by the present invention specifically includes the following steps:

[0007] S1. Production of microbial image dataset: Use an electron microscope imaging device to collect low-light and blurred microbial pictures in the application scenario of microbial image enhancement processing, and obtain a low-light and blurred microbial image dataset after size normalization;

[0008] S2. Construct the attention module MI-SAM: Introduce attention adaptive weight parameters to improve the conventional spatial channel attention module, and extract and fuse the local and global spatial features of microbial images through multi-layer convolution operations combined with spatial grouping enhancement and spatial channel attention;

[0009] S3. Construct the frequency-domain feedforward network MI-FDFN: Through the combination of fast Fourier transform and inverse fast Fourier transform, perform branch processing and recombination on the frequency module and phase information of microbial image features;

[0010] S4. Construct the downsampling encoding module DECONV: Serially cascade an attention module MI-SAM and a frequency-domain feedforward network MI-FDFN;

[0011] S5. Construct the extended attention module MI-ESAM: Adjust and enhance the receptive field through multi-branch depth convolution combined with different dilation coefficients, and fuse the spatial grouping enhancement and spatial channel attention of microbial image features;

[0012] S6. Construct the feedforward network module MIG-FFN: Through normalization processing combined with local spatial feature extraction and spatial grouping enhancement, achieve efficient multi-dimensional feature fusion of microbial image features;

[0013] S7. Construct the upsampling decoding module UDCONV: Serially cascade an extended attention module MI-ESAM and a feedforward network module MIG-FFN;

[0014] S8. Construct the microbial image enhancement processing network model MIEPNM, which sequentially includes input, lol stage, deblur stage and output;

[0015] S9. Train and detect the microbial detection model: Use the constructed low-light and blurred microbial image dataset to train the microbial image enhancement processing network model MIEPNM. After training, input the image to be enhanced into the model, and optimize the image through the enhancement processing network to output a clear and detailed enhanced image.

[0016] Preferably, in step S2, the attention module MI-SAM is used to extract spatial local features and enhance attention; First, introduce attention adaptive weight parameters to the conventional spatial channel attention module , and the spatial channel attention SCA module applicable to microbial image enhancement processing is:

[0017] ;

[0018] where is the channel attention weight, is the spatial attention weight, and the attention adaptive weight parameter takes values as:

[0019] ;

[0020] where is to calculate the global importance by taking the mean of the weights; subsequently, the attention module MI-SAM is designed, and the microbial image is input into the module. At this time , H is the feature height, W is the feature width, and C is the number of feature channels; subsequently, normalization processing is performed to obtain . At this time ; is input into to extract local spatial features. represents performing a convolution operation using a 3×3 convolution kernel. At this time, the number of channels doubles to ; subsequently, features are further extracted, and is successively input into two convolution kernels of 1×1 and 3×3 to obtain ; at this time ; subsequently, is input into the SG module for spatial grouping enhancement to obtain . At this time ; subsequently, is input into the SCA module for fusing spatial and channel attention to obtain . At this time ; subsequently, is input into a 1×1 convolution to reduce the number of channels from 2C back to C, obtaining . At this time ; finally, the input feature is added back to the output for residual connection to obtain:

[0021] ;

[0022] At this time is the final output of the attention module MI-SAM.

[0023] Preferably, the attention module MI-SAM dynamically adjusts the feature weight distribution by combining spatial and channel attention mechanisms to achieve the balanced optimization of global information and local details; by using the weighted fusion of the channel attention weight and the spatial attention weight , and the adaptive parameter Dynamic calculation. This module can accurately extract key feature regions, and further enhance the feature expression through 3×3 and 1×1 convolutions and the spatial grouping SG enhancement module. At the same time, the integrity of the input features is retained through residual connections to ensure information fluency and gradient stability. The overall design takes into account the optimization of global and local features, has strong adaptability and high efficiency, and can significantly improve the visual quality and detail clarity of low-light and blurred microbial images in complex backgrounds.

[0024] Preferably, in the frequency-domain feedforward network MI-FDFN described in step S3, the fusion of phase and frequency information is achieved through frequency-domain feature extraction; the output of the attention module MI-SAM is used as the input. First, the input features are normalized to obtain , and at this time ; Subsequently, is subjected to a fast Fourier transform to obtain the frequency module and phase information. The formula is:

[0025] ;

[0026] where is the fast Fourier transform operation, is the feature of the frequency module, is the phase information feature, and j is the imaginary unit , and at this time , ; Subsequently, and are divided into two branches. Among them, the branch directly retains the phase information feature, the branch is first input into a 1×1 convolutional kernel to extract local spatial features, and the number of channels is doubled to obtain , and at this time ; Subsequently, is input into the ReLu activation function to obtain , and at this time ; Subsequently, is input into a 1×1 convolutional kernel to reduce the number of channels from 2C back to C, obtaining , and at this time ; is the total output, which can be summarized as:

[0027] ;

[0028] Subsequently, the frequency module feature output of the branch is recombined with the phase information feature of the branch and returned to the spatial domain through the inverse Fourier transform to obtain:

[0029] ;

[0030] At this time ; Subsequently, and the initial input of the frequency-domain feedforward network MI-FDFN are subjected to residual dot product connection to obtain:

[0031] ;

[0032] At this time , and finally is subjected to residual addition operation with the initial input of the frequency-domain feedforward network MI-FDFN to obtain:

[0033] ;

[0034] At this time is the final output of the frequency-domain feedforward network MI-FDFN.

[0035] Preferably, the frequency-domain feedforward network MI-FDFN maps the input features from the spatial domain to the frequency domain through the fast Fourier transform (FFT), extracts the frequency module and phase information, and realizes the separation processing of global features and local features. Among them, the frequency module enhances the expression of local spatial features through 1×1 convolution and ReLU activation function, and then recombines with the phase information, and returns to the spatial domain through the inverse Fourier transform (IFFT). Finally, it is fused with the original input features through residual connection to output the frequency-domain enhanced features; this module combines the advantages of the frequency domain and the spatial domain, not only improves the global feature extraction ability, but also strengthens the expression of detail features, and is particularly suitable for the enhancement processing of microbial images with low-light complex backgrounds, while improving the detail clarity and global contrast of microbial images, avoiding information loss and maintaining high computational efficiency.

[0036] Preferably, the downsampling encoding module DECONV described in step S4 includes an attention module MI-SAM at the front end and a frequency-domain feedforward network MI-FDFN connected at the tail.

[0037] Preferably, the extended attention module MI-ESAM described in step S5 is used to adjust the receptive field and enhance the features; first, the output of the DECONV module is used as the input, and first normalized and preliminarily convolved to obtain:

[0038] ;

[0039] At this time, the channel features are doubled to ; Subsequently, it is fed into depth convolutions of three parallel branches combined with different dilation coefficients to adjust the receptive field range, resulting in:

[0040] ;

[0041] Among them represents a 3×3 depth convolution with a dilation coefficient of 1, represents a 3×3 depth convolution with a dilation coefficient of 4, represents a 3×3 depth convolution with a dilation coefficient of 9. At this time ; Subsequently is input into the SG module for spatial group enhancement to obtain , at this time ; Subsequently is input into the SCA module for fusing spatial and channel attention to obtain , at this time ; Subsequently is input into a 1×1 convolution to reduce the number of channels from 2C back to C, obtaining , at this time ; Finally is subjected to a residual addition operation with the initial input of the extended attention module MI-ESAM to obtain:

[0042] ;

[0043] At this time is the final output of the extended attention module MI-ESAM.

[0044] Preferably, the extended attention module MI-ESAM adjusts the receptive field range through multi-scale dilated convolutions, effectively enhancing the feature extraction ability; after first normalizing and preliminarily convolving the input features, three parallel branches are introduced, and multi-scale feature capture is realized through 3×3 depth convolutions with dilation coefficients of 1, 4, and 9 respectively. Further, the local features are optimized through the spatial group enhancement module, and the attention expression of global and local features is enhanced by combining the channel attention fusion module; finally, the module reduces the channel dimension through a 1×1 convolution and retains the original input feature information through a residual connection, and fuses and outputs it with the enhanced features; the design of this module has both global and local optimization capabilities, can effectively capture multi-scale information in microbial images with low-light complex backgrounds, significantly improve the detail clarity and feature expression ability of microbial images, while maintaining information integrity and computational efficiency.

[0045] Preferably, in the feed-forward network module MIG-FFN described in step S6, taking the output of the extended attention module MI-ESAM as the input, first normalize the input features to obtain At this time ; Subsequently is input into to extract local spatial features, and at this time the number of channels doubles to ; Subsequently is input into the SG module for spatial grouping enhancement to obtain At this time ; Subsequently is input into a 1×1 convolution to reduce the number of channels from 2C back to C to restore the same channel dimension as the input feature, obtaining At this time ; Finally is subjected to a residual addition operation with the initial input of the extended attention module MI-ESAM to obtain:

[0046] ;

[0047] At this time is the final output of the feed-forward network module MIG-FFN.

[0048] Preferably, the feed-forward network MIG-FFN enhances feature expression through normalization and local convolution operations; First, after normalizing the input features, the spatial feature extraction ability is refined through the spatial grouping enhancement module, and then the number of channels is reduced from 2C back to C using a 1×1 convolution to be consistent with the input feature channels, thus efficiently fusing global and local features; Finally, the module adds the initial input features and the enhanced features through a residual connection, which not only preserves the original information but also strengthens the feature expression ability; The module design aims to improve the detailed expression and global consistency of features, is suitable for microbial image enhancement processing tasks in complex scenarios, and can significantly enhance the model's ability to capture key features while maintaining computational efficiency.

[0049] Preferably, the upsampling decoding module UDCONV described in step S7 includes an extended attention module MI-ESAM at the front end and a feed-forward network module MIG-FFN at the tail end.

[0050] Preferably, the microbial image enhancement processing network model MIEPNM described in step S8 is divided into an upsampling stage and a downsampling stage.

[0051] Preferably, in the microbial image enhancement processing network model MIEPNM described in step S8, in the downsampling stage, the low-light and blurred input image is first input into a 3×3 convolution for preliminary feature extraction to obtain At this time ; Subsequently The input is obtained in the above-mentioned downsampling encoding module DECONV with one layer , at this time , and perform downsampling and 2-layer DECONV depth convolution on , to obtain:

[0052] ;

[0053] Among them is a 2×2 convolution, with a stride of 2 set to reduce the feature space resolution to half of the original and double the number of channels. At this time , similarly perform downsampling and 3-layer DECONV depth convolution on , to obtain:

[0054] ;

[0055] At this time ; Subsequently, perform downsampling and 2-layer DECONV depth convolution on , to obtain:

[0056] ;

[0057] At this time , the downsampling stage ends.

[0058] Preferably, in the microbial image enhancement processing network model MIEPNM described in step S8, in the upsampling stage, first input into 2-layer upsampling decoding module UDCONV and perform upsampling operation and perform residual connection with , to obtain:

[0059] ;

[0060] Among them is a 1×1 convolution layer to reduce the feature channels to half, and then concatenate 1 layer of pixel rearrangement operation to double the spatial resolution width and height. At this time ; Subsequently, input into 3-layer upsampling decoding module UDCONV and perform upsampling operation and perform residual connection with , to obtain:

[0061] ;

[0062] At this time ; Subsequently, continue to input into 2-layer upsampling decoding module UDCONV and perform upsampling operation and perform residual connection with , to obtain:

[0063] ;

[0064] At this time ; Finally, Input into the 1-layer upsampling decoding module UDCONV and the 1-layer 1×1 ordinary convolution to obtain:

[0065] ;

[0066] At this time That is the final output image of the microbial image enhancement processing network model MIEPNM.

[0067] Preferably, in the upsampling and downsampling stages of the microbial image enhancement processing network model MIEPNM, through the depth convolution module and specific upsampling and downsampling operations, multi-scale feature extraction and reconstruction are effectively achieved; in the downsampling stage, after the input features are initially extracted by 3×3 convolution, the spatial resolution is gradually reduced and the channel dimension is increased by using multiple DECONV modules and downsampling operations, so as to capture global features and retain key information; while in the upsampling stage, the spatial resolution of the input features is gradually restored through multiple UDCONV modules and upsampling operations, and the features in the downsampling stage are integrated into the reconstruction process by combining residual connections to ensure the integrity of the information flow; in the final output stage, the features are further refined by 1×1 convolution; the overall design takes into account both the extraction of global features and the restoration of local details, and can significantly improve the clarity and enhancement effect of microbial images under complex backgrounds and low-light conditions, while maintaining the efficiency and robustness of the model.

[0068] Preferably, in the training and detection of the microbial detection model described in step S9, that is, using the microbial low-light blurred image dataset constructed in step S1, the microbial image enhancement processing network model MIEPNM is trained. After training, the image to be enhanced is input into the model, and the image is optimized by the enhancement processing network to output a clear and detail-prominent enhanced image.

[0069] Compared with the prior art, for the problem of low-light blur of microbial images, a new method combining spatial and frequency domain feature processing is proposed; by introducing innovative structures such as the attention module (MI-SAM), the frequency domain feedforward network (MI-FDFN), and the extended attention module (MI-ESAM), the effect of image enhancement processing is significantly improved; compared with the traditional enhancement method based on global adjustment, the present technology can restore details more accurately while retaining the global information of the image, especially showing higher robustness under low-light complex backgrounds. Description of the Drawings

[0070] Figure 1 It is the overall flowchart of a microbial image enhancement processing method.

[0071] Figure 2 It is a schematic diagram of the structure of the downsampling encoding module DECONV.

[0072] Figure 3 It is a schematic diagram of the structure of the upsampling decoding module UDCONV.

[0073] Figure 4 It is the network structure diagram of the microbial image enhancement processing network model MIEPNM.

[0074] Figure 5 It is a schematic diagram for comparing the enhancement processing of microbial low-light blurred images. Specific implementation manners

[0075] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0076] Please refer to the attached Figure 1 - attached Figure 5 , and the present invention provides a method for enhancing the processing of microbial images.

[0077] A method for enhancing the processing of microbial images proposed by the present invention, as shown in the overall flowchart of the attached Figure 1 , specifically includes the following steps:

[0078] S1. Production of a microbial image data set. Use an electron microscope imaging device to collect microbial low-light blurred pictures in the application scenario of microbial image enhancement processing, and obtain a microbial low-light blurred image data set after size normalization;

[0079] S2. Construct an attention module MI-SAM, introduce attention adaptive weight parameters to improve the conventional spatial channel attention module, and extract and fuse the local and global spatial features of microbial images through multi-layer convolution operations combined with spatial grouping enhancement and spatial channel attention;

[0080] S3. Construct a frequency domain feedforward network MI-FDFN, and perform branch processing and recombination on the frequency module and phase information of microbial image features by combining the fast Fourier transform and the inverse fast Fourier transform;

[0081] S4. Construct a downsampling encoding module DECONV, and serially cascade an attention module MI-SAM and a frequency domain feedforward network MI-FDFN;

[0082] S5. Construct an extended attention module MI-ESAM to adjust and enhance the receptive field through multi-branch depth convolution combined with different dilation coefficients, and fuse the spatial grouping enhancement and spatial channel attention of microbial image features;

[0083] S6. Construct a feed-forward network module MIG-FFN to achieve efficient multi-dimensional feature fusion of microbial image features through normalization processing combined with local spatial feature extraction and spatial grouping enhancement;

[0084] S7. Construct an upsampling decoding module UDCONV, serially concatenating an extended attention module MI-ESAM and a feed-forward network module MIG-FFN;

[0085] S8. Construct a microbial image enhancement processing network model MIEPNM, which sequentially includes an input, a lol stage, a deblur stage, and an output;

[0086] S9. Train and detect the microbial detection model. Utilize the constructed microbial low-light blurred image dataset to train the microbial image enhancement processing network model MIEPNM. After training, input the image to be enhanced into the model, and optimize the image through the enhancement processing network to output a clear and detail-rich enhanced image.

[0087] Furthermore, in the production of the microbial image dataset described in step S1, after collecting low-light blurred microbial images using an electron microscope, use the Resize() and ToTensor() methods in torchvision.transforms of PyTorch to perform size normalization and tensorization processing on the images to obtain the final dataset.

[0088] Furthermore, as shown in the left module structure diagram of the appendix Figure 2 In this embodiment, in the attention module MI-SAM described in step S2, a custom module containing 3×3 and 1×1 convolutions is constructed using PyTorch and combined with the channel attention mechanism. The input initial microbial image feature map has a feature size of 256 in height, 256 in width, and 64 in channels. First, introduce microbial image adaptive parameters to the conventional spatial channel attention module to obtain a spatial channel attention SCA module suitable for microbial image enhancement processing:

[0089] ;

[0090] where is the channel attention weight, is the spatial attention weight, and the microbial image adaptive parameter takes values as:

[0091] ;

[0092] Among them is to calculate the global importance by averaging the weights. In this embodiment, the adaptive weight parameter is initially set to 0.5; Subsequently, the attention module MI-SAM is designed, and the microbial image is input into the module. At this time , where H is the feature height, W is the feature width, and C is the feature channel number; Subsequently, normalization processing is performed to obtain . At this time ; is input into to extract local spatial features. represents performing a convolution operation using a 3×3 convolution kernel. At this time, the number of channels is doubled to ; Subsequently, features are further extracted, and is sequentially input into two convolution kernels of 1×1 and 3×3 to obtain ; At this time ; Subsequently, is input into the SG module for spatial grouping enhancement to obtain . At this time ; Subsequently, is input into the SCA module for fusing spatial and channel attention to obtain . At this time ; Subsequently, is input into a 1×1 convolution to reduce the number of channels from 2C back to C, obtaining . At this time ; Finally, the input feature is added back to the output for residual connection to obtain:

[0093] ;

[0094] At this time is the final output of the attention module MI-SAM.

[0095] Furthermore, as shown in the right module structure diagram of Appendix Figure 2 , in this embodiment, in the frequency-domain feedforward network MI-FDFN described in step S3, the torch.fft of PyTorch is used for FFT and IFFT operations. The initial feature size of the input microbial image is H = 256, W = 256, C = 64, and a custom convolution layer is used to process the frequency-domain features; First, the output microbial image feature map of the inter-attention module MI-SAM is used as the input, and normalization processing is performed to obtain , at this time ; Subsequently, is subjected to a fast Fourier transform to obtain a frequency module and phase information. The formula is:

[0096] ;

[0097] Where is the fast Fourier transform operation, is the frequency module feature, is the phase information feature, and j is the imaginary unit , at this time , ; Subsequently, and are divided into two branches. Among them, branch directly retains the phase information feature, branch is first input into a 1×1 convolutional kernel to extract local spatial features, and the number of channels is doubled to obtain , at this time ; Subsequently, is input into the ReLu activation function to obtain , at this time ; Subsequently, is input into a 1×1 convolutional kernel to reduce the number of channels from 2C back to C, obtaining , at this time ; is 's total output, which can be summarized as:

[0098] ;

[0099] Subsequently, branch's frequency module feature output and branch's phase information feature are recombined and returned to the spatial domain through an inverse Fourier transform to obtain , at this time ; Subsequently, and the initial input of the frequency-domain feedforward network MI-FDFN are subjected to a residual dot product connection to obtain , at this time , and finally is subjected to a residual addition operation with the initial input of the frequency-domain feedforward network MI-FDFN to obtain , at this time is the final output of the frequency-domain feedforward network MI-FDFN.

[0100] Further, as shown in the appendix Figure 2As shown in the structural diagram, in this embodiment, the downsampling encoding module DECONV described in step S4 includes an attention module MI-SAM at the front end and a frequency domain feedforward network MI-FDFN connected at the tail.

[0101] Further, as shown in the left module structure diagram of the appendix Figure 3 In this embodiment, the extended attention module MI-ESAM described in step S5 uses torch.nn.Conv2d in PyTorch to set different dilation rates to achieve multi-scale feature extraction; the output microbial image feature map of the DECONV module is used as the input, with the initial feature size of H = 256, W = 256, and C = 64; first, normalization and preliminary convolution operations are performed to obtain:

[0102] ;

[0103] At this time, the channel features are doubled to ; then it is passed into the depth convolution of three parallel branches combined with different dilation coefficients to adjust the receptive field range to obtain:

[0104] ;

[0105] Among them represents a 3×3 depth convolution with a dilation coefficient of 1, represents a 3×3 depth convolution with a dilation coefficient of 4, represents a 3×3 depth convolution with a dilation coefficient of 9. At this time ; then is input into the SG module for spatial group enhancement to obtain , at this time ; then is input into the SCA module for fusing spatial and channel attention to obtain , at this time ; then is input into a 1×1 convolution to reduce the number of channels from 2C back to C to obtain , at this time ; finally is subjected to a residual addition operation with the initial input of the extended attention module MI-ESAM to obtain , at this time is the final output of the extended attention module MI-ESAM.

[0106] Further, as shown in the appendix Figure 3As shown in the right module structure diagram, in this embodiment, the feed-forward network module MIG-FFN described in step S6 is implemented based on the normalization and convolution operations of PyTorch, and the output microbial image feature map of the extended attention module MI-ESAM is used as the input, and the initial feature size of the input is H = 256, W = 256, C = 64; first, the input feature is normalized to obtain , and at this time ; then is input into to extract local spatial features, and at this time the number of channels is doubled to ; then is input into the SG module for spatial group enhancement to obtain , and at this time ; then is input into a 1×1 convolution to reduce the number of channels from 2C back to C to restore the same channel dimension as the input feature, obtaining , and at this time ; finally is subjected to a residual addition operation with the initial input of the extended attention module MI-ESAM to obtain:

[0107] ;

[0108] At this time is the final output of the feed-forward network module MIG-FFN.

[0109] Furthermore, as shown in the structure diagram of Appendix Figure 3 , in this embodiment, the upsampling decoding module UDCONV described in step S7 includes an extended attention module MI-ESAM at the front end and a feed-forward network module MIG-FFN at the tail end.

[0110] Furthermore, as shown in the structure diagram of Appendix Figure 4 , in this embodiment, the microbial image enhancement processing network model MIEPNM described in step S8 uses torch.nn.Sequential of PyTorch to combine the above modules into a complete network and defines the forward propagation logic; the model is specifically divided into an upsampling stage and a downsampling stage.

[0111] Furthermore, as shown in the left process structure diagram of Appendix Figure 4 , in this embodiment, the microbial image enhancement processing network model MIEPNM described in step S8, where the input of the downsampling stage is a low-light blurred microbial image , and the initial feature size of the input is H = 256, W = 256, C = 64; first Input into a 3×3 convolution for preliminary feature extraction to obtain , at this time ; Subsequently, is input into one layer of the above downsampling encoding module DECONV to obtain , at this time that is, H = 256, W = 256, C = 64. Then, is subjected to downsampling and 2-layer DECONV depth convolution to obtain:

[0112] ;

[0113] Among them, is a 2×2 convolution with a stride of 2, which reduces the feature space resolution to half of the original and doubles the number of channels. At this time that is, H = 128, W = 128, C = 128. Similarly, continues to be subjected to downsampling and 3-layer DECONV depth convolution to obtain:

[0114] ;

[0115] At this time that is, H = 64, W = 64, C = 256; Subsequently, continues to be subjected to downsampling and 2-layer DECONV depth convolution to obtain:

[0116] ;

[0117] At this time that is, H = 32, W = 32, C = 512, and the downsampling stage ends.

[0118] Further, as shown in the right process structure diagram of the appendix Figure 4 , in this embodiment, in the microbial image enhancement processing network model MIEPNM described in step S8, the initial input in the upsampling stage is the output microbial image feature map of the downsampling stage , with an initial feature size of H = 32, W = 32, C = 512; First, is input into 2 layers of the upsampling decoding module UDCONV and the upsampling operation and is subjected to residual connection to obtain:

[0119] ;

[0120] Among them, is a 1×1 convolution to reduce the feature channels to half, and then a 1-layer pixel rearrangement operation is concatenated to double the spatial resolution width and height. At this time that is, H = 64, W = 64, C = 256; Subsequently, The input 3-layer upsampling decoding module UDCONV and the upsampling operation are combined with to perform a residual connection, resulting in:

[0121] ;

[0122] At this time i.e., H = 128, W = 128, C = 128; Subsequently, continue to combine the input 2-layer upsampling decoding module UDCONV and the upsampling operation with to perform a residual connection, resulting in:

[0123] ;

[0124] At this time i.e., H = 256, W = 256, C = 64; Finally, input into the 1-layer upsampling decoding module UDCONV and a 1×1 ordinary convolution layer to obtain:

[0125] ;

[0126] At this time the feature size is H = 256, W = 256, C = 64, which is the final output image of the microbial image enhancement processing network model MIEPNM.

[0127] Furthermore, in this embodiment, in the training and detection of the microbial detection model in step S9, that is, using the microbial low-light blurred image dataset constructed in step S1, the microbial image enhancement processing network model MIEPNM is trained. The training is carried out under the python3.10.12 version, using the PyTorch2.5.1 framework and the Adam optimizer for 1000 iterations of training. Since the video memory size is 12GB, the batch size is set to 32, the initial learning rate is 0.004, and the learning rate is dynamically adjusted during the training process; after the training data is completed, the best enhancement model file is obtained. The image to be enhanced is input into the enhancement model, and the image is optimized through the enhancement processing network to output a clear and detailed enhanced image; the schematic diagram of the result of the microbial image enhancement processing is shown in the appendix Figure 5 as shown, where part a is the input original image and part b is the output image after the enhancement processing.

Claims

1. A microbial image enhancement processing method, characterized in that: The following steps are involved: S1. Preparation of microbial image dataset: using electron microscope imaging equipment to collect microbial low-light blurry images in the microbial image enhancement processing application scenario, and obtaining the microbial low-light blurry image dataset after size normalization; S2. Construct an attention module MI-SAM, introduce an attention adaptive weight parameter to improve the conventional spatial channel attention module, and extract and fuse the local and global spatial features of microbial images by combining spatial grouping enhancement and spatial channel attention through multi-layer convolution operations; S3, constructing a frequency domain feedforward network MI-FDFN, which branches and reorganizes the frequency modules and phase information of microbial image features by combining fast Fourier transform and inverse fast Fourier transform; S4, construct a downsampling encoding module DECONV, serially cascade an attention module MI-SAM and a frequency domain feedforward network MI-FDFN; S5. Construct an extended attention module MI-ESAM, which adjusts and enhances the receptive field through multi-branch deep convolution combined with different expansion coefficients, and integrates the spatial grouping enhancement and spatial channel attention of microbial image features; S6. Construct the feedforward network module MIG-FFN, and achieve efficient fusion of multi-dimensional features of microbial image features through standardization processing combined with local spatial feature extraction and spatial grouping enhancement; S7, construct an upsampling decoding module UDCONV, serially cascade an extended attention module MI-ESAM and a feedforward network module MIG-FFN; S8, constructing a microbial image enhancement processing network model MIEPNM, which includes an input, a downsampling stage implemented by a multi-layer DECONV module and a downsampling operation, an upsampling stage implemented by a multi-layer UDCONV module and an upsampling operation, and an output; S9. Train the microbial detection model and conduct tests. Use the constructed microbial low-light blurry image dataset to train the microbial image enhancement processing network model MIEPNM. After the training is completed, input the image to be enhanced into the model, optimize the image through the enhancement processing network, and output a clear enhanced image with prominent details.

2. A microorganism image enhancement processing method according to claim 1, characterized in that: The attention module MI-SAM described in step S2; first, the attention adaptive weight parameter λ is introduced into the conventional spatial channel attention module, and the spatial channel attention SCA module suitable for microbial image enhancement processing is obtained as follows: F SCA =SCA(F SG )=λ·(WSA⊙F)+(1-λ)·(WCA⊙F); Where W CA is the channel attention weight, W SA is the spatial attention weight, and the attention adaptive weight parameter λ is taken as: Where Mean(W) is the mean operation of the weights to calculate the global importance; then the attention module MI-SAM is designed to convert the microbial image I MI In the input module, I MI =∈R H×W×C , H is the feature height, W is the feature width, and C is the number of feature channels; Then normalization is performed to obtain F0 = LNorm (I MI ), then F0∈R H×W×C ; Input F0 to F1=CONV3(F0) to extract local spatial features. CONC3(*) indicates that a 3×3 convolution kernel is used for convolution operation. At this time, the number of channels is doubled to F1∈R H×W×2C ; Then further extract features and input F1 into the two convolution kernels of 1×1 and 3×3 in turn to obtain F2=CONV3(CONV1(F1)); at this time F2∈R H×W×2C ; Then F2 is input into the SG module for spatial grouping enhancement to obtain F SG =SG(F2), at this time F SG ∈R H×W×2C ; Then F SG Input the SCA module to fuse spatial and channel attention to obtain F SCA =SCA(F SG ), at this time F SCA ∈R H×W×2C ; Then F SCA Input to a 1×1 convolution to reduce the number of channels from 2C back to C, and get F3 = CONV1 (F SCA ), then F3∈R H×W×C ; Finally, input feature I MI Add back the output F3 for residual connection to get F MI-SAM =F3+I MI , at this time F MI-SAM ∈R H×W×C It is the final output of the attention module MI-SAM.

3. A microorganism image enhancement processing method according to claim 1, characterized in that: The frequency domain feedforward network MI-FDFN described in step S3; the output F of the attention module MI-SAM MI-SAM ∈R H×W×C As input, the input features are first normalized to obtain F0 = LNorm (F MI-SAM ), then F0∈R H×W×C ; Then, F0 is subjected to fast Fourier transform to obtain frequency module and phase information, the formula is: FFT(F0)=Module(F0)+j·Phase(F0); Where FFT(*) is the fast Fourier transform operation, Module(F0) is the frequency module feature, Phase(F0) is the phase information feature, and j is the complex unit j 2 =-1, then Module(F0)∈R H×W×C ,Phase(F0)∈R H×W×C ; Then, Module(F0) and Phase(F0) are divided into two branches, where the Phase(F0) branch directly retains the phase information features, and the Module(F0) branch is first input into a 1×1 convolution kernel to extract local spatial features, and the number of channels is doubled to obtain F M1 =CONV1(Module(F0)), at this time F M1 ∈R H×W×2C ; Then F M1 Input to the ReLu activation function to get F M2 =ReLu(F M1 ), at this time F M2 ∈R H ×W×2C ; Then F M2 Input to a 1×1 convolution kernel to reduce the number of channels from 2C back to C, and get F M3 =CONV1(F M2 ), at this time F M3 ∈R H×W×C ; F M3 is the total output of Module (F0), which can be summarized as: F M3 =CONV1(ReLu(CONV1(Module(F0)))); Then the frequency module feature output F of the Module (F0) branch is M3 Recombining with the phase information characteristics of the Phase (F0) branch and returning to the spatial domain through inverse Fourier transform, we get: At this time F IFFT ∈R H×W×C ; Then F IFFT and the initial input F of the frequency domain feedforward network MI-FDFN MI-SAM Perform residual multiplication and connection to obtain: F sk =F IFFT ·F MI-SAM ; At this time F sk ∈R H×W×C Finally, F sk The initial input F of the frequency domain feedforward network MI-FDFN MI-SAM Perform the residual addition operation to obtain: F MI-FDFN =F sk +F MI-SAM At this time F MI-FDFN ∈R H×W×C It is the final output of the frequency domain feedforward network MI-FDFN.

4. The microorganism image enhancement processing method according to claim 1, characterized in that: The extended attention module MI-ESAM described in step S5 converts the output F of the DECONV module MI-FDFN ∈R H×W×C As input, it is first normalized and subjected to a preliminary convolution operation: F E1 =CONV3(CONV1(LNorm(F MI-FDFN ))); At this time, the channel characteristic doubles to F E1 ∈R H×W×2C ; Then the deep convolution of the three parallel branches is combined with different expansion coefficients to adjust the receptive field range to obtain: F E2 =DW1CONV3(F E1 )+DW4CONV3(F E1 )+DW9CONV3(F E1 ); Among them, DW1CONV3() represents a 3×3 depth convolution with a dilation factor of 1, DW4CONV3() represents a 3×3 depth convolution with a dilation factor of 4, and DW9CONV3() represents a 3×3 depth convolution with a dilation factor of 9. At this time, F E2 ∈R H×W×2C ; Then F E2 Input to the SG module for spatial grouping enhancement to obtain F ESG =SG(F E2 ), at this time F ESG ∈R H×W×2C ; Then F ESG Input the SCA module to fuse spatial and channel attention to obtain F ESCA =SCA(F ESG ), at this time F ESCA ∈R H×W×2C ; Then F ESCA Input to a 1×1 convolution to reduce the number of channels from 2C back to C, and get F E3 =CONV1(F ESCA ), at this time F E3 ∈R H×W×C ; Finally, F E3 With the initial input F of the extended attention module MI-ESAM MI-FDFN ∈R H×W×C Perform the residual addition operation to obtain: F MI-ESAM =F E3 +F MI-FDFN ; At this time F MI-ESAM ∈R H×W×C It is the final output of the extended attention module MI-ESAM.

5. The microorganism image enhancement processing method according to claim 1, characterized in that: The feedforward network module MIG-FFN described in step S6 first converts the output F of the extended attention module MI-ESAM into MI-ESAM ∈R H×W×C As input, the input features are first normalized to obtain F E4 =LNorm(F MI-ESAM ), at this time F E4 ∈R H×W×C ; Then F E4 Pass input to F E5 =CONV1(F E4 ) extracts local spatial features, and the number of channels is doubled to F E5 ∈R H×W×2C ; Then F E5 Input to the SG module for spatial grouping enhancement to obtain F ESG2 =SG(F E5 ), at this time F ESG2 ∈R H×W×2C ; Then F ESG2 Input to a 1×1 convolution to reduce the number of channels from 2C back to C to restore the same channel dimension as the input feature, and get F E6 =CONV1(F ESG2 ), at this time F E6 ∈R H×W×C ; Finally, F E6 With the initial input F of the extended attention module MI-ESAM MI-ESAM ∈R H×W×C Perform the residual addition operation to obtain: F MIG-FFN =F E6 +F MI-ESAM ; At this time F MIG-FFN ∈R H×W×C It is the final output of the feedforward network module MIG-FFN.

6. The microorganism image enhancement processing method according to claim 1, characterized in that: The microbial image enhancement processing network model MIEPNM described in step S8 is divided into an upsampling stage and a downsampling stage, wherein the downsampling stage first converts the low-light blurred input image I IN Input to a 3×3 convolution for preliminary feature extraction to obtain F D0 =CONV3(I IN ), at this time F D0 =∈R H×W×C ; Then F D0 Input a layer of the above downsampling encoding module DECONV to get F D1 =DECONV(F D0 ), at this time F D1 ∈R H×W×C , F D1 Perform downsampling and 2 layers of DECONV depth convolution to obtain: F D2 =DECONV(DECONV(DownSampling(F D1 ))); DownSampling() is a 2×2 convolution with a stride of 2, which reduces the feature space resolution to half of the original and doubles the number of channels. Similarly, F D2 Continue with downsampling and 3 layers of DECONV depth convolution to get: F D3 =DECONV 3 (DownSampling(F D2 )); at this time Then F D3 Continue downsampling and 2 layers of DECONV depth convolution to get: F D4 =DECONV(DECONV(DownSampling(F D3 ))); at this time The downsampling phase ends; The upsampling stage first converts F D4 Input 2 layers of upsampling decoding module UDCONV and upsampling operation and F D3 Perform residual connection and get: F U3 =UpSampling(UDCONV(UDCONV(F U4 )))+F D3 ; UpSampling() is a 1×1 convolution to reduce the dimension of the feature channel to half, and then a layer of pixel rearrangement operation is connected in series to double the width and height of the spatial resolution path. Then F U3 Input 3 layers of upsampling decoding module UDCONV and upsampling operation and F D2 Perform residual connection and get: F U2 =UpSampling(UDCONV 3 (F U3 ))+F D2 ; at this time Then continue to F U2 Input 2 layers of upsampling decoding module UDCONV and upsampling operation and F D1 Perform residual connection and get: F U1 =UpSampling(UDCONV(UDCONV(F U2 )))+F D1 ; At this time F U1 =∈R H×W×C ; Finally, F U1 Input to the 1-layer upsampling decoding module UDCONV and 1-layer 1×1 ordinary convolution to get: I out =CONV1(UDCONV(F U1 )); At this time I out This is the final output image of the microbial image enhancement processing network model MIEPNM.

Images