Single bacterium image denoising technology for raman optical tweezers bacterial detection

Abstract

The application belongs to the technical field of image processing, and discloses a single bacterium image denoising technology for Raman optical tweezers bacterial detection. The single bacterium image denoising method simultaneously utilizes the time and space information of noise to solve the denoising problem of the single bacterium image, and realizes the denoising of the single bacterium microscopic image sequence through a pre-trained STBasicNet network. The network is composed of four modules, and the features are summarized according to the position, structural component information in the single image and the time correlation between the images. Compared with the conventional bacterial image denoising method, the method proposed by the application can adapt to different types of noise and image structures, is more flexible and has universality in actual application, so that the laser tweezers can more accurately grasp the target bacteria, improves the success rate of grasping, and brings new possibilities for the development of the field of bacterial image denoising.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and in particular relates to a single-bacterial image denoising technique for Raman optical tweezers bacterial detection. Background Technology

[0002] Raman optical tweezers bacterial detection technology is an analytical technique that combines Raman spectroscopy, microscopic imaging, and laser tweezers technology, and it has broad application prospects in the field of bacterial detection. Raman spectroscopy can rapidly obtain comprehensive information on the chemical composition of bacteria, enabling rapid identification and analysis of bacterial species, quantities, and states. By combining laser tweezers technology with microscopic imaging for sample manipulation and positioning, precise control of minute samples is achieved, allowing for the acquisition and analysis of Raman spectra at specific locations. Utilizing the advantages of both laser tweezers and Raman spectroscopy, efficient analysis and quantitative measurement of minute samples can be realized. In the field of bacterial detection, it can be used to rapidly analyze the chemical composition and structural information of samples, exhibiting good reproducibility and high resolution across different cell types, and can simultaneously handle the detection of bacterial composition in complex environments.

[0003] In Raman optical tweezers bacterial detection technology, noise in the microscopic image of bacteria has a significant impact on the grasping ability of laser tweezers. Noise is a common problem in microscopic images, originating from various sources such as microscope lens defocusing, light source noise, and electronic noise. These factors can all cause the bacterial image to become blurry, distorted, or jittery. If the noise in the microscopic image is not properly processed, errors or failures may occur when using laser tweezers for grasping, because the bacterial outlines and positions in the microscopic image may not be clear enough, leading to the laser tweezers grasping the wrong bacteria or failing to grasp the target bacteria altogether. Conversely, if the processed microscopic image has less noise, the success rate of grasping with laser tweezers will be higher. In high-quality microscopic images, the bacterial outlines and positions are more clearly defined, allowing the laser tweezers to grasp the target bacteria more accurately, thus improving the success rate of grasping.

[0004] Therefore, removing noise and interference from bacterial microscopic images is a key issue. To address these problems, this patent proposes for the first time a single-bacterial image denoising technique for Raman optical tweezers bacterial detection. By optimizing the single-bacterial image in both temporal and spatial dimensions, and utilizing the temporal and spatial information of noise to solve the denoising problem, the technique further improves the generalization ability and efficiency of single-bacterial image denoising, and the denoised single-bacterial image has reliable quality. Summary of the Invention

[0005] The technical solution adopted by the embodiments of the present invention to solve the technical problem is: a single-bacterial image denoising technique for Raman optical tweezers bacterial detection, mainly including the following steps:

[0006] By training the Space and Time Basic Net (STBasicNet) network, the network adaptively learns. The alignment module in the network adopts a coarse-to-fine processing method, which propagates the learned offsets from a lower level to a higher level, thereby gradually improving the offset accuracy and achieving the alignment of targets in the image sequence. The information extraction module uses adjacent image sequences to extract key features to reconstruct the target image. The fusion module aggregates the structural, positional, and temporal information of the sample to improve the feature representation. The reconstruction module uses the fused information to reduce noise and generate a denoised image of the sample. The trained network can be used to denoise single bacterial images of the sample. The input is a sequence of single bacterial microscopic images obtained by continuous imaging of bacteria using a Raman optical tweezers bacterial detection system (the image sequence contains the sample's feature information and temporal correlation). The trained STBasicNet network then uses the temporal and spatial information of the image sequence to denoise the image. Through multi-path learning of image information, the network makes full use of the positional, spatial, and temporal correlations in the image sequence. The network outputs a sequence of 5 denoised and clear images, which greatly improves the image intensity and clarity. This effectively removes interfering components from the specific details in the bacterial images, allowing the laser tweezers to grasp the target bacteria more accurately, thereby increasing the success rate of grasping. Subsequently, Raman spectra of the bacteria can be collected and analyzed at specific locations.

[0007] The present invention provides a single-bacterial image denoising technique for Raman optical tweezers bacterial detection, which has the following advantages:

[0008] 1. This invention utilizes deep learning to develop and refine a novel image denoising network structure, STBasicNet, for microscopic image sequences of single bacteria samples. This supervised learning network consists of four modules: alignment, information extraction, fusion, and reconstruction. It can not only extract pixel-level feature points from single-frame images but also fully utilize the temporal redundancy between features to recover sample characteristics. This significantly improves the efficiency of single-bacterial image denoising and provides a feasible solution for more accurately grasping target bacteria with laser tweezers.

[0009] 2. This invention expands upon traditional bacterial image denoising methods. Unlike traditional methods, which require refined processing for different types of bacterial images, traditional methods rely heavily on manual experience in adjusting denoising parameters and selecting filter window sizes, resulting in significant time consumption. This invention utilizes the STBasicNet network, built based on multi-path learning, to learn features from data through network training. This avoids the hassle of manually designing algorithms and parameters, allowing for better adaptation to different types of noise and image structures. Although the training process is time-consuming, it is unnecessary in practical applications, greatly improving denoising efficiency.

[0010] 3. The single-bacterial image denoising technique for Raman optical tweezers bacterial detection provided in this invention has a wider application scope compared to other deep learning methods because it can utilize the temporal redundancy information in the microscopic image sequence and has strong generalization ability. The STBasicNet network structure proposed in this invention adopts a multi-path learning approach to learn the positional, spatial, and temporal features from single-bacterial microscopic image sequences, which can adapt to different types of noise and image structures, while also effectively reducing the risk of overfitting. This improved generalization ability makes the technique more flexible and universal in practical applications, bringing new possibilities to the development of the field of bacterial image denoising. Attached Figure Description

[0011] Figure 1 This is a flowchart of the STBasicNet network training process provided by the present invention.

[0012] Figure 2 This is a flowchart of single-bacterial image denoising in Raman optical tweezers bacterial detection provided by the present invention.

[0013] Figure 3 This is a denoising effect image of a single bacterium in Raman optical tweezers bacterial detection provided by the present invention. Among them, (a) is an unprocessed noisy microscopic image sequence of *Toxoplasma hygrophila* obtained by the microscopic imaging system for Raman optical tweezers bacterial detection, and (b) is a denoising effect image sequence obtained after processing by the method described in this patent. Detailed Implementation

[0014] The technical solution of the present invention will now be further described in conjunction with the embodiments and accompanying drawings.

[0015] In Raman optical tweezers bacterial detection, noise in the microscopic images of bacteria significantly affects the grasping ability of laser tweezers. Bacterial image denoising techniques require refined processing for different types of noise signals, and the denoising effect depends on manually designed algorithms and parameters, with limited adaptability to different types of noise and image structures. This invention proposes a single-bacterial image denoising technique for Raman optical tweezers bacterial detection. Based on a deep learning neural network, it simultaneously acquires the location information, spatial structure information, and temporal correlation of the sample by inputting a single-bacterial microscopic image sequence and using multi-path learning. A detailed description is provided below with reference to the accompanying drawings. This mainly describes the implementation method of a single-bacterial image denoising technique in Raman optical tweezers bacterial detection.

[0016] The target of this invention is a sequence of noisy single-bacterial microscopic images. Figure 3 (a) The image is an unprocessed, noisy microscopic image sequence of *Toxoplasma psychrophilum* obtained by a microscopic imaging system for Raman optical tweezers bacterial detection. By employing a single-bacterial image denoising technique for Raman optical tweezers bacterial detection as described in this patent, such as… Figure 3 As shown in (b), noise and interference in the image sequence are well removed. The denoised image can enable the laser tweezers to more accurately grasp the target bacteria, thereby improving the success rate of grasping. Subsequently, Raman spectra of the bacteria can be collected and analyzed at specific locations.

[0017] Now combined Figure 1 , Figure 2 , Figure 3 This invention provides a detailed description of a single-bacterial image denoising technique for Raman optical tweezers bacterial detection:

[0018] 1. The core of this invention is the STBasicNet network structure. The model is trained using a supervised learning method, and the training process is as follows: Figure 1As shown. Training is for learning the parameter settings of the denoising network, meaning it needs to be completed before use, but this part is not required when officially used for denoising image sequences. First, single-bacterial microscopic image sequences and corresponding denoised bacterial image sequences are created. Each sequence contains 5 images. This part uses single-bacterial images with different types of noise obtained from the microscopic imaging system in Raman optical tweezers bacterial detection of the test samples *Toxoplasma hygrophila* and *Toxoplasma brucellosis*. The microscopic magnification is 2000x, and the bacterial image resolution is 197*140. The image acquisition method is timed continuous acquisition. A ADLINK CameraLink acquisition card is used, with an operating voltage of 24V+1V (ripple ≤0.2V). Then, denoising methods such as singular value decomposition, principal component analysis, and wavelet transform are manually used to refine different types of image noise, resulting in 150 pairs of single-bacterial image sequences that can be used for model training. The training data is input, and the network can then perform supervised learning to set parameters.

[0019] like Figure 1 As shown, paired single-bacterial image sequences are input into the STBasicNet network as learning information, with a batch size of 5 and a learning rate of 0.005. The network has four modules: alignment, information extraction, fusion, and reconstruction, totaling 20 layers. LeakyRELU is used to perform nonlinear transformation on the output. The alignment module has a 7-layer neural network with an encoder-decoder architecture. Each convolutional layer has a 3D convolution kernel to extract temporal and spatial information, with sizes of 3*3*3 and 1*3*3, and an output feature mapping of 64. The 3rd and 7th layers use residual connections to alleviate the gradient vanishing problem and improve the model's learning ability. To extract information at different levels of the data, a parallel information extraction module is set up based on a multi-path learning approach. This module also has a 7-layer neural network with an encoder-decoder architecture, but uses 2D dilated convolutions with a dilation rate of 2, mainly used to extract the positional and structural information of bacteria. The output feature mapping is also 64. The outputs of the alignment and information extraction modules are concatenated and then input into the fusion module to aggregate the structural, positional, and temporal information of the sample under test, thus refining the feature representation. This module is a 3-layer neural network using 3x3 2D convolutional kernels, with an output mapping of 32, and employs an attention mechanism to focus on effective features. The final module is the aggregation module, also a 3-layer neural network. It reconstructs the denoised image sequence using features extracted by the convolutional integral fusion module. This part uses a residual learning strategy to help estimate the final denoised image, with an output mapping of 5, consistent with the initial number of input image sequences.

[0020] The total loss function L used in training the STBasicNet network model is... jhComposed of two parts, the model parameters are optimized using a hybrid loss method, and the entire loss function can be expressed as follows (1):

[0021] L jh =λ1L spa (x,x′)+λ2L det (x, x′) (1)

[0022] In the formula L spa and L det λ1 and λ2 represent the reconstruction loss and detail preservation loss, respectively. The hyperparameters λ1 and λ2 are used to adjust the weights of the reconstruction loss and detail preservation loss, respectively. x represents the noise-free ground truth image, and x′ represents the image after denoising by the model. The reconstruction loss ensures the consistency of information between the denoised single-bacterial image and the ground truth image, improving fidelity. The detail loss enables the denoised single-bacterial image to recover more details.

[0023] For reconstruction loss, this invention adopts a loss function (mean absolute error) based on L1 loss, which can be expressed as the following equation (2):

[0024]

[0025] In the formula, B represents the batch size of training images, and D(x′) t ;Θ) represents the output x′ of the noisy image sequence after passing through the model, Θ represents the learnable parameters, and t represents the temporal correlation of the sequence.

[0026] For the detail preservation loss, a method based on SSIM (structural similarity) to recover pixel details was adopted, which can be expressed as the following equation (3):

[0027] L det (x, x′) = 1 - SSIM(x) t , x′ t (3)

[0028] Finally, the parameters of each module are optimized by minimizing the loss function and backpropagating, and the network is trained through 30 iterations.

[0029] 2. The trained STBasicNet network can be used to denoise the Raman spectrum bacterial images of the bacteria under test. The denoising process is as follows: Figure 2 As shown, the input is a sequence of microscopic images obtained by a microscopic imaging system for bacterial detection using Raman optical tweezers, which continuously acquires images of a single bacterium. Figure 3(a) By using multipath encoding and decoding of image information to extract feature information and temporal information of samples contained in the image sequence, the network can fully utilize the positional, spatial, and temporal correlation information in the image sequence. The alignment module adopts a coarse-to-fine processing method, which propagates the learned offset from a lower level to a higher level, thereby gradually improving the offset accuracy and achieving the alignment of targets in the image sequence; the information extraction module uses key features extracted from adjacent image sequences to reconstruct the target image; the fusion module aggregates the structural, positional, and temporal information of the sample to be tested to improve the feature representation; the reconstruction module uses the fused information to reduce noise and outputs a denoised single-bacterial image sequence, such as... Figure 3 As shown in (b).

[0030] 3. To achieve rapid denoising of single-bacterial images in Raman optical tweezers bacterial detection, this invention utilizes acceleration on a Titan X GPU to complete the denoising task. The network learns features from the data, avoiding the hassle of manually designing algorithms and parameters, and can better adapt to different types of noise and image structures. Furthermore, the training process is accelerated using a GPU, reducing time costs. The proposed STBasicNet network structure employs a multi-path learning approach to learn positional, spatial, and temporal features from single-bacterial microscopic image sequences, adapting to different types of noise and image structures. Figure 3 (b) shows the final denoising result. It can be seen that most of the interference noise in the denoised single bacterial image has been removed, while the key information of the image has been preserved. This allows the laser tweezers to more accurately grasp the target bacteria and improve the success rate of grasping.

Claims

1. A method for denoising single-bacterial images in Raman optical tweezers bacterial detection, characterized in that, The method includes the following steps: a microscopic imaging system for Raman optical tweezers bacterial detection is selected as the implementation body; the microscopic image sequence of the sample to be tested is input into a trained STBasicNet network; the STBasicNet network has four modules: alignment module, information extraction module, fusion module, and reconstruction module; the alignment module uses a coarse-to-fine processing method to align the target in the image sequence; the information extraction module uses adjacent image sequences to extract key features to recover the target image; the fusion module aggregates the structural, positional, and temporal information of the sample to be tested to improve the feature representation; the reconstruction module uses the fused information to reduce noise and generate a denoised image of the sample; the STBasicNet network has a total of 20 neural network layers, and LeakyRELU is used to perform nonlinear transformation on the output; The alignment module has a 7-layer neural network with an encoder-decoder architecture. Each convolutional layer has a 3D convolution kernel to extract temporal and spatial information, with sizes of 3*3*3 and 1*3*3. The output feature mapping is 64. The 3rd and 7th layers use residual connections. Based on multi-path learning, a parallel information extraction module is set up. This module also has a 7-layer neural network with an encoder-decoder architecture and uses 2D dilated convolutions with a dilation rate of 2 to extract the positional structure information of bacteria. The output feature mapping is also 64. The fusion module is a 3-layer neural network with 3*3 2D convolution kernels and an output mapping of 32. It uses an attention mechanism to focus on effective features. The reconstruction module has a 3-layer neural network. It recovers the denoised image sequence by fusing the features extracted by the convolutional integral fusion module. The output mapping is 5, which is consistent with the number of the initial input image sequence. The STBasicNet network takes the noisy single bacterial image sequence as the initial input and the denoised single bacterial image sequence as the final output.

2. The method according to claim 1, characterized in that, The STBasicNet network training process described above is a supervised multi-path learning method, and the training uses a Titan X GPU to accelerate the training process.

3. The method according to claim 2, characterized in that, One iteration of training involves inputting pre-made single-bacterial image sequences containing different types of noise and denoised images into the STBasicNet network in batches. First, the alignment module and information extraction module learn the position, space, and time information in the image sequence through encoding and decoding methods in parallel. Then, the sequences are merged and connected and input into the fusion module to improve the feature representation. Finally, the reconstruction loss and detail preservation loss are calculated by comparing the denoised image output by the reconstruction module with the real denoised image. The total loss is minimized and the parameters of each module are optimized through backpropagation. The network is trained through 30 iterations.

Images