A two-photon calcium ion image neuron cell body segmentation method

Abstract

The present application belongs to the field of computer vision, image processing technology and neuroscience, and relates to a two-photon calcium ion image neuron cell body segmentation method, comprising the following steps: S1. Adopting a neuron cell body enhancement network, removing noise and neural fiber network background signals in a two-photon calcium ion video, enhancing the spatiotemporal signal of the cell body, and obtaining a cell body enhanced video; S2. Using an anchor-based target detector to detect the position of the cell body on the maximum projection image of the cell body enhanced video; S3. Through a correlation clustering module, a mask of the cell body located at the center is segmented from each anchor output by the target detector. The method of the present application takes into account the advantages of method-driven and data-driven, compared with traditional algorithms, greatly improves the efficiency of processing data, and greatly improves the precision of overlapping cell segmentation for Bessel imaging data with high cell overlap characteristics.

Description

Technical Field

[0001] This invention belongs to the fields of computer vision, image processing technology and neuroscience, and relates to a method for segmenting neuronal cell bodies in two-photon calcium ion images. Background Technology

[0002] Two-photon calcium imaging (Two-Photon Calcium Imaging) is a widely used technique in neuroscience research that allows scientists to observe neuronal activity in living organisms at single-cell resolution. Its advantages include greater penetration depth and lower light scattering, enabling imaging in deeper brain tissue. This technique is primarily implemented using two methods: Gaussian beam and Bessel beam. Gaussian beams typically offer higher lateral and axial resolution, but because they require point-by-point scanning of the volume of interest, the imaging rate is slower, making them suitable for applications requiring high resolution. Bessel beams, due to their unique shape (a very long, narrow focal point along the axis), have lower axial resolution but offer better depth penetration, allowing for faster scans and capturing a wide range of activity in a short time while maintaining consistent lateral resolution.

[0003] Following the imaging process, the first step is typically to identify the location of neuronal cell bodies (somas) from the acquired image data. Common algorithms for this task include image segmentation-based methods such as thresholding and level sets, and deep learning methods, particularly convolutional neural networks (CNNs). These methods can effectively identify neuronal cell bodies from complex and noisy backgrounds, providing a foundation for subsequent signal extraction and analysis. Existing mature open-source software, including Suite2p and CaImAn, primarily utilizes nonnegative matrix factorization (CNMF) to implement a data channel that takes calcium ion video as input and neuronal spatial morphology and emission signals as output, greatly simplifying the calcium ion data processing workflow. However, both image segmentation-based and CNMF-based methods have their limitations.

[0004] First, nonnegative matrix factorization (NMF) methods cannot simultaneously achieve high efficiency and accuracy. Generally, the algorithm's runtime increases exponentially with the data scale (including the number of neurons and time points), a problem particularly pronounced for large field-of-view, high-density neuronal video data. Second, the algorithm is less sensitive to neurons with low firing intensity or far from the focal plane (for Bessel beam imaging data), easily missing neurons with low signal strength, leading to decreased recall. Image segmentation methods, especially semantic segmentation methods based on convolutional neural networks, can more quickly and accurately obtain the neuronal cell masks on the projected image of the average or maximum value of the input video frame. Combined with post-processing methods such as threshold segmentation (e.g., watershed algorithm), most single-cell instance segmentation masks can be obtained. However, for neurons with high overlap, NMF is still unavoidably needed to separate overlapping cells. Even if the two-photon video is divided into multiple frames in the time dimension, neurons are segmented on the projected image of each frame, and the results from all "frames" are finally merged, it cannot be guaranteed that the neurons segmented on a single frame block are not overlapping, especially for Bessel beam acquisition data. Summary of the Invention

[0005] To address the aforementioned technical problems in existing technologies, this invention provides a two-photon calcium ion image neuron cell body segmentation method based on morphological detection and deep correlation clustering. The specific technical solution is as follows:

[0006] A method for segmenting neuronal cell bodies in two-photon calcium ion imaging includes the following steps:

[0007] S1. A neuronal cell body enhancement network is used to remove noise and background signals from the neural fiber network in the two-photon calcium ion video, and to enhance the spatiotemporal signals of the cell body to obtain a cell body enhanced video.

[0008] S2. Use an anchor-based target detector to detect the position of cell bodies on the maximum projection image of the cell body enhancement video;

[0009] S3. The mask of the cell body located at the center is segmented from each anchor output by the target detector through the relevant clustering module.

[0010] Furthermore, the neuronal cell body enhancement network is a Double Unet model, which is based on a 3D Unet structure video enhancement network, with the addition of a U-former structure information extraction branch and a spatiotemporal attention TSA module.

[0011] Furthermore, the 3D Block module of the video enhancement network contains two 3D convolutional layers, each followed by a ReLU activation layer and a Group Normalization layer. In the video enhancement network, the input original 3D video features are subjected to convolution operations in the temporal and spatial dimensions to remove noise and background signals of the neural fiber network in the video, thereby obtaining the enhanced 3D video features.

[0012] The U-former structural information extraction branch uses the CBAM module to filter and reorganize the input 2D structural features to obtain 2D spatial features;

[0013] The Spatiotemporal Attention (TSA) module processes 2D spatial features using the Tensor Broadcast mechanism and then multiplies and fuses them with enhanced 3D video features. The multiplication result is then subjected to depth-separable 3D convolution to obtain cell-enhanced video.

[0014] Furthermore, the U-former structural information extraction branch uses the average image of the Ground Truth video as a separate supervision signal, and is trained and optimized simultaneously with the video enhancement network, as shown in the following expression:

[0015] V enhanced ,I enhanced =DoubleUnet(V raw ,I raw ),

[0016] V enhanced For enhanced video, I enhanced For enhanced images, V raw For the original video, I raw Original image;

[0017] The optimization objective of the Double Unet model is to minimize the L1 and L2 losses, as well as the temporal first-order difference loss (Temporal), expressed by the following formula:

[0018] L1 loss =|V enhanced -V gt |+|I enhanced -I gt |

[0019] L2 loss =(V enhanced -V gt ) 2 +(I enhanced -I gt ) 2

[0020]

[0021] Among them, V gt For the target video of the simulation, I gt The average projected image of the target video is represented by T, where T represents the total number of frames in the input video.

[0022] The overall loss function is:

[0023]

[0024] Furthermore, the anchor-based target detector employs the Matrix NMS algorithm, which is an improvement upon the YOLOv7 target detector.

[0025] Furthermore, the improvement of the YOLOv7 target detector is as follows: only the maximum output layer with a downsampling rate of 8 of the YOLOv7 network is retained, and the output layers with downsampling rates of 16 and 32 are removed. At the same time, the CIou Loss loss function is used to constrain the network.

[0026] Furthermore, the specific content of S3 is as follows: First, extract the video block corresponding to the cell body position output by the target detector, then calculate the Pearson correlation coefficient between each pixel in the video block and all other pixels to obtain a two-dimensional correlation matrix, and finally input the correlation matrix into the ResNet18 network, and output a one-dimensional vector representing the probability that each pixel belongs to the central cell body, set a threshold, and obtain the mask of the anchor central cell.

[0027] Furthermore, for each neuron cell body bounding box L∈R obtained by the target detector detection and matching... 1×4 Video blocks Bi are cropped from the 3D video, i.e., the cell-enhanced video V, and their spatial dimensions are padded to a size of 20*20. All video blocks Bi are collected to form a video block set B, where the expression for video block Bi is:

[0028] B i =Resize(Crop(V,L)).

[0029] Furthermore, for each pixel p in video block Bi, the Pearson correlation coefficient γ(p,q) between it and all other pixels is calculated, resulting in a 400-dimensional correlation vector, where q represents all pixels except p:

[0030] γ(p,q)=Pearson(B i (p),B i (q))

[0031] Perform the same operation on all 400 pixels and concatenate all vectors to obtain a 400×400 correlation matrix P. iThis process involves relevant coding:

[0032] P i =[γ(p1,q1),γ(p1,q2),…,γ(p1,q400);γ(p2,q1)…;γ(p400,q400)].

[0033] Furthermore, each correlation matrix P i Input a trained ResNet18 network and output the probability of whether each pixel p belongs to the central target neuron. All The output feature map of the 1*400 network

[0034]

[0035] Dice Loss was chosen as the loss function for the ResNet18 network.

[0036]

[0037] Where i is the element index in the prediction and target tensors, and smooth is set to a constant 1 to smooth gradients and maintain numerical stability;

[0038] In the reasoning stage, Binarization yields the final classification R for whether each pixel belongs to the target neuron cell body. i ,in:

[0039]

[0040] R i =[r1,r2,…,r 400 ]

[0041] R i Transforming a one-dimensional vector into a 20x20 matrix, ultimately, each B... i ∈R 1000×20×20 Each corresponds to an R i ∈R 20 ×20 This indicates the target neuron to which the pixel belongs, and R i Cropping to the original size before filling yields the final segmentation result of the neuron cell bodies within the target bounding box.

[0042] The beneficial effects of this invention are as follows: The method of this invention is based on supervised deep learning and involves image enhancement, object detection and clustering, as well as signal dealiasing methods. It combines the advantages of method-driven and data-driven approaches and greatly improves the efficiency of data processing compared with traditional algorithms. For Bessel imaging data with high cell overlap, it greatly improves the accuracy of overlapping cell segmentation. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the two-photon neuron cell body segmentation method based on morphological detection and deep correlation clustering of the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and technical effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0045] like Figure 1 As shown, the present invention provides a method for neuronal cell body segmentation in two-photon calcium ion images based on morphological detection and deep correlation clustering. The specific process is as follows:

[0046] S1. Cell body video enhancement: A neuronal cell body enhancement network is used to remove noise and background signals from the neural fiber network in the two-photon calcium ion video image, enhance the spatiotemporal signal of the cell body, and improve the performance indicators of subsequent cell body detection.

[0047] The neuronal cell body enhancement network is a Double Unet model, consisting of two Unet branches. The main branch is a 3D Unet structure, which takes the original 3D video data as input and outputs a denoised and background-removed enhanced video. The 3D Block module in the main branch contains two 3D convolutional layers, each followed by a ReLU activation layer and a Group Normalization layer. Since the cell positions in the calcium ion video do not change (after registration), in other words, the structural information of neurons in the video hardly changes over time. Furthermore, although the noise in a single frame is very high, the average image of multiple frames is exceptionally clear. Therefore, the average image can provide more structural prior information for video enhancement, avoiding excessive artifacts and signal drift in the generated result.

[0048] Therefore, this invention adds a U-former structural information extraction branch to the video enhancement network based on 3D Unet, and designs a spatiotemporal attention TSA module to fuse the structural feature maps extracted at each scale with the main video enhancement branch. First, the original 3D video feature maps are sequentially subjected to simple convolutions in the temporal dimension (one-dimensional) and spatial dimension (two-dimensional). Then, in the spatiotemporal attention TSA module, the convolved 3D video features are multiplied with the 2D spatial features obtained from the U-former structural information extraction branch. The multiplication result is then subjected to a depthwise separable 3D convolution and added to the original 3D video features, making the training process more stable. The function of this spatiotemporal attention module is described as follows:

[0049] F ′ 3d

[0050] =DSConv3d(GN(Relu(conv2d(conv1d(F 3d )))))⊙Broadcast(CBAM(F spitial ))

[0051] Among them, F 3d F represents the 3D video feature map input to the main branch. spatial This represents the 2D structural feature map input to the U-former structural information extraction branch. This feature map is first processed by the CBAM (Convolutional Block Attention Module) for feature selection and reorganization, and then input into the spatiotemporal attention TSA module. It is then fused with 3D video features via tensor broadcasting. F′ 3d This represents the 3D video feature map output by depth-separable 3D convolution after feature fusion via the spatiotemporal attention TSA module.

[0052] Furthermore, the structural information extraction branch uses the average image from the Ground Truth video as a separate supervision signal and is trained simultaneously with the enhancement main branch, allowing the network to iteratively optimize both its cell body feature enhancement and structural information extraction capabilities. The overall network function is described as follows:

[0053] V enhanced ,I enhanced =DoubleUnet(V raw ,I raw )

[0054] The network optimization objective is to minimize the L1 and L2 losses and the temporal first-order difference loss, expressed by the following formula:

[0055] L1 loss =|V enhanced -Vgt |+|I enhanced -I gt |

[0056] L2 loss =(V enhanced -V gt ) 2 +(I enhanced -I gt ) 2

[0057]

[0058] Among them, V gt For simulated denoising and background removal (removal of nerve fibers and capillary networks) of target videos, I gt The average projected image of the target video is represented by T, and the total number of frames in the input video is represented by T.

[0059] The overall loss function is:

[0060]

[0061] S2. Cell body position detection and counting: Using a cell body detector, i.e., an anchor-based target detector, the position of cell bodies is detected on the maximum projection image of the enhanced video.

[0062] The anchor-based object detector described is an improvement upon the YOLOv7 object detector. Considering the characteristics of small, uniformly scaled, and high-density objects in images, this invention retains only the maximum output layer of the YOLOv7 network with a downsampling rate of 8, removing the other two output layers with downsampling rates of 16 and 32 to improve the network's computational efficiency. Simultaneously, the CIouLoss loss function is used to constrain the accuracy of the network's predicted bounding box regression. Due to the high degree of object overlap, to improve the accuracy and efficiency of non-maximum suppression (NMS) during the inference stage, this invention employs the Matrix NMS algorithm, an improvement upon the Soft NMS object detection algorithm, which retains more potential targets while improving the algorithm's execution efficiency.

[0063] S3. Cell Body Instance Clustering Segmentation: The relevant clustering module segments the masks of the central cells from each anchor output by the cell body detector. Specifically, the video block corresponding to the position output by the cell body detector is extracted, and the Pearson correlation coefficient is calculated between each pixel in the video and all other pixels to obtain a correlation matrix, which is a two-dimensional matrix. This two-dimensional matrix is ​​input into the ResNet18 network, and a one-dimensional vector representing the probability that each pixel belongs to the central cell is output. A threshold is set to obtain the mask of the central cell of the anchor.

[0064] During the training phase, the network is trained within the ground truth bounding box of each neuron cell body, based on the known original 3D video and the location labels of the neuron cell bodies in the video. Specifically, for each neuron cell body ground truth bounding box L∈R... 1×4 Video blocks Bi are cropped from the 3D video V. Assume that the spatial dimensions of video block Bi are padded to a size of 20*20, because the largest ground truth bounding box side length in the dataset is 19 pixels. All video blocks Bi are collected to form a video block set B. This process can be described by the following formula:

[0065] B i =Resize(Crop(V,L))

[0066] Then, for each pixel p, the Pearson correlation coefficient γ(p,q) between it and all other pixels is calculated, resulting in a 400-dimensional correlation vector, where q represents all pixels except p:

[0067] γ(p,q)=Pearson(B i (p),B i (q))

[0068] Perform the same operation on all 400 pixels and concatenate all vectors to obtain a 400×400 correlation matrix P. i This process is called correlation coding:

[0069] P i =[γ(p1,q1),γ(p1,q2),…,γ(p1,q400);γ(p2,q1)…;γ(p400,q400)];

[0070] Finally, a small but efficient ResNet18 model was trained, which converts each correlation matrix P... i As input, the output is the probability of whether each pixel p belongs to the central target neuron. All The output feature map of the 1*400 network

[0071]

[0072] Dice Loss is chosen as the loss function for the network:

[0073]

[0074] Where i is the element index in the prediction and target tensors (which have been flattened to one dimension), and smooth is set to a constant 1 to smooth gradients and maintain numerical stability.

[0075] In the reasoning stage, Binarization yields the final classification R for whether each pixel belongs to the target neuron. i ,in:

[0076]

[0077] R i =[r1,r2,…,r 400 ]

[0078] R i Transforming a one-dimensional vector into a 20x20 matrix, ultimately, each B... i ∈R 1000×20×20 Each corresponds to an R i ∈R 20 ×20 This indicates which pixels belong to the target neuron. R... i By cropping to the size before padding, you get the final segmentation result within the original target box.

[0079] The training and testing data for the network are described as follows: In this invention, all data used for network training and testing was generated by NAOMi two-photon simulation software. A total of 100 sets of video data with a size of 490*490*1000 (xyt) were generated. Each set includes the original noisy video and the corresponding denoised and background-removed (simulating only neurons, removing all nerve fibers and blood vessels) video. Among them, 50 sets of videos were used to train the neuronal cell body enhancement network, and the 1000 maximum projection images generated from all 100 background-removed videos were used to train the cell body detector. One original noisy video and all corresponding cell body anchor labels were used to train the correlation clustering network.

[0080] It was found that even when multiple neurons in the target bounding box overlap with the central target, the network can still effectively cluster the target pixels. The network can easily learn the design intent of this invention, distinguishing pixels belonging to neurons around the target bounding box from pixels of neurons located at the center of the target bounding box. Experiments show that even in data with low signal-to-noise ratios, the method of this invention can still effectively segment low-signal-intensity neurons located within the full width at half maximum (FWHM) of the fluorescence microscope axis.

[0081] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Although the implementation process of the present invention has been described in detail above, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A two-photon calcium ion image neuron cell body segmentation method, characterized in that, Includes the following steps: S1. A neuronal cell body enhancement network is used to remove noise and background signals from the neural fiber network in the two-photon calcium ion video, and to enhance the spatiotemporal signals of the cell body to obtain a cell body enhanced video. S2. Use an anchor-based target detector to detect the position of cell bodies on the maximum projection image of the cell body enhancement video; S3. First, extract the video block corresponding to the cell body position output by the target detector. Then, calculate the Pearson correlation coefficient between each pixel in the video block and all other pixels to obtain a two-dimensional correlation matrix. Finally, input the correlation matrix into the Resnet18 network and output a one-dimensional vector representing the probability that each pixel belongs to the central cell body. Set a threshold to obtain the mask of the anchor central cell. The neuronal cell body enhancement network is a Double Unet model, which is based on a 3D Unet structure video enhancement network, with the addition of a U-former structure information extraction branch, and includes a spatiotemporal attention TSA module; The 3D Block module of the video enhancement network contains two 3D convolutional layers, each followed by a ReLU activation layer and a Group Normalization layer. In the video enhancement network, the input original 3D video features are subjected to convolution operations in the temporal and spatial dimensions to remove noise and background signals of the neural fiber network in the video, thereby obtaining the enhanced 3D video features. The U-former structural information extraction branch uses the CBAM module to filter and reorganize the input 2D structural features to obtain 2D spatial features; The Spatiotemporal Attention (TSA) module processes 2D spatial features through a tensor broadcast mechanism and then multiplies and fuses them with enhanced 3D video features. The multiplication result is then subjected to a depth-separable 3D convolution to obtain the cell-body enhanced video. The U-former structural information extraction branch uses the average image of the Ground Truth video as a separate supervision signal, and is trained and optimized simultaneously with the video enhancement network. The expression is as follows: ， for an enhanced video, for an enhanced image, for an original video, for an original image; The optimization objective of the Double Unet model is to minimize the L1 and L2 losses, as well as the temporal first-order difference loss, expressed by the following formula: in, For the target video of the simulation, denoising and background removal, This represents the average projected image of the target video. Indicates the total number of frames in the input video; The overall loss function is: 。 2. The method for segmenting neuronal cell bodies in a two-photon calcium ion image as described in claim 1, characterized in that, The anchor-based target detector uses the Matrix NMS algorithm, which is an improvement on the YOLOv7 target detector.

3. The method for segmenting neuronal cell bodies in a two-photon calcium ion image as described in claim 2, characterized in that, The improvement to the YOLOv7 target detector is as follows: only the maximum output layer with a downsampling rate of 8 of the YOLOv7 network is retained, and the output layers with downsampling rates of 16 and 32 are removed. At the same time, the CIou Loss loss function is used to constrain the network.

4. The method for segmenting neuronal cell bodies in a two-photon calcium ion image as described in claim 1, characterized in that, For each neuron cell body bounding box L obtained by the target detector detection and matching Video blocks Bi are cropped from the 3D video, i.e., the cell-enhanced video V, and their spatial dimensions are padded to a size of 20 * 20. All video blocks Bi are collected to form a video block set B, where the expression for video block Bi is: 。 5. The method for segmenting neuronal cell bodies in a two-photon calcium ion image as described in claim 4, characterized in that, For each pixel p in video block Bi, calculate its Pearson correlation coefficient with all other pixels. This yields a 400-dimensional correlation vector, where q represents all pixels except p: Perform the same operation on all 400 pixels and concatenate all vectors to obtain a 400 × 400 correlation matrix. This process involves relevant coding: 。 6. The method for segmenting neuronal cell bodies in a two-photon calcium ion image as described in claim 5, characterized in that, Each correlation matrix Input a trained ResNet18 network and output the probability of whether each pixel p belongs to the central target neuron. All The output feature map of the network is composed of 1 * 400. : Dice Loss was chosen as the loss function for the ResNet18 network. in, It is the element index in the prediction and target tensors, and smooth is set to a constant 1 to smooth gradients and maintain numerical stability. In the reasoning stage, Binarization yields the final classification of whether each pixel belongs to the cell body of the target neuron. ,in: Will Transforming a one-dimensional vector into a 20x20 matrix, ultimately, each Each corresponds to one This indicates the target neuron to which the pixel belongs. Cropping to the original size before filling yields the final segmentation result of the neuron cell bodies within the target bounding box.

Images