Three-dimensional neuron image segmentation method based on segmentation and super-resolution joint model

Abstract

The application relates to a three-dimensional neuron image segmentation method based on a segmentation and super-resolution joint model, which comprises the following steps: step S1, synthesizing training data by using optimized neuron structure and cell body segmentation results; step S2, training the constructed segmentation and super-resolution joint model by using the training data synthesized in step S1; and step S3, inputting a neuron image for testing into the segmentation and super-resolution joint model trained in step S2 to obtain an image segmentation result. Compared with the prior art, the application has the advantages that the image resolution is improved while the continuous protrusion signal in the segmentation is ensured, and the problem that a neuron tracking algorithm has poor tracking effect when facing optical microscopic images with obvious image noise, uneven protrusion signal distribution and interlaced neurite protrusions is solved.

Description

Technical Field

[0001] This invention relates to an image segmentation method, and more particularly to a three-dimensional neuron image segmentation method based on a joint segmentation and super-resolution model. Background Technology

[0002] Studying the morphology and structure of neurons is crucial for revealing the diversity of nerve cells, determining neuronal connectivity, signal strength, and their role in the entire neural circuit. However, due to limitations in imaging equipment and sample labeling, neuronal images are often of poor quality, exhibiting significant noise and uneven signal distribution. Furthermore, the intricate structure of neurons makes reconstructing them from optical microscopic images extremely challenging. With the development of deep learning, the performance of automatic reconstruction algorithms has significantly improved. However, due to the unique characteristics of neuronal image imaging, the resolution of the three axes in a three-dimensional neuronal image is anisotropic, with the z-axis resolution typically only one-quarter that of the x and y axes. Therefore, neuronal images may show one protrusion very close to another, which can mislead reconstruction algorithms. This situation, where the distance between two non-parallel neurons is less than the spatial resolution, is called a crossing structure, and the pattern of crossings increases with the number of crossing structures, making it difficult for reconstruction algorithms to correctly identify them. Additionally, factors such as the point spread function, background noise, and signal radius further worsen the algorithm's performance when encountering crossing structures.

[0003] While numerous algorithms exist for automatic reconstruction, methods for addressing the aforementioned problems are very limited. G-Cut searches for optimal connections in reconstructed neurons, ensuring that the morphological features of the reconstructed neurons conform to statistically inductive patterns. However, due to the dense distribution of processes and the low resolution of the images, automatic tracking results in a large number of erroneous connections, making it difficult to find correct connections using this method. Even with careful consideration of morphological features and cell body location, a significant portion of erroneous connections, particularly those from processes far from the cell body, remain uncorrected.

[0004] A fundamental approach to solving this problem is to improve the spatial resolution of 3D neuron images. Developing optical systems or super-resolution imaging methods is theoretically feasible, but improving the resolution of optical imaging systems requires complex designs and sacrifices imaging speed, thus limiting their application in neuroscience. Deep learning-based image super-resolution reconstruction methods can establish a good mapping relationship between low-resolution and high-resolution images through data training, and then infer the corresponding high-resolution image from the low-resolution image using this mapping. However, deep learning-based image super-resolution methods require a large amount of training data, and the registration accuracy between low-resolution and high-resolution images in the training data also significantly affects the image super-resolution reconstruction results. Many methods utilize deep learning to segment 3D neuron images. These algorithms distinguish between the foreground and background of the image, enhancing the signal of the neuron image and greatly improving the tracking accuracy of tracking algorithms for weak signals. However, these segmentation algorithms are not ideal for dense neural protrusions, sometimes even increasing the radius of the foreground signal, causing protrusion signals to stick together and reducing tracking accuracy.

[0005] Therefore, how can we simultaneously solve the two problems existing in current neuron tracking: the under-tracking error caused by the quality of the image signal and the crossover error caused by the complex structure of neurons and the anisotropy of the image? Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a three-dimensional neuron image segmentation method based on a joint segmentation and super-resolution model.

[0007] The objective of this invention can be achieved through the following technical solutions:

[0008] According to one aspect of the present invention, a three-dimensional neuron image segmentation method based on a joint segmentation and super-resolution model is provided, the method comprising the following steps:

[0009] Step S1: Synthesize training data using the optimized neuron structure and cell body segmentation results;

[0010] Step S2: Use the training data synthesized in step S1 to train the constructed joint segmentation and super-resolution model;

[0011] Step S3: Input the neuron image used for testing into the joint segmentation and super-resolution model trained in step S2 to obtain the image segmentation result.

[0012] As a preferred technical solution, in step S1, NRRS and MeanShift are combined to optimize the matching degree between neuron structure and signal.

[0013] As a preferred technical solution, the NRRS is based on GD and optimizes the position of the bifurcation point and branch line segment through secondary re-tracing. GD consists of two steps: a) finding a gray-scale weighted shortest path p from the start and end points; b) optimizing the shape and structure of p.

[0014] As a preferred technical solution, MeanShift is a non-parametric estimation algorithm, specifically:

[0015] Calculate the centroid of a given point within a certain region, move the given point to the centroid, and iterate this process until the position of the point no longer changes, which is called convergence;

[0016] For a given point C k The centroid is calculated as follows:

[0017]

[0018] Where m is the centroid to be determined, and C is the centroid to be determined. k r is a given point C k The set of points within a radius r, where q is C k , where q is the pixel value of any point q in r and I(q).

[0019] As a preferred technical solution, in step S1, region growing and nnUNet are used to segment cell body signals in neuron images.

[0020] As a preferred technical solution, the joint segmentation and super-resolution model in step S2 adopts an encoder-decoder structure that combines Transformer and CNN.

[0021] As a preferred technical solution, the encoder includes a super-resolution module and a segmentation module. The super-resolution module includes a 3-layer convolutional structure, and the segmentation module includes a 4-layer downsampling module. The image is processed by the 4-layer downsampling module to obtain a feature map at low resolution. The feature map is then serialized and used as the input to the encoder. Each downsampling module includes 2 convolutional layers.

[0022] As a preferred technical solution, the decoder includes an upsampling module, which decodes the sequence into a segmentation probability map. Each upsampling module includes an upsampling operation and two transposed convolutional layers.

[0023] As a preferred technical solution, the joint segmentation and super-resolution model employs the generalized cross-entropy loss function (GCE) and the Dice loss function. Specifically, the generalized cross-entropy loss function is as follows:

[0024]

[0025] Where L q Let f(x) be the loss value of the input image x at position i. i ) represents the predicted value at position i, and y represents the predicted value at position i. i Label the point, x i The input image x has a point at position i, and q is a factor of the GCE function. When q = 0, it is equivalent to cross-entropy loss, and when q = 1, it is equivalent to average absolute error.

[0026] The Dice loss function is defined as follows:

[0027]

[0028] Where L dice This is the DICE loss, where Dice is used to measure the similarity between two masks.

[0029] As a preferred technical solution, this method combines Dice and GCE and employs a false negative detection strategy, assigning different weights to different regions of the label, as specifically defined below:

[0030] L total =w dice ·L dice (f(x i ),y i )+w gce ·L gce (f(x i ),y i )

[0031]

[0032] FN represents a false negative, L total The total loss for image point i, w dice DICE loss weights for image point i, w gce The weights of the GCE loss for image point i, L gce Let w be the GCE loss for image point i, L be the total loss for all points in the input image, and w be the loss for image point i. FN This is the weight of the false negative pixels, which is usually greater than 1 to reduce the probability of false negative predictions.

[0033] Compared with the prior art, the present invention has the following advantages:

[0034] 1) This invention can acquire fine neural protrusion segmentation images in high-resolution space and can effectively reduce the radius of neural protrusions;

[0035] 2) By increasing the resolution, the present invention can expand the distance between the boundaries of protrusion signals, thereby improving the tracking accuracy of densely intertwined neural protrusions;

[0036] 3) By segmenting neuron images, this invention can recall a large number of weak neuronal bulge signals, thereby improving the recall rate of tracking results;

[0037] 4) Experimental results demonstrate the application value of this invention in neuron reconstruction, providing a new and effective approach to solving the problem of neuron reconstruction. Attached Figure Description

[0038] Figure 1 This is a flowchart illustrating the specific process of the method of the present invention;

[0039] Figure 2 A schematic diagram of the joint segmentation and super-resolution model provided for an example of the present invention;

[0040] Figure 3 This is a schematic diagram of neuron structure optimization provided in an example of the present invention;

[0041] Figure 4 This is a schematic diagram of cell body segmentation based on region growth provided in this invention.

[0042] Figure 5 This is a schematic diagram of cell body segmentation provided in an example of the present invention;

[0043] Figure 6 This is a schematic diagram of the training data provided in this invention example;

[0044] Figure 7 This is a schematic diagram of the prediction results provided in the example of the present invention. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0046] This invention presents a 3D neuron image segmentation method based on a joint segmentation and super-resolution model. This method utilizes a Transformer structure and a super-resolution module to improve image resolution while ensuring the continuity of the segmented neurite signals. This improves the poor tracking performance of neuron tracking algorithms when faced with optical microscopic images containing significant image noise, uneven neurite signal distribution, and intersecting neurites. This invention can acquire detailed neurite segmentation images in high-resolution space and effectively reduce the radius of neurites. By combining super-resolution and segmentation, this invention simultaneously solves two problems in current neuron tracking: under-tracking errors due to image signal quality and crossover errors due to the complex structure of neurons and image anisotropy. By comparing the results of different tracking algorithms, this invention demonstrates the application value of the method in neuron tracking.

[0047] like Figure 1 As shown, the method of the present invention includes the following steps:

[0048] Step S1: Synthesize training data using the optimized neuron structure and cell body segmentation results;

[0049] Step S2: Use the training data synthesized in step S1 to train the constructed joint segmentation and super-resolution model;

[0050] Step S3: Input the neuron image used for testing into the joint segmentation and super-resolution model trained in step S2 to obtain the image segmentation result.

[0051] In step S1, NRRS ( Figure 1 F) and MeanShift Figure 1 G) Optimize the morphological structure of manually labeled neurons to fit the signal; utilize region growing ( Figure 1 B) and nnU-Net( Figure 1 C) is used to segment neuron cell bodies; the optimized neuron structure and cell body segmentation results are used to synthesize a training dataset. Figure 1 H);

[0052] Step S2 combines the segmentation module and the super-resolution module, employing a hybrid encoder-decoder structure that combines Transformer and CNN. Figure 1 I) used GCE to improve the network's prediction performance on noisy datasets. Figure 1 I).

[0053] This invention combines NRRS (Image-Annotation Morphological Matching Optimization) and MeanShift (Morphological Keypoint Position Optimization) to optimize the matching degree between neuron structure and signal. NRRS, based on Geometric Digitization (GD), optimizes the positions of bifurcation points and branch segments through secondary retracing, but neglects the tail node and its connected segments. This invention optimizes neuron structure by combining this with MeanShift, such as... Figure 3 As shown.

[0054] NRRS divides the neuron structure into segments based on bifurcation points. Segments longer than 50 μm require further segmentation. For the segmented segments, Gross Divergence (GD) is used for the first retracing. GD consists of two steps: 1) finding a gray-scale weighted shortest path p from the start and end points; 2) optimizing the morphological structure of p. For details on NRRS and GD, please refer to the following literature:

[0055] 1.NRRS: a re-tracing strategy to refine neuron reconstruction.Li etal.,2023,10.1093 / bioadv / vbad054;

[0056] 2. Automatic reconstruction of 3D neuron structures using a graph-augmented deformable model. Peng et al., 2010, 10.1093 / bioinformatics / btq212.

[0057] MeanShift is a nonparametric, density-based estimation algorithm. Its main idea is to calculate the centroid of a given point within a certain region, and then move the point to that centroid. This process is iterated until the point's position no longer changes, at which point it converges.

[0058] This invention uses nnUNet to segment cell body signals in neuron images. Figure 4 Cell body images may contain poor signal at the cell center or hollow cells due to limitations in imaging equipment and sample labeling, which is a challenge for cell body segmentation. Hollow cells are manually labeled, while images with better signal are labeled using a region growing algorithm. Figure 5 ).

[0059] This invention fuses the optimized neuron structure with cell body signals, collectively referred to as the training dataset. Figure 6 ).

[0060] The network model of this invention is based on a hybrid mechanism of CNN and Transformer, and adopts the encoder-decoder framework of U-Net. Figure 2 The encoder consists of a super-resolution module and a segmentation module. The super-resolution module contains a 3-layer convolutional structure. In the segmentation module, the image is passed through a 4-layer downsampling module to obtain a low-resolution feature map. Each downsampling module includes 2 convolutional layers. The feature map is then serialized to serve as input to the Transformer encoder. The decoder decodes the sequence into a segmentation probability map through an upsampling module. The sequence is then transformed and processed by a 5-layer cascaded upsampling module to generate the probability map. Each upsampling module contains an upsampling operation and 2 transposed convolutional layers. Simultaneously, features generated at different resolutions during the encoding stage are aggregated in the decoding stage through residual connections.

[0061] Loss Function. Sparse labeling significantly reduces the probability of dendrites from different neurons interleaving. However, neuronal axons are long and slender with wide projection ranges, some even spanning half the brain. Since the labeled data is for individual neurons, other neurons that intersect with the target neuron will not be labeled. This unlabeled portion, still representing foreground signals, indicates the presence of noise in the labeled data. To address this issue, this invention employs the generalized cross-entropy loss function. In addition to GCE, this invention also uses the Dice loss function, commonly used in segmentation tasks, to improve training performance when data is imbalanced, focusing more on the foreground region during training. Dice measures the similarity between two samples and works well for imbalanced data, focusing more on the foreground region during training. However, the loss may be unstable during training, and gradient saturation may occur under some extreme conditions. Therefore, this invention combines Dice and GCE and employs a false negative mining strategy, assigning different weights to different regions of the label. Here, FN represents a false negative, i.e., the portion of the label representing foreground where the prediction result P < 0.5. This strategy assigns higher weights to false negative regions to help the model identify neural processes with weak signals. The optimizer used during model training employs stochastic gradient descent. The learning rate decay strategy uses the "ploy" method.

[0062] Specific implementation process

[0063] This invention combines NRRS and MeanShift to optimize the matching degree between neuron structure and signal. NRRS is based on GD and optimizes the position of bifurcation points and branch segments through secondary retracing, but it ignores the tail node and its connected segments. This invention optimizes the neuron structure by combining it with MeanShift.

[0064] NRRS divides the neuron structure into segments based on bifurcation points. Segments longer than 50 μm require further segmentation. For the segmented segments, Geometric Divergence (GD) is used for the first retracing. GD consists of two steps: 1) Finding a gray-scale weighted shortest path p from the start and end points; 2) Optimizing the morphological structure of p.

[0065] 1) Find the shortest path p. For a given 3D image, construct a graph G = (V, E), where V are the vertices of the graph, i.e., the voxels of the image. E are the edges between the vertices in the graph. For two points v0 and v1, e(v0, v1) can be defined as:

[0066]

[0067] Where g I The grayscale value used to measure image voxels is defined as:

[0068] g I =exp(λ I (1-I(p) / I max ) 2 )

[0069] Where I(p) is the gray value at point p, I max λ represents the maximum gray level of the entire image. I It is a constant used to control the weight of this item. After the graph G is constructed, the shortest path p is found using Dijkstra's algorithm.

[0070] 2) Path p structure optimization. GD uses the path p generated in the first step to initialize the control points {C} of the deformed curve C. k The curve is then optimized using a local search method, where k = 1, ..., K. GD minimizes the energy function E. GD To optimize the discrete curve, E GD Defined as:

[0071] E GD =αE image +βE length +γE smoothness

[0072] Among them, E image E length and E smoothness The position of C relative to the signal, the length of C, and the smoothness of C are controlled respectively, and are defined as follows:

[0073]

[0074]

[0075]

[0076] Among them, E I (k) and E C (k) is used to determine the current point C. k grayscale value and point C k The distances relative to the signal center are defined as follows:

[0077]

[0078]

[0079] Where Θ(C) k r) represents the image with C k Let r be the center and r be the radius of the region.

[0080] GD does not change the starting and ending points. Therefore, after the first re-tracing, the main body of the line segment basically fits the signal, but the two endpoints may still deviate from the signal. Therefore, the second re-tracing is performed with the midpoint of the line segment generated in the first re-tracing as the starting point and the midpoint of the adjacent line segment as the ending point.

[0081] NRRS does not address the tail node offset signal issue; therefore, this section uses MeanShift to handle this problem. MeanShift is a non-parametric, density-based estimation algorithm. Its main idea is to calculate the centroid of a given point within a certain region, and then move the point to that centroid. This process is iterated until the point's position no longer changes, which is considered convergence. For a given point C... k The centroid is calculated as follows:

[0082]

[0083] The network model of this invention is based on a hybrid mechanism of CNN and Transformer, and adopts the encoder-decoder framework of U-Net.

[0084] Encoder. In the decoder of this invention, a three-dimensional image First, a high-resolution feature map F is obtained through a single-layer upsampling module. h The upsampling module includes transposed convolutions, two 3×3 convolutional layers, and two Elu activation functions. x then passes through four downsampling modules to obtain the low-resolution feature map F. u Each downsampling module consists of two 3×3 convolutional layers and two ELU activation functions. Then, F... u Perform feature map serialization to serve as input for the Transformer encoder.

[0085] F u First, it is segmented into a sequence of tiles. Where (P, P, P) is the 3D size of the tile, and N = DHW / P 3 C represents the number of channels. Each tile is then unfolded into a one-dimensional vector, and a tile embedding sequence is generated through a linear mapping. in In addition, location coding was added to capture spatial information. Position encoding follows the approach of Transformer, using sine and cosine functions at different frequencies:

[0086] pos 2i =sin(pos / 10000) 2i / Dim )

[0087] pos 2i+1 =cos(pos / 10000) 2i / Dim )

[0088] Where pos represents the position code and i represents the dimension. Each dimension of the position code corresponds to a sine wave, and the wavelength forms a geometric series from 2π to 10000·2π. Finally, the sequence z0 = x0 + pos is obtained.

[0089] The Transformer encoder consists of L layers, which can embed the context of the sequence z0 generated above into the sequence. Each layer of the Transformer encoder consists of two parts: a multi-headed self-attention (MSA) mechanism and a multi-layer perceptron (MLP). Layer normalization (LN) is applied before both parts, and residual connections are introduced after them.

[0090] a i-1 =MSA(LN(z) i-1 ))+z i-1

[0091] z i =MLP(LN(a i-1 ))+a i-1

[0092] Where i∈{1,…,L}. The self-attention mechanism transforms the input sequence into three distinct representation queries through three linear mappings. key Sum The calculation is as follows:

[0093]

[0094] MSA consists of n parallel heads, each corresponding to a self-attention mechanism, and then a linear transformation W is applied. O This method aggregates the results of different self-attention mechanisms, allowing the model to jointly process different subspaces. The computation is as follows:

[0095] MSA(Q,K,V)=Concat(head1,...,head n W O

[0096] The Transformer encoder takes the input sequence z0 = [z 0,1 ,…,z 0,N Mapped to z L =[z L,1 ,…,z L,N ], z L It contains rich semantic information that can be used by subsequent decoders.

[0097] Decoder. The decoder of this invention uses an upsampling module to upsample the sequence. Decoded into a segmentation probability map Where K is the number of categories. Sequence z L First, it was transformed into Then, the features are transformed into a probability map of size 4D×H×W through a 5-layer cascaded upsampling module. Each upsampling module contains an upsampling operation, two 3×3 convolutional layers, and two Elu activation functions. Simultaneously, features generated at different resolutions during the encoding stage are aggregated in the decoding stage via residual connections.

[0098] Loss Function. Sparse labeling significantly reduces the probability of dendrites from different neurons intersecting. However, neuronal axons are long and slender with wide projection ranges, some even spanning half the brain. Since the labeled data is for individual neurons, other neurons that intersect with the target neuron will not be labeled. This unlabeled portion, still representing the foreground signal, indicates the presence of a small amount of noise in the labeled data. To address this issue, this invention employs a generalized cross-entropy loss function, defined as:

[0099]

[0100] This is equivalent to the Cross Entropy (CE) loss function. When q = 1, L q (f(x i ),y iThis is equivalent to the Mean Absolute Error (MAE) loss function. MAE is a symmetric loss function, insensitive to noise, and has good robustness when dealing with noisy labels. However, since MAE performs poorly when dealing with complex datasets, GCE generalizes both MAE and CE methods.

[0101] In addition to GCE, this invention also uses the Dice loss function commonly used in segmentation tasks, defined as:

[0102]

[0103] Dice is used to measure the similarity between two samples and works well for imbalanced data, focusing more on the foreground region during training. However, the loss may be unstable during training, and gradient saturation may occur under some extreme conditions. Therefore, this invention combines Dice and GCE and adopts a false negative detection strategy, assigning different weights to different regions of the label, as defined below:

[0104] L total =w dice ·L dice (f(x i ),y i )+w gcn ·L gcn (f(x i ),y i )

[0105]

[0106] Here, FN represents false negatives, i.e., the portion of the label that is foreground but has a prediction result P < 0.5. This strategy assigns higher weights to false negative regions to help the model identify neural neurites with weak signals. In specific experiments, L gcn In this context, q is 0.1, and w... dice and w gcn Both are 1, w FN It is version 3.0.

[0107] Implementation Details. In the encoder stage of this invention, the convolutional kernel sizes of the four downsampling layers are 1×3×3, 3×3×3, 3×3×3, and 3×3×3, with strides of 1×2×2, 2×2×2, 2×2×2, and 2×2×2, respectively. This results in a feature map of size 512×4×4×4. In the Transformer encoder, the patch size is 2×2×2, the number of Transformer coding layers is 6, the head size of the multi-head self-attention mechanism is 16, the dimension of each head is 64, the dimension in the MLP is 2048, and the final output dimension is 1024. In the decoder stage, the 1024×2×2×2 feature map is linearly transformed into 512×4×4×4. Then, it undergoes four upsampling layers with strides opposite to those in the encoder: 2×2×2, 2×2×2, 2×2×2, and 1×2×2, respectively. The feature map is fused with the features upsampled in the initial stage through an upsampling layer with a stride of 4×1×1, and finally the prediction result is generated through a linear transformation.

[0108] Data augmentation. The input image size is 128×256×256. Data augmentation was performed before training. The input image was randomly scaled from 0.85 to 1.2 times and randomly flipped along three axes. Gaussian noise was added to 50% of the image after flipping. Finally, a 32×64×64 patch was randomly cut out as the model input.

[0109] Optimizer. The optimizer used during model training is Stochastic Gradient Descent (SGD), with a base learning rate γ0 of 0.001, momentum of 0.99, and weight decay of 0. The learning rate decay strategy uses the "ploy" method. N iter and N total These represent the current iteration number and the total number of iterations, respectively. During training, the batch size was 16, and the number of iterations was 24,000. The experiment used a single Nvidia RTX 3090 GPU.

[0110] This invention predicts and tracks test set data, achieving an accuracy of 0.80 and a recall of 0.87. These results are better than current neuron tracking methods, demonstrating the application value of this invention in neuron reconstruction and providing a new and effective approach to solving the neuron reconstruction problem.

[0111] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A three-dimensional neuron image segmentation method based on a joint segmentation and super-resolution model, characterized in that, The method includes the following steps: Step S1: Synthesize training data using the optimized neuron structure and cell body segmentation results; Step S2: Use the training data synthesized in step S1 to train the constructed joint segmentation and super-resolution model; Step S3: Input the neuron image used for testing into the joint segmentation and super-resolution model trained in step S2 to obtain the image segmentation result; In step S1, NRRS and MeanShift are combined to optimize the matching degree between neuron structure and signal. MeanShift is a nonparametric estimation algorithm, specifically: Calculate the centroid of a given point within a certain region, move the given point to the centroid, and iterate this process until the position of the point no longer changes, which is called convergence; For a given point The centroid is calculated as follows: in For the center of mass to be sought For a given point The set of points within a radius r for Any point q in the middle The pixel value of point q.

2. The three-dimensional neuron image segmentation method based on a joint segmentation and super-resolution model according to claim 1, characterized in that, The NRRS is based on GD and optimizes the position of bifurcation points and branch segments through secondary re-tracing. GD consists of two steps: a) finding a gray-scale weighted shortest path p from the start and end points; b) optimizing the shape structure of p.

3. The three-dimensional neuron image segmentation method based on a joint segmentation and super-resolution model according to claim 1, characterized in that, In step S1, region growing and nnUNet are used to segment cell body signals in neuron images.

4. The three-dimensional neuron image segmentation method based on a joint segmentation and super-resolution model according to claim 1, characterized in that, In step S2, the joint segmentation and super-resolution model adopts an encoder-decoder structure that combines Transformer and CNN.

5. The three-dimensional neuron image segmentation method based on a joint segmentation and super-resolution model according to claim 4, characterized in that, The encoder includes a super-resolution module and a segmentation module. The super-resolution module contains a 3-layer convolutional structure, and the segmentation module contains a 4-layer downsampling module. The image is processed by the 4-layer downsampling module to obtain a feature map at low resolution. The feature map is then serialized and used as the input to the encoder. Each downsampling module contains 2 convolutional layers.

6. The three-dimensional neuron image segmentation method based on a joint segmentation and super-resolution model according to claim 4, characterized in that, The decoder includes an upsampling module, which decodes the sequence into a segmentation probability map. Each upsampling module includes an upsampling operation and two transposed convolutional layers.

7. The three-dimensional neuron image segmentation method based on a joint segmentation and super-resolution model according to claim 4, characterized in that, The joint segmentation and super-resolution model employs the generalized cross-entropy loss function (GCE) and the Dice loss function. Specifically, the generalized cross-entropy loss function is as follows: in The loss value of the input image x at position i, The predicted value for position i, For the label of position i, Given the input image x at position i, It is a factor of the GCE function; when q=0, it is equivalent to cross-entropy loss, and when q=1, it is equivalent to average absolute error. The Dice loss function is defined as: in This is the DICE loss, where Dice is used to measure the similarity between two masks.

8. The three-dimensional neuron image segmentation method based on a joint segmentation and super-resolution model according to claim 7, characterized in that, This method combines Dice and GCE and employs a false negative detection strategy, assigning different weights to different regions of the label, as defined below: in False negative The total loss for image point i, DICE loss weights for image point i The weights of the GCE loss for image point i, For the GCE loss of image point i, The total loss for all points in the input image. The weight of the false negative pixel is greater than 1, in order to reduce the probability of false negative prediction.

Images