Automatic verification method and system suitable for neuron three-dimensional reconstruction

Abstract

The invention discloses an automatic verification method and system suitable for neuron three-dimensional reconstruction, and the method comprises the steps: S1, obtaining a three-dimensional neuron image and a neuron reconstruction result corresponding to the three-dimensional neuron image, and carrying out the preprocessing of the three-dimensional neuron image; s2, extracting all continuous path segments in the neuron reconstruction result, sampling along each path segment in the three-dimensional neuron image to obtain a plurality of slices of cross sections perpendicular to the path direction, and splicing the slices to obtain a neuron fusion image with morphological information; and S3, inputting the neuron fusion image into a verification model constructed based on a three-dimensional convolutional neural network and an attention mechanism hybrid network, and outputting a verification result. The invention provides an automatic, high-precision, high-efficiency and end-to-end neuron reconstruction result verification method and system so as to solve the core technical problems of multi-modal data fusion, neuron complex structure detection accuracy and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of biological image data processing, and particularly to an automatic checking method and system suitable for neuron three-dimensional reconstruction. BACKGROUND

[0002] The brain, as the most important organ of the human body, has a basic functional unit, which is a neuron. The neuron is the basic unit of the brain circuit, and accurate acquisition of the morphology and axon projection of the neuron is of great significance to the research of neuron classification and brain network connection. Neuron three-dimensional reconstruction is a core technology in neuroscience research, which acquires three-dimensional images (TIF format) of brain tissue through microscopic imaging (such as electron microscope / fluorescence microscope), and reconstructs the topological structure (SWC format) of the neuron. The accuracy of the reconstruction result directly affects the reliability of the research of neuron morphology analysis and brain connection map construction. However, due to the complex morphology of the neuron, the dense distribution of fibers, and the weak fluorescence signal, it is very difficult to identify and reconstruct the neuron from the image. Therefore, the bottleneck of high-throughput neuron reconstruction is neuron reconstruction checking.

[0003] In the prior art, the checking method of the neuron three-dimensional reconstruction result mainly includes manual checking and automatic algorithm checking:

[0004] The manual checking method is the checking mode commonly applied by the vast majority of current neuron reconstruction schemes, and is widely used in neuron reconstruction platforms to verify the accuracy of the automatic reconstruction result. After obtaining the automatic neuron reconstruction result, manual checking requires a person to process all unexamined nodes, and correct the local errors of the automatic reconstruction. In this process, the labeler will identify the missing branches and manually add them, disconnect the wrong connections, adjust the node position, radius or type, and mark the nodes that are difficult to judge, leaving them to be processed by experts. After the manual checking is completed, experienced neuroscientists will repair and solve complex errors (such as loop structure and cross point misjudgment). As can be seen, the manual checking method relies on expert experience, has very low work efficiency (single neuron needs several hours), and is highly subjective;

[0005] To improve the inspection efficiency, the existing solution is to use automatic algorithm inspection to replace manual inspection. The existing automatic algorithm inspection mainly realizes reconstruction inspection by constructing a local three-dimensional image analysis centered on the reconstructed node. After obtaining the automatic reconstruction result of the neuron image, the automatic reconstruction result is first constructed as a discrete point cloud sequence, and then each point of the reconstruction result is checked one by one to obtain the checking result of each point. As disclosed in the invention patent with the authorization announcement number CN110929779B, the local features corresponding to each node are extracted according to the node data of the reconstructed neuron to be detected, and the local features of each node are input into the trained convolutional neural network. The classification result is obtained through the convolutional neural network, and finally the quality detection result of the reconstructed neuron to be detected is determined according to the classification result. This scheme directly inputs the local features corresponding to each node into the deep learning model to classify the automatically reconstructed nodes, so there are the following defects: first, the point-by-point checking method easily loses the connection information between points due to the neglect of the global morphological connection relationship of the neuron fiber, so it is difficult to judge whether the reconstructed connection relationship is correct, especially in complex intersection areas, which is prone to high misjudgment rate; second, the information of a single node is too discrete, and a single CNN architecture cannot simultaneously ensure local morphological continuity and global topological rationality, and cannot explain whether a complete reconstruction is correct. SUMMARY

[0006] Therefore, to solve the above problems, the present application provides an automatic verification method and system suitable for neuron three-dimensional reconstruction.

[0007] The present application is realized by the following technical solutions:

[0008] An automatic verification method suitable for neuron three-dimensional reconstruction, comprising the following steps:

[0009] S1: Obtain a three-dimensional neuron image and its corresponding neuron reconstruction result, and pre-process the three-dimensional neuron image and the neuron reconstruction result;

[0010] S2: Extract all continuous path segments in the neuron reconstruction result, sample each path segment in the three-dimensional neuron image, obtain a plurality of cross-section slices perpendicular to the path direction, and reassemble the slices in order to obtain a neuron fusion image with morphological information;

[0011] S3: Input the neuron fusion image into a verification model constructed based on a three-dimensional convolutional neural network and an attention mechanism hybrid network, and output a verification result.

[0012] Preferably, in step S1, the three-dimensional neuron image is in TIF format, and the neuron reconstruction result is in SWC format, including the spatial coordinates, radius, type and connection relationship of each neuron node.

[0013] Preferably, in step S1, the preprocessing of the three-dimensional neuron image and the neuron reconstruction result comprises the following steps:

[0014] S11: normalizing the three-dimensional neuron image, linearly mapping all pixel values in the three-dimensional neuron image to the interval [0, 1];

[0015] S12: performing logarithmic mapping on the three-dimensional neuron image, enhancing the contrast of weak nerve fibers and suppressing background noise;

[0016] S13: cleaning the data of the neuron reconstruction result to remove empty results.

[0017] Preferably, the step S2 comprises the following steps:

[0018] S21: extracting all continuous path segments in the neuron reconstruction result, and establishing a local coordinate system centered on each node on each path segment;

[0019] S22: taking each node in each path segment as the center, sampling in a plane perpendicular to the path direction of the node, rotating the local coordinate system to align with the global coordinate system of the three-dimensional neuron image, calculating the global coordinates of all pixels in the sampling region in the global coordinate system, extracting the pixel value corresponding to each pixel from the three-dimensional neuron image through interpolation, and forming a fusion image slice of the current sampling region;

[0020] S23: splicing the fusion image slices in each path segment in order to obtain a neuron fusion image with morphological information.

[0021] Preferably, the step S21 comprises the following steps:

[0022] S211: traversing the neuron reconstruction result to extract all continuous path segments in the neuron tree structure;

[0023] S212: calculating the principal direction vector of each path segment, defining the principal direction vector of the path segment where the current node is located as the Z axis, and then defining the X axis and Y axis according to the Z axis, thereby obtaining the local coordinate system corresponding to the current node.

[0024] Preferably, the step S22 comprises the following steps:

[0025] S221: sampling in the XY plane perpendicular to the Z-axis direction of each node in each path segment by a preset radius r to obtain a circular sampling region;

[0026] S222: obtaining a square two-dimensional sampling slice by intercepting the sampling region with a fixed size;

[0027] S223: extracting all pixels in the two-dimensional sampling slice and obtaining the local coordinates of each pixel in the current local coordinate system;

[0028] S224: calculating the global coordinates of each pixel in the global coordinate system according to the local coordinates of each pixel, and obtaining the pixel value corresponding to each pixel in the three-dimensional neuron image by a linear interpolation algorithm;

[0029] S225: generating a fusion image slice of the current node according to the pixel values of all pixels in the current two-dimensional sampling slice.

[0030] Preferably, in step S3, the framework of the verification model comprises:

[0031] Input layer: for receiving the neuron fusion image and unifying the size of all fusion image slices in the current batch;

[0032] CNN network layer: for outputting a high-dimensional feature tensor, including 4 convolutional blocks connected in turn, each convolutional block having the same structure and including a 3D convolutional layer, a batch normalization layer and a LeakyReLU activation function (α = 0.01) connected in turn, and each convolutional block directly using residual connection;

[0033] Attention network layer: for outputting encoded features, including 6 layers of self-attention encoders, each layer of self-attention encoder having 8 heads;

[0034] Classifier: for decoding the encoded features and outputting a judgment result, including two fully connected layers, and the output of the second fully connected layer being connected to a Softmax function.

[0035] Preferably, the verification model is obtained by the following steps:

[0036] S31: obtaining a training set, the training set including three-dimensional neuron images and their corresponding neuron reconstruction results;

[0037] S32: at each network iteration, extracting N training data from the training set according to the batch size, pre-processing the training data by the preprocessing method of step S1, and generating the pre-processed training data into neuron fusion images with morphological information by step S2;

[0038] S33: input all neuron fusion images in the training set into the verification model for forward propagation calculation, output encoded features, and decode the encoded features using a fully connected layer to obtain model prediction classification results;

[0039] S34: calculate classification loss, thereby performing reverse gradient propagation, and optimize the verification model.

[0040] Preferably, the step S34 comprises the following steps:

[0041] S341: calculate classification loss using a Focus Loss loss function, thereby performing reverse gradient propagation;

[0042] S342: update network parameters using an AdamW optimization algorithm combined with cosine annealing learning rate, wherein the initial learning rate is set to 3e-4, the minimum learning rate 1e-6 is set as the lower limit, and a complete learning rate fluctuation is completed every 50 cycles;

[0043] S343: use a regularization method to reduce model overfitting, including applying Dropout (p=0.2) after the fully connected layer, and setting the weight decay to 1e-4;

[0044] S344: perform data augmentation on the training data, including random rotation, inversion, and adding noise to train the classification model.

[0045] The automatic verification system suitable for three-dimensional reconstruction of neurons comprises:

[0046] A preprocessing module is configured to preprocess three-dimensional neuron images and neuron reconstruction results corresponding to the three-dimensional neuron images.

[0047] A morphology information fusion module is configured to extract all continuous path segments in the neuron reconstruction results, sample each path segment in the three-dimensional neuron images to obtain a plurality of cross-section slices perpendicular to the path direction, and reassemble the slices in order to obtain neuron fusion images with morphology information.

[0048] A morphology learning module comprises a verification model constructed based on a three-dimensional convolutional neural network and an attention mechanism hybrid network, and is configured to input the neuron fusion images into the verification model and output verification results.

[0049] The beneficial effects of the technical scheme of the present application mainly include:

[0050] 1. The automatic verification method for neuron three-dimensional reconstruction disclosed in the application, through local coordinate system alignment and bilinear interpolation, a neuron fusion image with morphological information is generated, and the neuron fusion image is sequentially reassembled along the reconstruction path, the abstract SWC format neuron path is converted into a learnable neuron fusion image with morphological information, and the connection relationship between nodes is ensured to be preserved, which is more in line with the biological continuity of neuron dendrites, facilitating subsequent verification of neuron reconstruction results, in the neuron fusion image, the morphological relationship of the point cloud structure is encoded into a regular image format, realizing unified representation of heterogeneous data, and providing an input retaining global morphological features for a deep learning model.

[0051] 2. The application also designs a verification model constructed based on a three-dimensional convolutional neural network and an attention mechanism hybrid network, which combines the local feature extraction capability of the three-dimensional convolutional neural network with the global self-attention mechanism, simultaneously captures the detailed morphology and long-range connection relationship of neuron fibers, and significantly improves the classification accuracy and robustness of complex neuron structures such as intersections and windings.

[0052] 3. In the training and optimization process of the verification model, the classification loss is calculated through Focus Loss, which specifically improves the sensitivity of rare error types, at the same time, the optimizer is combined with the cosine annealing strategy to greatly improve the model training efficiency, in addition, regularization is used to prevent overfitting and data enhancement strategy is used to improve the generalization ability and robustness of the model.

[0053] 4. The automatic verification system for neuron three-dimensional reconstruction disclosed in the application has good scalability, wide adaptability, and does not depend on specific imaging equipment or neuron types, and is verified to be effective on various whole-brain neuron data sets, can automatically judge whether the neuron reconstruction result is correct or not, replaces manual inspection, and provides a reliable automated inspection tool for brain science research. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1 is a flowchart of the automatic verification method for neuron three-dimensional reconstruction in the application;

[0055] Figure 2 is a working schematic diagram of each functional module in the automatic verification system for neuron three-dimensional reconstruction in the application;

[0056] Figure 3 is a working process schematic diagram of step S2 in the automatic verification method for neuron three-dimensional reconstruction;

[0057] Figure 4 is a schematic diagram of the connection relationship of each functional layer in the verification model. DETAILED DESCRIPTION

[0058] In order to make the purposes, advantages and characteristics of the present application more clearly and in detail, the following non-restrictive description of preferred embodiments will be used to illustrate and explain the present application. The embodiments are only typical examples of the application of the technical solutions of the present application, and any technical solutions formed by equivalent replacement or equivalent transformation are within the scope of the present application.

[0059] The present application discloses an automatic verification method suitable for three-dimensional reconstruction of neurons, as shown in the formula (I), comprising the following steps: Figure 1

[0060] S1: Obtain a three-dimensional neuron image and a neuron reconstruction result corresponding thereto, and pre-process the three-dimensional neuron image and the neuron reconstruction result.

[0061] In some embodiments, the three-dimensional neuron image in step S1 is in TIF format, and in an embodiment, the resolution of the three-dimensional neuron image is 0.32x0.32x1 μm 3 , and the neuron reconstruction result is in SWC format, including the spatial coordinates, radius, type and connection relationship of each neuron node.

[0062] In some embodiments, in step S1, the pre-processing of the three-dimensional neuron image and the neuron reconstruction result comprises the following steps:

[0063] S11: Perform normalization processing on the three-dimensional neuron image, linearly map all pixel values in the three-dimensional neuron image to the interval [0, 1] as normalized data of the three-dimensional neuron image;

[0064] S12: Perform logarithmic mapping on the three-dimensional neuron image to enhance the contrast of weak nerve fibers and suppress background noise;

[0065] S13: Clean up the data of the neuron reconstruction result to remove empty results; thereby obtaining the pre-processed three-dimensional neuron image and neuron reconstruction result.

[0066] S2: Extract all continuous path segments in the neuron reconstruction result, for the convenience of subsequent splicing, the continuous path segment refers to a path segment in the neuron reconstruction result without a bifurcation structure and with consistent path direction, in the three-dimensional neuron image, sampling is performed along each path segment to obtain a plurality of cross-section slices perpendicular to the path direction, and the slices are spliced in order to obtain a neuron fusion image with morphological information.

[0067] In some embodiments, as shown in the formula (II), the step S2 comprises the following steps: Figure 3

[0068] ​​S21: Extract all continuous path segments in the neuron reconstruction result, and establish a local coordinate system respectively with each node on each path segment as the center.

[0069] In a preferred embodiment, the step S21 comprises the following steps:

[0070] S211: Traverse the neuron reconstruction result to extract all continuous path segments in the neuron tree structure;

[0071] S212: Calculate the main direction vector (i.e. path direction) of each path segment respectively, and when each node establishes its local coordinate system, take the current node as the origin, and define the main direction vector of the path segment where the node is located as the Z axis, and then define the X axis and Y axis according to the Z axis in turn, thereby obtaining the local coordinate system corresponding to the current node. Specifically, since the path direction of each path segment is consistent, the z axis of the local coordinate system of each node on the same path segment should be consistent with the main direction vector of the current path segment.

[0072] S22: Respectively with each node in each path segment as the center, sample in the plane perpendicular to the path direction of the node (i.e. the xy plane of the local coordinate system), rotate the local coordinate system to align with the global coordinate system of the three-dimensional neuron image, and calculate the global coordinates of all pixels in the sampling region in the global coordinate system. Extract the pixel value corresponding to each pixel from the three-dimensional neuron image by interpolation, and form a fusion image slice of the current sampling region.

[0073] In some embodiments, the step S22 comprises the following steps:

[0074] S221: Respectively with each node in each path segment as the center, sample in the XY plane perpendicular to the Z axis direction of the node by a predetermined radius r to obtain a circular sampling region; in a preferred embodiment, the radius r = 20 pixels;

[0075] S222: Take the sampling region by a fixed size to obtain a square two-dimensional sampling slice; in a preferred embodiment, the two-dimensional sampling slice is a three-dimensional image block with a voxel size of 41x41xL (L is the length of the current path segment);

[0076] S223: Extract all pixels in the two-dimensional sampling slice, and obtain the local coordinates of each pixel in the current local coordinate system, i.e. the coordinate position of each pixel in the XY plane in the current local coordinate system;

[0077] S224: Calculate the global coordinates of each pixel in the global coordinate system according to the local coordinates of each pixel, and obtain the pixel value corresponding to each pixel in the three-dimensional neuron image by linear interpolation algorithm;

[0078] S225: generating a fusion image slice of the current node according to the pixel values of all the pixels within the current two-dimensional sampling slice.

[0079] wherein the three-dimensional image data is discretely stored in a voxel grid, each voxel has a corresponding integer index and pixel value (gray value or intensity value), since the global coordinates are generally not integers, after obtaining the global coordinates of the current pixel, the pixel value of the corresponding coordinate point needs to be obtained using interpolation; in interpolation, the bilinear interpolation algorithm is used, for a target pixel point (x, y, z), the coordinates of the eight nearest neighboring pixel points are obtained, and the pixel value of the target pixel point is calculated by weighting the distances between the target pixel point and each neighboring pixel point, the pixel values of all the pixels within the current two-dimensional sampling slice are calculated to generate a fusion image slice.

[0080] Further, when extracting the fusion image slices at the beginning and end of the path segment, if there is a neuron fiber branch at the end of the path segment, due to the continuity of the neuron fiber, the starting end of the branch will also be retained in the fusion image slice of the current path segment, thereby retaining the connection relationship between the current path segment and its branch; at the same time, since the splicing order of the fusion image slices is strictly sorted according to the node connection order in the neuron reconstruction result, the connection relationship between the path segments is indirectly retained along with the splicing order of the fusion image slices, thereby ensuring the correctness of the connection relationship between the path segments.

[0081] S23: splicing the fusion image slices in each path segment in order to obtain a neuron fusion image with morphological information, wherein, as shown in Figure 3 after obtaining the fusion image slice of each node in each path segment, the three-dimensional neuron fusion image can be spliced by arranging each node in order in the path segment.

[0082] S3: inputting the neuron fusion image into a verification model constructed based on a three-dimensional convolutional neural network and an attention mechanism hybrid network, and outputting a verification result.

[0083] In some embodiments, the verification model is constructed based on a three-dimensional convolutional neural network and an attention mechanism hybrid network, and the framework includes:

[0084] input layer: used for receiving the neuron fusion image and unifying the size of all the fusion image slices in the current batch; since different fusion image slices may have different sizes in the same training set or image batch, in a preferred embodiment, the fusion image slice with the largest size in the current batch needs to be found, and other fusion image slices are output after being automatically filled to the size consistent with the size of the fusion image slice with the largest size, so that the size of each fusion image slice in the current batch is consistent.

[0085] CNN network layer: Used to output high-dimensional feature tensors, including 4 sequentially connected convolutional blocks. Each convolutional block has the same structure, including a sequentially connected 3D convolutional layer, batch normalization layer, and LeakyReLU activation function (α = 0.01). Each convolutional block directly uses residual connections, so that the input and output of the convolutional block are added together. The 3D convolutional layer uses a 3D convolutional kernel to perform sliding window calculation on the input fused image slices to extract its local spatial features. Each convolutional layer contains multiple convolutional kernels, and each kernel learns different feature patterns. The size of each convolutional kernel is 3×3×3, and the stride is 1×1×1.

[0086] Attention network layer: This layer receives the high-dimensional feature vectors output from the CNN network layers and outputs encoded features. It includes 6 self-attention encoder layers, each with 8 heads. The self-attention encoders calculate weights through a query-key-value mapping, using the following formula:

[0087]

[0088] In one embodiment, attention encoding is implemented using the PyTorch framework on an NVIDIA A6000 GPU server. Its operation is similar to existing technologies and will not be elaborated upon here. Figure 4 As shown, in this embodiment, after each self-attention encoder, there is usually a feedforward neural network containing a non-linear activation function (such as ReLU or GELU) to independently transform the features at each position. Layer normalization and residual connections are applied before and after each self-attention encoder and feedforward neural network. After processing by 6 layers of self-attention encoders, each feature vector in the output encoded feature sequence incorporates the global information of the high-dimensional feature tensor.

[0089] The classifier decodes the encoded features and outputs a judgment result. It includes two fully connected layers. The output of the second fully connected layer is connected to a Softmax function. Specifically, two fully connected layers are used to decode the encoded features. The first fully connected layer receives the 512-dimensional attention-encoded features output by the attention network layer and performs non-linear combination and dimensionality reduction on the attention-encoded features to output a more compact 256-dimensional hidden feature. The input of the second fully connected layer is connected to the output of the first fully connected layer, and the output dimension is 2, which is the binary classification prediction result. Then, the multiple values ​​output by the second fully connected layer are normalized by the Softmax function to generate a probability distribution. The probability value of each category is between 0 and 1, and the sum of the probabilities of all categories is 1. After obtaining the judgment result, the judgment result is output.

[0090] In some embodiments, the verification model is trained by the following steps:

[0091] S31: Obtain a training set, which includes three-dimensional neuron images and their corresponding neuron reconstruction results;

[0092] S32: At each network iteration, extract N training data from the training set according to a batch size, in this embodiment, the batch size is set to 16, pre-process the training data using the pre-processing method of step S1, and generate the pre-processed training data into neuron fusion images with morphological information using step S2;

[0093] S33: Input all neuron fusion images in the training set into the verification model for forward propagation calculation, output encoded features, and decode the encoded features using a fully connected layer to obtain model prediction classification results;

[0094] S34: Calculate the classification loss, thereby performing backward gradient propagation, and optimize the verification model.

[0095] In a preferred embodiment, the step S34 includes the following steps:

[0096] S341: Calculate the classification loss using the Focus Loss loss function, thereby performing backward gradient propagation;

[0097] S342: Update the network parameters using the AdamW optimization algorithm combined with the cosine annealing learning rate, wherein the initial learning rate is set to 3e-4, the minimum learning rate 1e-6 is set as the lower limit, and a complete learning rate fluctuation is completed every 50 cycles;

[0098] S343: Use a regularization method to reduce model overfitting, including applying Dropout (p=0.2) after the fully connected layer, and setting the weight decay to 1e-4;

[0099] S344: Perform data augmentation on the training data, including random rotation, inversion, and adding noise to train the classification model;

[0100] Wherein, the above steps are based on NVIDIA A6000 GPU server and PyTorch framework implementation, when the verification model is trained, it is deployed in the neuron reconstruction platform, and the subsequent neuron reconstruction results to be automatically verified are generated into neuron fusion images after steps S1 and S2, and the neuron fusion images are input into the trained verification model to obtain the final classification results.

[0101] In some embodiments, an automatic verification system for neuron three-dimensional reconstruction is also disclosed, comprising:

[0102] a preprocessing module configured to preprocess the three-dimensional neuron image and the neuron reconstruction result corresponding to the three-dimensional neuron image;

[0103] a morphology information fusion module configured to extract all continuous path segments in the neuron reconstruction result, sample each path segment in the three-dimensional neuron image to obtain a plurality of cross-section slices perpendicular to the path direction, and reassemble the slices in sequence to obtain a neuron fusion image with morphology information;

[0104] a morphology learning module including a verification model constructed based on a three-dimensional convolutional neural network and an attention mechanism hybrid network, configured to input the neuron fusion image into the verification model and output a verification result;

[0105] As shown in Figure 2 , Figure 4 In an embodiment, the verification model includes an input layer, an encoder, a classifier, and an output layer, wherein the encoder includes a CNN network layer and an attention network layer, the CNN network layer includes four convolutional blocks connected in sequence, each convolutional block has the same structure and includes a 3D convolutional layer, a batch normalization layer, and a LeakyReLU activation function (α = 0.01) connected in sequence, each convolutional block directly uses a residual connection, so that the input and output of the convolutional block are added, wherein the 3D convolutional layer uses a three-dimensional convolution kernel to perform sliding window calculation on the input neuron fusion image to extract local spatial features, each convolutional layer includes a plurality of convolution kernels, each kernel learns different feature patterns, and each convolution kernel has a size of 3x3x3 and a step of 1x1x1; the attention network layer includes six layers of self-attention encoders, each layer of self-attention encoder has eight heads; the classifier includes two fully connected layers, and the output of the second fully connected layer is connected to a Softmax function.

[0106] The specific limitations of the automatic verification system suitable for three-dimensional neuron reconstruction can be referred to the limitations of the automatic verification method suitable for three-dimensional neuron reconstruction described above, which will not be repeated here. Each module in the automatic verification system suitable for three-dimensional neuron reconstruction described above can be realized by software, hardware, or a combination thereof, and in an embodiment, the automatic verification system suitable for three-dimensional neuron reconstruction described above is realized based on a NVIDIA A6000 GPU server and a PyTorch framework.

Claims

1. A method for automatic verification of a three-dimensional reconstruction of neurons, characterized in that: The method comprises the following steps: S1: obtaining a three-dimensional neuron image and a neuron reconstruction result corresponding to the three-dimensional neuron image, and preprocessing the three-dimensional neuron image and the neuron reconstruction result; S2: extracting all continuous path segments in the neuron reconstruction result, sampling along each path segment in the three-dimensional neuron image to obtain a plurality of cross-section slices perpendicular to the path direction, and reassembling the slices in order to obtain a neuron fusion image with morphological information; S3: inputting the neuron fusion image into a verification model constructed based on a three-dimensional convolutional neural network and an attention mechanism hybrid network, and outputting a verification result.

2. The automatic verification method suitable for three-dimensional reconstruction of neurons according to claim 1, characterized in that: In step S1, the three-dimensional neuron image is in TIF format, and the neuron reconstruction result is in SWC format, including the spatial coordinates, radius, type and connection relationship of each neuron node.

3. The automatic verification method suitable for three-dimensional reconstruction of neurons according to claim 2, characterized in that: In step S1, the preprocessing of the three-dimensional neuron image and the neuron reconstruction result comprises the following steps: S11: performing normalization processing on the three-dimensional neuron image, and linearly mapping all pixel values in the three-dimensional neuron image to the [0, 1] interval; S12: performing logarithmic mapping on the three-dimensional neuron image to enhance the contrast of weak nerve fibers and suppress background noise; S13: cleaning the data of the neuron reconstruction result to remove empty results.

4. The automatic verification method suitable for three-dimensional reconstruction of neurons according to claim 1, characterized in that: The step S2 comprises: S21: extracting all continuous path segments in the neuron reconstruction result, and establishing a local coordinate system with each node on each path segment as the center; S22: taking each node in each path segment as the center, sampling in a plane perpendicular to the path direction of the node, rotating the local coordinate system to align with the global coordinate system of the three-dimensional neuron image, calculating the global coordinates of all pixels in the sampling region in the global coordinate system, extracting the pixel value corresponding to each pixel from the three-dimensional neuron image through interpolation, and forming a fusion image slice of the current sampling region; S23: sequentially splicing the fusion image slices in each path segment to obtain a neuron fusion image with morphological information.

5. The automatic verification method suitable for three-dimensional reconstruction of neurons according to claim 4, characterized in that: The step S21 comprises the following steps: S211: traversing the neuron reconstruction result to extract all continuous path segments in the neuron tree structure; S212: calculating the main direction vector of each path segment, taking the current node as the origin, defining the main direction vector of the path segment where the current node is located as the Z axis, and then defining the X axis and the Y axis according to the Z axis, thereby obtaining the local coordinate system corresponding to the current node.

6. The automatic verification method suitable for three-dimensional reconstruction of neurons according to claim 5, characterized in that: The step S22 comprises the following steps: S221: taking each node in each path segment as the center, sampling in the XY plane perpendicular to the Z axis direction through a preset radius r to obtain a circular sampling region; S222: obtaining a square two-dimensional sampling slice by fixing the size of the sampling region; S223: extracting all pixels in the two-dimensional sampling slice to obtain the local coordinates of each pixel in the current local coordinate system; S224: Calculate the global coordinates of each pixel in the global coordinate system according to its local coordinates, and obtain the pixel value corresponding to each pixel in the three-dimensional neuron image through a linear interpolation algorithm; S225: Generate the fusion image slice of the current node according to the pixel values of all pixels in the current two-dimensional sampling slice.

7. The automatic verification method suitable for three-dimensional reconstruction of neurons according to claim 4, characterized in that: In step S3, the architecture of the verification model includes: Input layer: used to receive the neuron fusion image and unify the size of all fusion image slices in the current batch; CNN network layer: used to output a high-dimensional feature tensor, including 4 convolutional blocks connected in turn, each convolutional block has the same structure, including a 3D convolutional layer, a batch normalization layer and a LeakyReLU activation function (α=0.01) connected in turn, and each convolutional block directly uses residual connection; Attention network layer: used to output encoded features, including 6 layers of self-attention encoders, each layer of self-attention encoder has 8 heads; Classifier: used to decode the encoded features and output a judgment result, including two fully connected layers, and the output of the second fully connected layer is connected to a Softmax function.

8. The automatic verification method suitable for three-dimensional reconstruction of neurons according to claim 7, characterized in that: The verification model is trained by the following steps: S31: Obtain a training set, which includes three-dimensional neuron images and their corresponding neuron reconstruction results; S32: In each network iteration, extract N training data from the training set according to the batch size, pre-process the training data using the pre-processing method of step S1, and generate the neuron fusion image with morphological information using step S2; S33: Input all neuron fusion images in the training set into the verification model for forward propagation calculation, output encoded features, and decode the encoded features using a fully connected layer to obtain a model prediction classification result; S34: Calculate the classification loss to perform backward gradient propagation and optimize the verification model.

9. The automatic verification method suitable for three-dimensional reconstruction of neurons according to claim 8, characterized in that: The step S34 includes the following steps: S341: Calculate the classification loss using the Focus Loss loss function to perform backward gradient propagation; S342: Update the network parameters using the AdamW optimization algorithm combined with the cosine annealing learning rate, where the initial learning rate is set to 3e-4, the minimum learning rate 1e-6 is set as the lower limit, and a complete learning rate fluctuation is completed every 50 periods; S343: Use a regularization method to reduce model overfitting, including applying Dropout (p=0.2) after the fully connected layer and setting the weight decay to 1e-4; S344: Perform data augmentation on the training data, including random rotation, inversion and adding noise to train the classification model.

10. An automatic checking system suitable for three-dimensional reconstruction of neurons, characterized in that it comprises: It includes: A pre-processing module for pre-processing three-dimensional neuron images and their corresponding neuron reconstruction results; A morphological information fusion module for extracting all continuous path segments in the neuron reconstruction result, sampling along each path segment in the three-dimensional neuron image to obtain a number of cross-sectional slices perpendicular to the path direction, and reassembling the slices in order to obtain a neuron fusion image with morphological information; The morphological learning module comprises a verification model constructed based on a three-dimensional convolutional neural network and an attention mechanism hybrid network, is used for inputting the neuron fusion image into the verification model, and outputs a verification result.

Images