An automatic segmentation method for the cerebral cortex surface based on geometric neural networks

Abstract

The present invention belongs to the technical field of medical image segmentation methods and relates to an automatic segmentation method for the cerebral cortex surface based on a geometric neural network. The method comprises acquiring a brain sMRI image, reconstructing the cerebral cortex surface, and extracting multidimensional morphological features of the cerebral cortex surface; decomposing the multidimensional morphological features of the cerebral cortex surface using a graph Laplacian to obtain a spectral representation of the cerebral cortex surface manifold; extracting contextual information of the cerebral cortex surface of a target image using a segmentation module with a U-shaped hierarchical structure, extracting anatomical prior information of the cerebral cortex surface of an atlas image using an auxiliary module with the same structure, and fusing information from the segmentation module and the auxiliary module using a non-local attention feature fusion module to output a predicted probability map of the segmentation label of the cerebral cortex surface of the target image. The present invention has better real-time performance and makes full use of the intrinsic structural information of the cortical surface and the global information of brain regions.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image segmentation methods, and in particular relates to an automatic cerebral cortex surface segmentation method based on geometric neural networks. Background Art

[0002] The cerebral cortex plays a crucial role in cognition, vision, and perception, and underlies complex cognitive abilities. Differences in human brain anatomy, physiology, connectivity, function, and treatment response are crucial for understanding mental health. Consequently, there is a keen interest in finding surface statistical frameworks to study various aspects of the brain.

[0003] Three-dimensional structural magnetic resonance imaging (sMRI) is a common technique used in traditional clinical examinations. It features a high signal-to-noise ratio, is insensitive to artifacts, and can provide stable morphological features such as mean curvature, cortical thickness, and sulcal depth. For many years, brain sMRI images have been used to diagnose psychiatric disorders and obtain biomarkers, most notably for predicting the prognosis of patients with behavioral disorders, often more accurately than currently used behavioral tools such as scales and structured interviews.

[0004] Analysis of cerebral cortical surface data is crucial for understanding the mechanisms underlying cognition and perception. Therefore, parcellating the cerebral cortical surface into meaningful anatomical units, whether individual brain regions or networks composed of multiple discrete but closely interacting regions, provides a reliable basis for understanding the relationship between brain function, structure, and human behavior. Cortical surface parcellation involves subdividing the cerebral cortical surface into a number of regions with distinct neurobiological significance based on various criteria. Accurate cortical surface parcellation provides maps of distinct brain regions, helping to elucidate the functional and structural organization of the brain and enabling effective comparison of findings across studies, leading to a better understanding of how the brain works. This is a continuously evolving and highly significant research area. In short, accurate cortical surface parcellation is fundamental to decoding the human brain. However, most existing segmentation methods require mapping the cortical surface onto a topologically isomorphic sphere for segmentation, which is time-consuming and fails to fully utilize the intrinsic structural information of the original cortical surface and the global information of brain regions, thus failing to achieve optimal cortical surface segmentation results. Summary of the Invention

[0005] To solve the problems raised in the above background technology, the present invention provides a method for automatic segmentation of the cerebral cortex surface based on geometric neural network, which has better real-time performance and makes full use of the intrinsic structural information of the cortical surface and the global information of the brain region.

[0006] To achieve the above object, the present invention provides the following technical solution: a method for automatic segmentation of the cerebral cortex surface based on a geometric neural network, comprising the following steps:

[0007] Acquire a brain sMRI image, reconstruct the cerebral cortex surface, and extract multi-dimensional morphological features of the cerebral cortex surface; the brain sMRI image includes a target image to be segmented and a segmented atlas set image;

[0008] The multi-dimensional morphological features of the cerebral cortex surface are decomposed using graph Laplacian to obtain the spectral representation of the cerebral cortex surface manifold;

[0009] A segmentation module with a U-shaped hierarchical structure is used to extract contextual information of the cerebral cortex surface of the target image. An auxiliary module with the same U-shaped hierarchical structure is used to extract anatomical prior information of the cerebral cortex surface of the atlas image. The information of the segmentation module and the auxiliary module are fused using a non-local attention feature fusion module to output a predicted probability map of the segmentation label of the cerebral cortex surface of the target image. The present invention further designs a geometric neural network based on B-spline graph convolution to achieve automatic segmentation of the cerebral cortex surface. Specifically:

[0010] A geometric neural network is designed for automatic segmentation of the cerebral cortical surface. The network consists of a U-shaped segmentation module (SU-Net) and a multi-atlas auxiliary module (AU-Net). These modules encode contextual information on the cortical surface and anatomical priors of brain regions, respectively, and fuse this information via a non-local attention feature fusion module. The convolution, pooling, and up-pooling operations in the U-shaped structure are implemented in the spectral domain of the cortical surface using B-spline graph convolution.

[0011] Beneficial effects of the present invention:

[0012] The present invention realizes automatic segmentation of the cerebral cortex surface based on a geometric neural network. During the process, the cerebral cortex surface is reconstructed and the multi-dimensional morphological features of the cerebral cortex surface are extracted. The spectral representation of the cerebral cortex surface manifold is obtained by using graph Laplace decomposition, which captures the intrinsic shape information of the cerebral cortex and enables learning technology to operate directly on the surface, solving the problem that most existing segmentation methods require mapping the cerebral cortex surface to a topologically isomorphic sphere for segmentation, which is time-consuming, complex to operate and has excessive influence from human subjective factors. The present invention designs a geometric neural network for automatic segmentation of the cerebral cortex surface. The network mainly consists of a U-shaped segmentation module SU-Net and a multi-atlas auxiliary module AU-Net, which respectively encode contextual information on the cortical surface and anatomical prior information of the brain area. The network performs information fusion through the non-local attention feature fusion module designed by the present invention, fully utilizing the intrinsic structural information and anatomical prior information of the original cortical surface, solving the problem that most existing segmentation methods cannot fully utilize the intrinsic structural information of the original cortical surface. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a flow chart of the automatic segmentation method of the cerebral cortex surface based on geometric neural network of the present invention;

[0014] Figure 2 This is a framework diagram of automatic segmentation of the cerebral cortex surface based on geometric neural networks of the present invention;

[0015] Figure 3 Schematic diagram of the B-spline graph convolution kernel g of the present invention;

[0016] Figure 4 This is a diagram showing the structure of the adaptive pooling and up-pooling operations of the present invention;

[0017] Figure 5 This is the structural diagram of the non-local attention feature fusion module of the present invention. DETAILED DESCRIPTION

[0018] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0019] Figure 1 This is a flow chart of a method for automatic segmentation of the cerebral cortex surface based on a geometric neural network according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0020] S1. Acquire a brain sMRI image, reconstruct the cerebral cortex surface, and extract multi-dimensional morphological features of the cerebral cortex surface; the brain sMRI image includes a target image to be segmented and a segmented atlas image;

[0021] In an embodiment of the present invention, the acquired brain sMRI image needs to be preprocessed to reconstruct the cerebral cortex surface; and then feature extraction is performed on the reconstructed cerebral cortex surface to obtain multi-dimensional morphological features of the cerebral cortex surface.

[0022] Preprocessing the acquired brain sMRI image to reconstruct the cerebral cortex surface may include the following:

[0023] The acquired brain sMRI image is mapped to a standard space; the N3 algorithm is used to correct the inhomogeneity of the brain sMRI image, and after removing non-brain tissue, the brain white matter, brain gray matter and cerebrospinal fluid are segmented based on intensity values and neighborhood constraints; the inner surface of the cerebral cortex is reconstructed using the interface between the brain white matter and the brain gray matter, and then slowly deformed to the interface between the brain gray matter and the cerebrospinal fluid to generate the outer surface of the cerebral cortex.

[0024] In an embodiment of the present invention, the process of extracting features from the reconstructed cerebral cortex surface may include:

[0025] For each vertex on the inner surface of the cerebral cortex, five morphological features reflecting different geometric properties of the cerebral cortex were extracted, including: mean curvature, whose value is the inverse of the radius of the inscribed sphere at the vertex; Gaussian curvature, whose value is the product of the principal curvatures at the vertex, reflecting the degree of curvature of the surface in different directions at that point; cortical thickness, whose value is the distance between the corresponding vertices on the white matter surface and the gray matter surface; sulcus depth, whose value is the vertical distance from the vertex to the intermediate surface between the gray and white matter; surface area, whose value is the average area of all adjacent triangular facets at the vertex.

[0026] It is understandable that those skilled in the art may also obtain the corresponding cerebral cortex surface and multi-dimensional morphological features of the cerebral cortex surface according to other reconstruction algorithms and feature extraction methods based on actual conditions, and the present invention does not make any specific limitation on this.

[0027] S2. Use graph Laplacian to perform feature decomposition on the multidimensional morphological features of the cerebral cortex surface and obtain the spectral representation of the cerebral cortex surface manifold;

[0028] In an embodiment of the present invention, the process of constructing the spectral representation of the cerebral cortex surface manifold specifically includes:

[0029] The manifold of the cerebral cortex surface is modeled as a local connected graph G = {V, E, X, A}, where is a set of N nodes (i.e., a set of vertices on the surface of the cerebral cortex), E is a set of edges, X∈RN×d is the feature matrix of the node, d represents multi-dimensionality, A∈R N×N is the adjacency matrix of graph G, and the non-zero elements in A represent an edge between two nodes.

[0030] The normalized Laplacian matrix on the graph G is Where D is a diagonal matrix and I is the identity matrix.

[0031] Perform spectral decomposition of the L matrix L = UΛU -1 , get the normalized spectral coordinates of the node Where Λ is a diagonal matrix, the diagonal elements are eigenvalues, and U is the characteristic basis matrix;

[0032] The spectral coordinates are further transformed into the reference space to obtain Where T is the spectral transformation matrix, which is obtained by the iterative closest point algorithm.

[0033] At the same time, the morphological features of the cerebral cortex are also transformed into the reference spectral space through the matrix T, and finally the spectral representation of the cerebral cortical surface manifold is obtained.

[0034] In the embodiment of the present invention, the above method can be used to obtain the popularity spectrum representation of the cerebral cortex surface, which will be segmented in the subsequent model to obtain the desired segmented image.

[0035] S3. Use the segmentation module with a U-shaped hierarchical structure to extract the contextual information of the cerebral cortex surface of the target image, use the auxiliary module with the same U-shaped hierarchical structure to extract the anatomical prior information of the cerebral cortex surface of the atlas image, and use the non-local attention feature fusion module to fuse the information of the segmentation module and the auxiliary module, and output the predicted probability map of the segmentation label of the cerebral cortex surface of the target image.

[0036] In an embodiment of the present invention, considering that in an actual cerebral cortex image, the target feature object cannot exist alone, it must have more or less connections with other feature objects around it, which is what is usually called the contextual information of the cerebral cortex surface. The present invention extracts the contextual information of the cerebral cortex surface of the target image through a segmentation module of a U-shaped hierarchical structure, which can make full use of the intrinsic structural information of the cortical surface. At the same time, the present invention also extracts anatomical prior information through an auxiliary module of a U-shaped hierarchical structure, and anatomical prior information refers to the information obtained by encoding the size, shape and position of the brain region using the cortical vertex features and segmentation label information of the atlas image. Using the anatomical prior information of the known segmented image to guide the segmentation task of the target image can make full use of the global information of the brain region and obtain a better cerebral cortex surface segmentation result.

[0037] In the embodiment of the present invention, in order to solve the problem of segmenting the cerebral cortex surface, the present invention designs a geometric neural network for automatic segmentation of the cerebral cortex surface. The framework is as follows: Figure 2 As shown in the figure, the geometric neural network mainly consists of a U-shaped hierarchical segmentation module SU-Net and a U-shaped hierarchical auxiliary module AU-Net, which encode the contextual information on the cortical surface and the anatomical prior information of the brain area respectively, and the information is fused between them through the non-local attention feature fusion module.

[0038] In the embodiment of the present invention, Figure 2 As shown, the segmentation module SU-Net of the U-shaped hierarchical structure is used as the backbone network, and the auxiliary module AU-Net of the U-shaped hierarchical structure is used as the branch network; the popular spectrum representation information of the cerebral cortex surface of the target image is input into the backbone network, and the popular spectrum representation information of the cerebral cortex surface of the multi-atlas image and the corresponding segmentation label are input into the branch network; after the backbone network and the branch network are information-fused through the non-local attention feature fusion module, the auxiliary network is used to learn the anatomical prior information of the cerebral cortex surface, optimize the contextual information of the cerebral cortex surface, and finally output the predicted probability map of the segmentation label of the cerebral cortex surface of the target image from the backbone network.

[0039] Furthermore, the geometric neural network structure proposed in the embodiment of the present invention is as follows Figure 2 As shown in Figure 2, it consists of two U-shaped segmentation modules SU-Net and multi-atlas auxiliary modules AU-Net. The input of the segmentation module SU-Net is the cortical vertex feature I of the target image. T , which contains three standardized spectral coordinates in the reference spectral space and three morphological features: curvature, cortical thickness, and sulcus depth. In the encoding stage of SU-Net, two parallel paths are used to learn hidden features and clustering matrices for pooling, which are merged to obtain pooled feature maps, and then a feature fusion module is connected to integrate the information encoded in AU-Net. In the decoding stage of SU-Net, the pooling clustering matrix of the same level is used to obtain the upper pooled feature map, which is then concatenated with the feature map of the encoding stage and then connected to the feature fusion module. Finally, a 1-layer convolution layer with R channels is connected and activated with softmax to output the predicted probability map L of the cortical segmentation label. T , where R is the number of cortical segmentation regions. The structure of the multi-atlas auxiliary module AU-Net is similar to SU-Net, and its input is the cortical vertex features and segmentation labels of the atlas image group A. The clustering matrix learned in SU-Net is used for pooling and up-pooling, and information is fused with SU-Net via a non-local attention feature fusion module. AU-Net encodes anatomical prior information such as the size and location of brain regions, which can help improve SU-Net's segmentation performance.

[0040] In an embodiment of the present invention, the segmentation module of the U-shaped hierarchical structure includes a first encoding layer and a first decoding layer; the first encoding layer includes a first input layer, and a plurality of repeatedly arranged first feature extraction layers, a first pooling layer, and a first non-local attention feature fusion module; the first decoding layer includes a plurality of repeatedly arranged second feature extraction layers, a first upper pooling layer, a second non-local attention feature fusion module, and a first output layer;

[0041] The auxiliary module of the U-shaped hierarchical structure includes a second encoding layer and a second decoding layer; the second encoding layer includes a second input layer, and a plurality of repeatedly arranged third feature extraction layers and a second pooling layer; the second decoding layer includes a plurality of repeatedly arranged fourth feature extraction layers and a second upper pooling layer;

[0042] Among them, the first pooling layer is connected to the first feature extraction layer and the third feature extraction layer, and the second pooling layer is connected to the second feature extraction layer and the fourth feature extraction layer; the first non-local attention feature fusion module is connected to the first pooling layer and the second pooling layer, and the second non-local attention feature fusion module is connected to the first upper pooling layer and the second upper pooling layer.

[0043] It can be understood that in an embodiment of the present invention, the levels of the segmentation module and the auxiliary module correspond to each other, that is, the first input layer corresponds to the second input layer, the first encoding layer corresponds to the second encoding layer, the first decoding layer corresponds to the second decoding layer, the first pooling layer corresponds to the second pooling layer, and the first upper pooling layer corresponds to the second upper pooling layer.

[0044] In addition, the levels of the segmentation module and the auxiliary module can have multiple continuous levels. For example, the multiple repeated first feature extraction layers, the first pooling layer and the first non-local attention feature fusion module can be repeated N times, and then the multiple repeated third feature extraction layers and the second pooling layers are also repeated N times accordingly, where N is an integer greater than or equal to 2; after multiple repeated arrangements, more fine-grained information can be extracted.

[0045] In a preferred embodiment of the present invention, the convolution, pooling and up-pooling operations in the U-shaped hierarchical structure can be implemented by B-spline graph convolution in the cortical surface spectral domain.

[0046] Specifically, for the cerebral cortex surface located in the non-Euclidean domain, a B-spline convolution kernel is used to perform spatial graph convolution operations.i The output of the location is N(v i ) Weighted aggregation of all information:

[0047]

[0048] Among them, z(v i ) represents node v i Embedding features, N(v i ) represents node v i The set of neighboring nodes, x∈R d is the node feature, x l (v j ) represents node v j In the feature of the lth dimension, g is the B-spline convolution kernel, express The B-spline convolution kernel value in the lth dimension; is a pseudo-coordinate that determines the local spatial relationship on the manifold, and its value is the sum of the two nodes v i With v j In the standardized spectral coordinates The difference under The value of the B-spline convolution kernel g is dynamically learned based on the local geometric properties of the cortical surface, such as Figure 3 As shown. Given a set of m-order B-spline basis functions:

[0049]

[0050] in, represents the m-th B-spline basis function of the i-th convolution kernel in the q-th dimension, k q is the convolution kernel size of the q∈{1,…,Q}th dimension, and the convolution kernel g is expressed as:

[0051]

[0052] Where ρ is the Cartesian product of all basis functions, ω ρ,l is the trainable weight of the control point ρ∈p, is a B-spline surface, q∈{1,…,Q} is a pseudo-coordinate index, and each pseudo-coordinate index corresponds to a dimension.

[0053] In an embodiment of the present invention, the U-shaped hierarchical structure includes an encoding process and a decoding process. Based on this, the present invention designs an adaptive pooling and upper pooling operation for a U-shaped network, so that the first feature extraction layer includes a hidden feature extraction layer and a clustering feature extraction layer, and the second pooling layer is connected to the clustering feature extraction layer. In this way, the second pooling layer can pool the clustering features and the graph of the input layer of the auxiliary module, pool the node features within the cluster, and adaptively learn the clustering features related to the cluster. Among the numerous input information, it focuses on the clustering feature information that is more critical to the current segmentation task, reduces the attention to other information, and even filters out irrelevant information, which can solve the information overload problem and improve the efficiency and accuracy of the segmentation task.

[0054] like Figure 4 As shown, during the encoding process, the graph G of the backbone network input layer is (l) Input two different paths, one path uses the hidden feature extraction layer composed of B-spline graph convolution SplineCNN and ReLU activation function to calculate the hidden feature Z of the node (l) =ReLU(SplineCNN(G (l) ))∈R N×M , M is the node feature dimension; the other path uses the clustering feature extraction layer composed of B-spline graph convolution kernel softmax activation function to predict the node category K (l) =softmax(SplineCNN(G (l) ))∈[0,1] N×C , C is the number of node clusters, k ic ∈K gives the probability that node i belongs to the cth cluster. The feature matrix Z after pooling (l+1) ∈R C×M It can be obtained by calculating the expected sum of all node features in cluster c:

[0055] Z (l+1) =K (l)T Z (l)

[0056]

[0057] Among them, K () represents the probability matrix at layer l, Z (l) represents the feature matrix at layer l, represents the jth feature of node i at layer l+1, Indicates the probability that node i belongs to the cth cluster in the lth dimension, i.e., in the lth layer. represents the probability that node i in layer l belongs to the cth cluster.

[0058] At the same time, the adjacency matrix A needs to be updated (l+1) ∈R C×C , can be obtained by calculating the expected sum of all connections between cluster c and cluster d:

[0059] A (l+1) =K (l)T A (l) K (l)

[0060]

[0061] Among them, A (l+1) represents the adjacency matrix of the l+1th layer, A (l) represents the adjacency matrix of the lth layer, represents the adjacency matrix of cluster c and cluster d at the l+1th layer, represents the adjacency value between node i and node i′ at level l; represents the probability that node i′ belongs to the dth cluster at the lth layer.

[0062] During the decoding process, the matrix K of the corresponding level is used (l) Pass the features of cluster c back to all nodes in the cluster to obtain the feature matrix after up-pooling

[0063]

[0064] At the same time, using the adjacency matrix A of the corresponding level (l) Restore the graph to its original structure, i.e.

[0065] In the embodiment of the present invention, Figure 5 As shown, the non-local attention feature fusion module includes two input layers, six convolutional layers, a softmax layer and an output layer; the two input layers are respectively used as the input of the segmentation module and the input of the auxiliary module, that is, the third input layer is used as the input of the segmentation module, and the fourth input layer is used as the input of the auxiliary module; the third input layer and the fourth input layer are connected to the first convolutional layer and the second convolutional layer through series connection, the first convolutional layer and the second convolutional layer are connected to the softmax layer through matrix product operation, the third input layer is connected to the third convolutional layer, and the fourth input layer is connected to the fourth convolutional layer; the third convolutional layer and the softmax layer are connected to the fifth convolutional layer through matrix product operation, the fourth convolutional layer and the softmax layer are connected to the sixth convolutional layer through matrix product operation, and the fifth convolutional layer, the sixth convolutional layer and the input layer are connected to the output layer through summation operation.

[0066] Specifically, the feature fusion module structure of the non-local attention effectively integrates the context information encoded by the SU-Net module and the anatomical prior information encoded by the AU-Net module. The input of the module is the feature matrix Z of the two modules. S ∈R C×M and Z A ∈R C×M First, use a 1×1 convolutional layer to get Z S Embedded representation of and Z A Embedded representation of in is the dimension of the embedded feature. At the same time, Z S and Z A Concatenate to get Z f ∈R C×2M , and then sent to two parallel 1×1 convolutional layers to calculate Z f Two embedding representations of and Then multiply θ with the transposed Φ and connect a softmax layer to calculate the non-local attention matrix of the concatenated features Captures the dependency between any two locations. Acting on g S and g A , then connect a 1×1 convolution layer to restore the original feature dimension and obtain the feature matrix O after attention enhancement S and O A ∈R C ×M Finally, O S and O A With the input feature matrix Z S Add up to get the module output Z o =Z S +O S +O A ∈R C×M .

[0067] In an embodiment of the present invention, the process of outputting a predicted probability map of segmentation labels of the cerebral cortex surface of a target image includes:

[0068] The first hidden feature and the first cluster feature of the target image are extracted using the segmentation module, and merged into a pooling feature matrix through pooling;

[0069] Extracting the first adjacency feature of the atlas image using an auxiliary module, and merging the first clustering feature and the first adjacency matrix into a first adjacency feature matrix through pooling;

[0070] The non-local attention feature fusion module is used to fuse the pooled feature matrix and the first adjacent feature matrix to generate the first fused feature;

[0071] Extracting a second hidden feature of the first fusion feature using a segmentation module, and merging the second hidden feature and the first clustering feature into a pooled feature matrix through down-pooling;

[0072] The auxiliary module is used to extract the second adjacency matrix of the second hidden feature, and the second adjacency feature matrix is generated by down-pooling;

[0073] The non-local attention feature fusion module is used to fuse the down-pooled features and the second adjacent feature matrix to reconstruct the second fusion matrix;

[0074] The reconstructed second fusion feature is processed to output a predicted probability map of the cerebral cortex surface segmentation label.

[0075] In the embodiment of the present invention, for the training of the entire geometric neural network model, the Dice loss L is used to solve the common class imbalance problem in medical image analysis. Dice With cross entropy loss L CE Combined loss function, the model is trained end-to-end in a supervised learning manner:

[0076]

[0077] Among them, R is the number of cortical segmentation areas, N is the number of vertices on the cortical surface, They represent the true label of vertex i belonging to the rth class, the one-hot output of the network, and the predicted probability, respectively. The hyperparameter λ is used to adjust the weights between the loss terms, and ε is used to maintain numerical stability.

[0078] In some embodiments of the present invention, the accuracy of cortical segmentation methods can be evaluated using data from the internationally recognized Mindboggle-101 public dataset. Twenty-one subjects were randomly selected as the atlas, and the remaining subjects were randomly divided into five groups for five-fold cross-validation. Segmentation accuracy was quantitatively evaluated using the Dice index and Hausdorff distance. The Dice index measures the overlap between two segmentations; a higher value indicates better results.

[0079] The Mindboggle-101 public dataset is currently the largest and most complete manually labeled public dataset of cerebral cortical surfaces. This dataset includes 101 healthy subjects, with the number of cortical surface vertices ranging from 102,000 to 185,000. Each subject's left and right cerebral cortical surfaces were manually segmented into 32 brain regions by medical experts.

[0080] The sub-regions segmented by the method of the present invention are continuous and have clear boundaries. Compared with the corresponding manual segmentation results, the segmentation results of the present invention are improved in segmentation efficiency compared with the segmentation results of other existing mapping methods, and are most similar to manual segmentation in segmentation effect.

[0081] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, which may include: ROM, RAM, disk or CD, etc.

[0082] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for automatic segmentation of the cerebral cortex surface based on geometric neural network, characterized in that: The method comprises: Acquire a brain sMRI image, reconstruct the cerebral cortex surface, and extract multi-dimensional morphological features of the cerebral cortex surface; the brain sMRI image includes a target image to be segmented and a segmented atlas set image; The multi-dimensional morphological features of the cerebral cortex surface are decomposed using graph Laplacian to obtain the spectral representation of the cerebral cortex surface manifold; A segmentation module with a U-shaped hierarchical structure is used to extract contextual information of the cerebral cortex surface of the target image. An auxiliary module with the same U-shaped hierarchical structure is used to extract anatomical prior information of the cerebral cortex surface of the atlas image. The information of the segmentation module and the auxiliary module are fused using a non-local attention feature fusion module to output a predicted probability map of the segmentation label of the cerebral cortex surface of the target image. The segmentation module of the U-shaped hierarchical structure includes a first encoding layer and a first decoding layer; the first encoding layer includes a first input layer, and a plurality of repeated first feature extraction layers, a first pooling layer, and a first non-local attention feature fusion module; the first decoding layer includes a plurality of repeated second feature extraction layers, a first upper pooling layer, a second non-local attention feature fusion module, and a first output layer; The auxiliary module of the U-shaped hierarchical structure includes a second encoding layer and a second decoding layer; the second encoding layer includes a second input layer, and a plurality of repeatedly arranged third feature extraction layers and a second pooling layer; the second decoding layer includes a plurality of repeatedly arranged fourth feature extraction layers and a second upper pooling layer; The first pooling layer is connected to the first feature extraction layer and the third feature extraction layer, and the second pooling layer is connected to the second feature extraction layer and the fourth feature extraction layer; the first non-local attention feature fusion module is connected to the first pooling layer and the second pooling layer, and the second non-local attention feature fusion module is connected to the first upper pooling layer and the second upper pooling layer; The non-local attention feature fusion module is used to fuse the information of the segmentation module and the information of the auxiliary module, including: Input Z to the segmentation module S and auxiliary module input Z A Perform embedding processing and obtain the embedding representation g of the segmentation module input S and the embedding representation g of the auxiliary module input A ; The input Z of the split module A and the input Z of the auxiliary module A Perform concatenation and convert the concatenation result Z f Perform embedding processing separately to obtain the embedding representation θ of the segmentation module and the embedding representation φ of the auxiliary module; Multiply the embedding representation θ of the segmentation module with the transposed embedding representation Φ of the auxiliary module and connect a softmax layer to calculate the non-local attention matrix of the concatenated features The non-local attention matrix Act on g respectively S and g A , and restore the original feature dimension to obtain the feature matrix O after attention enhancement. S and O A ; The feature matrix O of the split module S And the feature matrix O of the auxiliary module A and the input feature matrix Z of the segmentation module S Add together to get the feature fusion result Z o =Z S +O S +O A .

2. The method for automatic segmentation of the cerebral cortex surface based on geometric neural network according to claim 1, characterized in that: The reconstructing of the cerebral cortex surface includes mapping the brain sMRI image into a standard space, correcting the inhomogeneity of the brain sMRI image using the N3 algorithm, removing non-brain tissue, and segmenting the brain white matter, brain gray matter, and cerebrospinal fluid based on intensity values and neighborhood constraints; The inner surface of the cerebral cortex is reconstructed using the interface between the white matter and the gray matter of the brain, and the outer surface of the cerebral cortex is generated using the interface between the gray matter and the cerebrospinal fluid.

3. The method for automatic segmentation of the cerebral cortex surface based on geometric neural network according to claim 1, characterized in that: The multidimensional morphological features of the cerebral cortex surface are extracted, for each vertex on the inner surface of the cerebral cortex, five morphological features reflecting different geometric properties of the cerebral cortex are extracted, including: mean curvature, whose value is the inverse of the radius of the inscribed sphere at the vertex; Gaussian curvature, whose value is the product of the principal curvatures at the vertex, reflecting the degree of curvature of the surface in different directions at that point; cortical thickness, whose value is the distance between the corresponding vertices on the white matter surface and the gray matter surface; sulcus depth, whose value is the vertical distance from the vertex to the intermediate surface between the gray and white matter; and surface area, whose value is the average area of all adjacent triangular facets at the vertex.

4. The method for automatic segmentation of the cerebral cortex surface based on geometric neural network according to claim 1, characterized in that: The first feature extraction layer includes a hidden feature extraction layer and a clustering feature extraction layer.

5. The method for automatic segmentation of the cerebral cortex surface based on geometric neural network according to claim 4, characterized in that: The second pooling layer is also connected to the clustering feature extraction layer.

6. A method for automatic segmentation of the cerebral cortex surface based on a geometric neural network according to any one of claims 1 or 4-5, characterized in that: The non-local attention feature fusion module includes two input layers, six convolutional layers, a softmax layer and an output layer; the two input layers serve as the input of the segmentation module and the input of the auxiliary module respectively, that is, the third input layer serves as the input of the segmentation module, and the fourth input layer serves as the input of the auxiliary module; the third input layer and the fourth input layer are connected to the first convolutional layer and the second convolutional layer in series, the first convolutional layer and the second convolutional layer are connected to the softmax layer through matrix product operation, the third input layer is connected to the third convolutional layer, and the fourth input layer is connected to the fourth convolutional layer; the third convolutional layer and the softmax layer are connected to the fifth convolutional layer through matrix product operation, the fourth convolutional layer and the softmax layer are connected to the sixth convolutional layer through matrix product operation, and the fifth convolutional layer, the sixth convolutional layer and the input layer are connected to the output layer through summation operation.

7. The method for automatic segmentation of the cerebral cortex surface based on geometric neural network according to claim 1, characterized in that: The process of outputting the predicted probability map of the segmentation label of the cerebral cortex surface of the target image includes: The first hidden feature and the first cluster feature of the target image are extracted using the segmentation module, and merged into a pooling feature matrix through pooling; Extracting the first adjacent features of the atlas image using an auxiliary module, and merging the first clustering features and the first adjacent features into a first adjacent feature matrix through pooling; The non-local attention feature fusion module is used to fuse the pooled feature matrix and the first adjacency feature matrix to generate the first fused feature; Extracting a second hidden feature of the first fusion feature using a segmentation module, and merging the second hidden feature and the first clustering feature into a pooled feature matrix through down-pooling; The auxiliary module is used to extract the second adjacent features of the second hidden feature, and the second adjacent feature matrix is generated by down-pooling; The non-local attention feature fusion module is used to fuse the lower pooled features and the second adjacent feature matrix to reconstruct the second fused features; The reconstructed second fusion feature is processed to output a predicted probability map of the cerebral cortex surface segmentation label.

Images