A brain surface point cloud reconstruction method, electronic equipment and storage medium

Abstract

The present application belongs to the technical field of medical image processing, and in particular to a brain surface point cloud reconstruction method, an electronic device and a storage medium. The method comprises the following steps: S1: obtaining a sample set, wherein the point cloud label is used to indicate the real surface feature point cloud of the corresponding three-way orthogonal MRI slice sample; S2: constructing a neural network framework; S3: training the neural network framework using a multiple geometric loss function, wherein the loss function comprises a Chamfer distance, an improved Hausdorff distance and a normal consistency loss; and S4: inputting the three-way orthogonal MRI slice to be reconstructed into the trained framework, and outputting a brain surface point cloud containing 2048 points. The present application uses a brain surface point cloud reconstruction model based on deep learning to process three-way orthogonal MRI slice images, can fully utilize multi-view MRI data, effectively explore the potential spatial correlation between different anatomical planes, and thus significantly improve the reconstruction effect and clinical applicability.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, specifically to a method for reconstructing point clouds on the surface of the brain, an electronic device, and a storage medium. Background Technology

[0002] Three-dimensional reconstruction of the brain surface is crucial for neurosurgical planning, diagnosis of brain diseases, and neuroscience research. With the rapid development of medical technology, innovative surgical techniques such as precise minimally invasive surgery have emerged in the field of neurosurgery. While these surgeries offer significant advantages such as minimal trauma and rapid recovery, they also face the serious challenge of limited visual access. Neurosurgeons often find it difficult to obtain complete pathological information of the surgical area through direct observation, especially within complex brain structures.

[0003] Traditional three-dimensional brain reconstruction methods typically rely on complete MRI sequences, requiring the acquisition of hundreds of sequential slices. This process is not only time-consuming and costly, but also unsuitable for the clinical needs of intraoperative real-time reconstruction. While existing intraoperative magnetic resonance imaging (iMRI) technology can improve surgical navigation accuracy, a complete iMRI scan still takes 10-15 minutes, which is too long for time-sensitive neurosurgery.

[0004] Existing deep learning-based brain surface reconstruction methods mostly use a single MRI slice as input, lack stereo depth perception, cannot accurately represent the complex concave and convex structures of the brain, and ignore the potential spatial relationships between different anatomical planes, resulting in poor reconstruction results.

[0005] Existing technologies have the following problems: traditional methods require complete 3D MRI sequences, which are costly and time-consuming to acquire, and are not suitable for real-time intraoperative reconstruction; single-slice reconstruction methods lack stereoscopic depth perception and cannot accurately represent the complex concave and convex structures of the brain; existing methods have insufficient reconstruction accuracy and poor structural integrity in complex structural regions such as sulci and gyri; there is a lack of effective multi-view information fusion mechanisms, which fail to fully utilize the complementary information of different anatomical planes; and there is a lack of specialized network architectures suitable for point cloud data, making it difficult for traditional convolutional networks to handle non-Euclidean structures.

[0006] Therefore, this invention proposes a method for reconstructing point clouds on the brain surface, an electronic device, and a storage medium. Summary of the Invention

[0007] The purpose of this section is to outline some aspects of the embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0008] To solve the above-mentioned technical problems, according to one aspect of the present invention, the present invention provides the following technical solution: a method for reconstructing point clouds on the surface of the brain, comprising the following steps:

[0009] S1: Obtain a sample set, which includes multiple pairs of tri-orthogonal MRI slice samples and point cloud labels. The point cloud labels are used to indicate the true surface feature point cloud of the corresponding tri-orthogonal MRI slice sample.

[0010] S2: Construct a neural network framework, the framework including:

[0011] The multi-view encoder consists of three parallel branches, each containing a multi-scale feature encoding module. It uses dilated convolutions with different dilation rates to capture multi-scale brain tissue structural features and generate target multi-view features.

[0012] The cross-slice feature aggregation module includes a cross-view correlation calculation submodule and a channel-spatial attention mechanism, which are used to fuse complementary information from three directions;

[0013] The 3D Transformer global context modeling module is used to perform global context modeling, establish long-range dependencies between multiple views, and output target global features.

[0014] The hierarchical graph convolutional network decoder uses edge convolution and residual graph attention mechanisms to generate detailed point clouds of the brain surface layer by layer.

[0015] S3: The neural network framework is trained using multiple geometric loss functions, including Chamfer distance, improved Hausdorff distance, and normal consistency loss.

[0016] S4: Input the three-dimensional orthogonal MRI slices to be reconstructed into the trained framework, and output a brain surface point cloud containing 2048 points.

[0017] In a preferred embodiment of the brain surface point cloud reconstruction method described in this invention, the multi-scale feature encoding module in step S2 includes:

[0018] The dilated convolution residual block contains multiple parallel dilated convolution branches with dilation rates of 1, 2, 3 and 6, forming a hybrid dilated pyramid structure.

[0019] The multi-level feature extractor consists of three layers of dilated convolutional residual blocks with the number of channels being base_channels×2, base_channels×4, and base_channels×8, respectively, outputting a hierarchical structure of low-level, mid-level, and high-level features.

[0020] In a preferred embodiment of the brain surface point cloud reconstruction method described in this invention, the processing procedure of the cross-view correlation calculation submodule in step S2 is as follows:

[0021] High-level features of three views ,in These represent the high-level semantic features obtained from sagittal, coronal, and horizontal MRI slices after feature encoding. To fully explore the complementary information between different views, the feature correlation between any two views is calculated using the following formula:

[0022]

[0023] in This represents the feature correlation matrix between the i-th view and the j-th view; The high-level semantic features corresponding to the i-th view are listed; This represents the high-level semantic feature corresponding to the j-th view; i and j represent the view indices, with values ​​ranging from 1 to 10. ,and T represents the matrix transpose operation; Represents the normalization function;

[0024] By mapping the features of one view to the feature space of another view using a correlation matrix, we obtain the features of the j-th view after cross-view correlation enhancement. :

[0025]

[0026] The original features and the relevance-enhanced features are fused to obtain the optimized view features. :

[0027]

[0028] in This represents the concatenation operation of two features along the channel dimension. This indicates a convolutional fusion operation, used for feature mapping and information integration of the concatenated features.

[0029] In a preferred embodiment of the brain surface point cloud reconstruction method described in this invention, the channel-spatial attention mechanism in step S2 includes:

[0030] The channel attention submodule performs global average pooling on the stacked three-view features and generates channel weights through a multilayer perceptron.

[0031] ,in Let F represent the features after channel attention processing, and let F represent the input feature tensor. For global average pooling, It is a multilayer perceptron. for Activation function;

[0032] The spatial attention submodule uses 3D convolution operations to capture salient regions of features in the spatial dimension.

[0033] ,in This represents the features after spatial attention enhancement. For 3D convolution operation, σ is Activation function.

[0034] In a preferred embodiment of the brain surface point cloud reconstruction method described in this invention, the 3D Transformer global context modeling module in step S2 includes:

[0035] The position encoder flattens the fused multi-view feature tensor in the spatial dimension to generate a two-dimensional coordinate position embedding.

[0036] A view token generator introduces a unique view token for each view as a representative of the view's global semantics;

[0037] The multi-layer Transformer encoder, which includes multi-head self-attention and feedforward networks, captures complex dependencies between different views and spatial locations through a self-attention mechanism.

[0038] In a preferred embodiment of the brain surface point cloud reconstruction method described in this invention, in step S2, the hierarchical graph convolutional network decoder includes:

[0039] The edge convolution module constructs a k-nearest neighbor graph and learns local geometric features by aggregating the feature differences between the center point and its neighbors:

[0040]

[0041] in This represents the local geometric features of the i-th point after edge convolution aggregation. Point of nearest neighbor set Represents the difference feature. Implemented by a multilayer perceptron;

[0042] The residual graph attention mechanism calculates the correlation between points through feature transformations of Query, Key, and Value:

[0043]

[0044]

[0045] in This represents the attention weight matrix between point cloud nodes. This represents the output features of the residual graph attention module, where Q, K, and V are the query, key, and value matrices, respectively. For the number of channels, This represents a one-dimensional convolution operation. This indicates a modified linear unit activation function. For input features;

[0046] The hierarchical upsampling structure contains two levels that generate point clouds of 512 and 2048 points respectively.

[0047] In a preferred embodiment of the brain surface point cloud reconstruction method described in this invention, the multiple geometric loss function in step S3 includes:

[0048] Chamfer distance loss measures the overall matching degree between the predicted point cloud and the target point cloud:

[0049] ,in Let P represent the predicted point cloud and G represent the ground truth point cloud. Here, p represents any point in the predicted point cloud set P, and g represents any point in the ground truth point cloud set G. |P| and |G| represent the number of points in the predicted point cloud set and the ground truth point cloud set, respectively. Indicates Euclidean distance calculation;

[0050] Improved Hausdorff distance loss, focusing on outlier regions and local salient features:

[0051] ,in This represents an improved Hausdorff distance loss function. Indicates from arrive The nearest distance is sorted in descending order and the first one is taken. indivual, Indicates from arrive The nearest distance is sorted in descending order and the first one is taken. indivual, This indicates the operation of calculating the average value;

[0052] Normal uniformity loss ensures surface smoothness and local geometric properties:

[0053] ,in Represents the normal consistency loss function. and It is a point and The normal vector;

[0054] Comprehensive loss function:

[0055] ,in This represents the overall loss function during model training. The values ​​are 1.0, 0.2, and 0.05, respectively.

[0056] As a preferred embodiment of the brain surface point cloud reconstruction method described in this invention, it further includes a data preprocessing step:

[0057] Convert MRI data from DICOM format to tensor format;

[0058] The center slices of the three orthogonal directions are extracted, normalized to the range of [-1, 1] and adjusted to a resolution of 224×224;

[0059] A brain surface point cloud containing 2048 points was constructed as a supervision signal through precise surface reconstruction and uniform sampling;

[0060] The coordinates are aligned to the standard coordinate system and normalized to the unit sphere using a rigid transformation.

[0061] An electronic device includes a processor; and a memory communicatively connected to the processor, the memory storing computer-executable instructions, the processor executing the computer-executable instructions to implement the steps of a brain surface point cloud reconstruction method when executed by the processor.

[0062] A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement a method for reconstructing point clouds on the surface of the brain.

[0063] Compared with existing technologies, the advantages of this invention are as follows: This invention acquires triaxial orthogonal MRI slice images of the brain to be reconstructed; inputs these triaxial orthogonal MRI slice images into a deep learning-based brain surface point cloud reconstruction model, which includes a multi-view encoder module, a cross-slice feature aggregation module, a 3D Transformer global context modeling module, and a hierarchical graph convolutional network decoder module; the multi-view encoder module extracts multi-scale features from the triaxial orthogonal MRI slice images to generate target multi-view features; the cross-slice feature aggregation module performs cross-view correlation calculation and channel-space attention fusion on the target multi-view features to output target fused features; and the target fused features are input into a 3D... The Transformer global context modeling module performs global context modeling and outputs target global features. The target global features are then input into the hierarchical graph convolutional network decoder module to generate hierarchical point clouds and output target brain surface point clouds. Based on the above scheme, this invention uses a deep learning-based brain surface point cloud reconstruction model to process triaxial orthogonal MRI slice images. This fully utilizes multi-view MRI data and effectively explores the potential spatial relationships between different anatomical planes, thereby significantly improving reconstruction results and clinical applicability. Attached Figure Description

[0064] To more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and detailed embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0065] Figure 1 This is a flowchart of the method in Embodiment 1 of the present invention;

[0066] Figure 2 This is a schematic diagram of the overall architecture of the brain surface point cloud reconstruction model based on deep learning in Embodiment 1 of the present invention.

[0067] Figure 3 This is a schematic diagram of the multi-view encoder module in Embodiment 1 of the present invention;

[0068] Figure 4 This is a schematic diagram of the cross-slice feature aggregation module in Embodiment 1 of the present invention;

[0069] Figure 5 This is a schematic diagram of the structure of the 3D Transformer global context modeling module in Embodiment 1 of the present invention;

[0070] Figure 6 This is a schematic diagram of the structure of the hierarchical graph convolutional network decoder module in Embodiment 1 of the present invention;

[0071] Figure 7 This is a schematic diagram of the edge convolution submodule in Embodiment 1 of the present invention;

[0072] Figure 8 This is a schematic diagram of the residual graph attention submodule in Embodiment 1 of the present invention;

[0073] Figure 9 This is a comparison chart of point cloud reconstruction results in Embodiment 1 of the present invention;

[0074] Figure 10 This is the reconstruction quality evolution diagram during the training process in Embodiment 1 of the present invention;

[0075] Figure 11 This is a flowchart of the model training process for the brain surface point cloud reconstruction model based on deep learning in Embodiment 2 of the present invention. Detailed Implementation

[0076] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0077] Secondly, the present invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of the present invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not according to the usual scale. Furthermore, the schematic diagrams are merely examples and should not limit the scope of protection of the present invention. In addition, actual fabrication should include three-dimensional spatial dimensions of length, width, and depth.

[0078] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0079] Example 1:

[0080] Please see Figure 1 This invention provides a method for reconstructing brain surface point clouds based on deep learning, comprising:

[0081] Step 101: Obtain triaxial orthogonal MRI slice images of the brain to be reconstructed, including sagittal slices, coronal slices, and horizontal slices.

[0082] Tri-orthogonal MRI slice images include MRI slices in three orthogonal directions: sagittal, coronal, and axial. These three directions provide complementary spatial information, effectively reducing geometric uncertainty, while the acquisition cost is significantly lower than that of a complete 3D MRI sequence. All slices are normalized to the range [-1, 1] and adjusted to 224×224 resolution, then aligned to the standard coordinate system through a rigid transformation.

[0083] Step 102: Input the three-dimensional orthogonal MRI slice image into a deep learning-based brain surface point cloud reconstruction model. The deep learning-based brain surface point cloud reconstruction model includes a multi-view encoder module, a cross-slice feature aggregation module, a 3D Transformer global context modeling module, and a hierarchical graph convolutional network decoder module.

[0084] Please see Figure 2 The deep learning-based brain surface point cloud reconstruction model proposed in this invention consists of a multi-view encoder module, a cross-slice feature aggregation module, a 3D Transformer global context modeling module, and a hierarchical graph convolutional network decoder module. This model can reconstruct high-precision brain surface point clouds from MRI slices in only three orthogonal directions, significantly reducing MRI acquisition costs while maintaining reconstruction accuracy.

[0085] Step 103: Use the multi-view encoder module to extract multi-scale features from the three-dimensional orthogonal MRI slice images to generate target multi-view features.

[0086] The multi-view encoder module includes a first multi-scale feature extractor, a second multi-scale feature extractor, and a third multi-scale feature extractor.

[0087] The target multi-view features include first view features (corresponding to sagittal slices), second view features (corresponding to coronal slices), and third view features (corresponding to horizontal slices).

[0088] Specifically, step 103 may include the following sub-steps:

[0089] Step S31: Use the first multi-scale feature extractor to extract the dilated convolution residual features of the sagittal slice and output the first view features;

[0090] Step S32: Use the second multi-scale feature extractor to extract the dilated convolution residual features of the coronal slice and output the second view features;

[0091] Step S33: Use the third multi-scale feature extractor to extract the dilated convolution residual features of the horizontal slice and output the third view features;

[0092] Please see Figure 3 Each multi-scale feature extractor comprises multiple dilated convolutional residual blocks with dilation rates of 1, 2, 3, and 6, forming a hybrid dilation pyramid. This design enables the network to capture both local details and global structure simultaneously, making it particularly suitable for representing complex structures such as gyri and sulci. The encoder consists of three layers of dilated convolutional residual blocks with channel numbers of base_channels×2, base_channels×4, and base_channels×8, respectively. Residual connections ensure stable gradient propagation, effectively mitigating the vanishing gradient problem during deep network training.

[0093] Step 104: Perform cross-view correlation calculation and channel-space attention fusion on the target multi-view features through the cross-slice feature aggregation module, and output the target fused features.

[0094] The cross-slice feature aggregation module includes a cross-view correlation calculation unit, a channel attention unit, and a spatial attention unit.

[0095] Specifically, step 104 may include the following sub-steps:

[0096] Step S41: Use the cross-view correlation calculation unit to perform inter-view correlation calculation on the first view feature, the second view feature, and the third view feature, and output the cross-view correlation matrix;

[0097] Step S42: Enhance the correlation of the first view feature, the second view feature, and the third view feature according to the cross-view correlation matrix, and output the correlation-enhanced feature;

[0098] Step S43: Stack the correlation enhancement features into a five-dimensional tensor;

[0099] Step S44: Use the channel attention unit to perform channel dimension attention calculation on the five-dimensional tensor and output the channel attention features;

[0100] Step S45: Use the spatial attention unit to perform spatial dimension attention calculation on the channel attention features and output the target fusion features.

[0101] Please see Figure 4 The cross-slice feature aggregation module is specifically designed to handle heterogeneous information from different anatomical planes. Cross-view correlation calculation is performed using the following formula:

[0102]

[0103] in This represents the feature correlation matrix between the i-th view and the j-th view; The high-level semantic features corresponding to the i-th view are listed; This represents the high-level semantic feature corresponding to the j-th view; i and j represent the view indices, with values ​​ranging from 1 to 10. ,and T represents the matrix transpose operation; This represents the normalization function. The correlation matrix can be used to map the features of one view to the feature space of another view, thereby enhancing the structural consistency between views.

[0104] Channel attention is calculated using the following formula:

[0105]

[0106] in Let F represent the features after channel attention processing, and let F represent the input feature tensor. For global average pooling, It is a multilayer perceptron. for Activation function;

[0107] Spatial attention is calculated using the following formula:

[0108]

[0109] in This represents the feature after spatial attention enhancement, where σ is... The activation function, Conv3D, is a 3D convolution operation. Channel attention and spatial attention mechanisms work together, enabling the network to automatically focus on the feature channels and spatial regions most critical for 3D reconstruction.

[0110] Step 105: Input the target fusion features into the 3D Transformer global context modeling module to perform global context modeling and output the target global features.

[0111] The 3D Transformer global context modeling module includes a position encoding unit, a view token generation unit, a multi-head self-attention unit, and a feedforward network unit.

[0112] Specifically, step 105 may include the following sub-steps:

[0113] Step S51: Flatten the target fusion features in the spatial dimension to generate a spatial feature sequence;

[0114] Step S52: Add two-dimensional position coding to the spatial feature sequence using the position coding unit;

[0115] Step S53: Use the view token generation unit to generate a dedicated view token for each view;

[0116] Step S54: Concatenate the viewpoint token with the spatial feature sequence to form an input sequence;

[0117] Step S55: Use the multi-head self-attention unit to perform self-attention calculation on the input sequence and output attention features;

[0118] Step S56: Use the feedforward network unit to process the attention features and output the target global features.

[0119] Please see Figure 5 The 3D Transformer global context modeling module aims to enhance the modeling capabilities of global structural consistency and complex geometric relationships by enabling information interaction between multiple views and spatial locations through a self-attention mechanism. The fused multi-view feature tensor is first flattened in the spatial dimension to obtain the spatial feature sequence for each view. To preserve spatial location information, two-dimensional coordinates are generated for each spatial location and mapped to a vector with the same number of feature channels using a multilayer perceptron, serving as the location embedding. A dedicated view token is introduced for each view, representing the global semantics of that view. The self-attention mechanism can capture complex dependencies between different views and spatial locations, achieving efficient interaction and fusion of global information.

[0120] Step 106: Input the target global features into the hierarchical graph convolutional network decoder module to generate hierarchical point clouds and output the target hierarchical point cloud features.

[0121] The hierarchical graph convolutional network decoder module includes a first-level decoder and a second-level decoder.

[0122] Specifically, step 106 may include the following sub-steps:

[0123] Step S61: The first-level decoder is used to perform edge convolution and residual map attention processing on the target global features to output the first-level point cloud features, which contain 512 points.

[0124] Step S62: Upsample the first-level point cloud features;

[0125] Step S63: The second-level decoder is used to perform edge convolution and residual map attention processing on the upsampled first-level point cloud features to output the second-level point cloud features, which contain 2048 points.

[0126] Step S64: Use the second-level point cloud features as the target-level point cloud features.

[0127] Please see Figure 6 The hierarchical graph convolutional network decoder employs a coarse-to-fine hierarchical structure, progressively refining the geometric details of the point cloud. Compared to traditional convolutional networks, graph-based neural networks are better suited for processing non-Euclidean data such as point clouds. The decoder contains two layers, generating point clouds of 512 and 2048 points respectively. Each layer includes edge convolution, residual graph attention, and coordinate prediction modules.

[0128] Edge convolution is achieved using the following formula:

[0129]

[0130] in This represents the local geometric features of the i-th point after edge convolution aggregation. Point of nearest neighbor set Represents the difference feature. Implemented by a multilayer perceptron;

[0131] Please see Figure 7 The edge convolution module effectively captures the local geometry of the point cloud. For each point, EdgeConv constructs a k-nearest neighbor graph and learns local geometric features by aggregating the feature differences between the center point and its neighbors.

[0132] Residual plot attention is achieved using the following formula:

[0133]

[0134]

[0135] in This represents the attention weight matrix between point cloud nodes. This represents the output features of the residual graph attention module, where Q, K, and V are the query, key, and value matrices, respectively. For the number of channels, This represents a one-dimensional convolution operation. This indicates a modified linear unit activation function. For input features;

[0136] Please see Figure 8The residual graph attention module combines self-attention with residual connections, enabling each point to adaptively aggregate global information while ensuring the stability of feature flow and the trainability of the network. The global aggregated features for each point are obtained by weighted summation of the Value features of all points using attention weights, and the quality of feature transfer is ensured through residual connections.

[0137] The hierarchical graph convolutional network decoder directly performs the final coordinate prediction and geometric refinement, outputting a brain surface point cloud containing 2048 points.

[0138] For comparison of technical effectiveness, existing technologies can be referenced. Among existing technologies, traditional reconstruction methods based on complete MRI sequences and deep learning methods based on single slices are two implementation schemes that are relatively close to the present invention. Although traditional methods have high accuracy, they require the acquisition of hundreds of continuous slices, which is time-consuming and costly, and is not suitable for real-time intraoperative reconstruction. Although single-slice methods have high computational efficiency, the reconstruction results often lack accuracy due to the lack of stereoscopic depth perception, especially in complex structural areas such as sulci and gyri.

[0139] Based on the above, existing brain surface reconstruction methods have the following drawbacks: 1) Difficulty in balancing acquisition cost and reconstruction accuracy: Traditional methods require complete 3D MRI sequences, resulting in high acquisition costs and long acquisition times; while single-slice methods, although low-cost, lack sufficient reconstruction accuracy and cannot meet clinical needs. 2) Lack of effective multi-view information fusion: Existing methods fail to fully utilize the complementary information of different anatomical planes and ignore the potential spatial relationships between multi-view data, leading to poor reconstruction results. 3) Network architecture unsuitable for point cloud data: Traditional convolutional networks struggle to process non-Euclidean structure data such as point clouds, and cannot effectively capture the local geometric characteristics and global topological relationships of point clouds.

[0140] To address the aforementioned problems, this invention proposes a deep learning-based method for reconstructing brain surface point clouds. This method solves the technical issues of existing methods failing to fully utilize multi-view information and achieve low-cost, high-precision reconstruction. This invention fully leverages the complementary information of tri-dimensional orthogonal MRI slices to uncover potential spatial relationships between different anatomical planes. Furthermore, it processes point cloud data using a specially designed graph convolutional network to improve the accuracy and efficiency of brain surface point cloud reconstruction, thereby better meeting the clinical needs of neurosurgical planning and intraoperative navigation.

[0141] Please see Figure 9 , Figure 9 The reconstruction results of this method are compared with those of existing methods. It can be observed that the method shows better integrity in the topological structure of complex gyri in the frontal and occipital lobes, without obvious collapse or holes, and shows better geometric accuracy, especially in complex structural regions such as sulci and gyri.

[0142] Please see Figure 10 , Figure 10 The model's gradual reconstruction process at different training stages is demonstrated. As training iterations progress, the point cloud gradually transitions from sparse and blurry to a high-density and textured surface, fully illustrating the synergistic effect of the multi-view encoder and hierarchical graph convolutional decoder in capturing global-local geometric information.

[0143] In this embodiment of the invention, a method for reconstructing brain surface point clouds based on deep learning is provided. The method involves acquiring triaxial orthogonal MRI slice images of the brain to be reconstructed; inputting these images into a deep learning-based brain surface point cloud reconstruction model; using a multi-view encoder module to extract multi-scale features from the triaxial orthogonal MRI slice images to generate target multi-view features; performing cross-view correlation calculation and channel-spatial attention fusion on the target multi-view features through a cross-slice feature aggregation module to output target fused features; inputting these fused features into a 3DTransformer global context modeling module for global context modeling to output target global features; inputting these global features into a hierarchical graph convolutional network decoder module for hierarchical point cloud generation to output target hierarchical point cloud features; and using a point cloud generation head to reconstruct the target hierarchical point cloud features to output the target brain surface point cloud. Based on the above scheme, this invention utilizes a deep learning-based brain surface point cloud reconstruction model to process triaxial orthogonal MRI slice images, which can fully leverage multi-view MRI data and effectively uncover potential spatial relationships between different anatomical planes, thereby significantly improving reconstruction results and clinical applicability.

[0144] For better explanation, refer to Figure 11 The diagram illustrates the steps of the model training process for the deep learning-based brain surface point cloud reconstruction model provided in Embodiment 2 of the present invention. This process may include the following steps:

[0145] Step 1101: Obtain the training brain MRI slice image and the corresponding real brain surface point cloud data, perform data augmentation on the training brain MRI slice image, and output the target training brain MRI slice image.

[0146] The training dataset contains 1000 brain MRI samples, including MRI data from 650 patients with gliomas and 350 healthy controls. All MRI scans were preprocessed using FSL software and registered in 91×109×91 format. 800 samples were randomly selected to construct the training set, and the remaining 200 were used for the test set. For each sample, three orthogonal central slices were extracted as model input. Through precise surface reconstruction and uniform sampling, a brain surface point cloud containing 2048 points was constructed as the supervision signal.

[0147] Step 1102: Input the target training brain MRI slice image into the deep learning-based brain surface point cloud reconstruction model to be trained, and output the training point cloud reconstruction result.

[0148] Step 1103: Substitute the training point cloud reconstruction result and the real brain surface point cloud data into the multiple geometric loss function and calculate the derivative to output the model gradient.

[0149] Step 1104: Update the model parameters of the deep learning-based brain surface point cloud reconstruction model to be trained using the model gradient, output the intermediate deep learning-based brain surface point cloud reconstruction model, and count the number of model updates in real time.

[0150] Step 1105: Determine whether the number of model updates has reached the preset number of training iterations.

[0151] Step 1106: If so, use the intermediate deep learning-based brain surface point cloud reconstruction model as the trained deep learning-based brain surface point cloud reconstruction model.

[0152] It should be noted that the multiple geometric loss functions include Chamfer distance loss, improved Hausdorff distance loss, and normal consistency loss.

[0153] Chamfer distance loss is calculated using the following formula:

[0154]

[0155] in Let P represent the predicted point cloud and G represent the ground truth point cloud. Here, p represents any point in the predicted point cloud set P, and g represents any point in the ground truth point cloud set G. |P| and |G| represent the number of points in the predicted point cloud set and the ground truth point cloud set, respectively. Indicates Euclidean distance calculation;

[0156] The improved Hausdorff distance loss is calculated using the following formula:

[0157]

[0158] in This represents an improved Hausdorff distance loss function. Indicates from arrive The nearest distance is sorted in descending order and the first one is taken. indivual, Indicates from arrive The nearest distance is sorted in descending order and the first one is taken. indivual, This indicates the operation of calculating the average value;

[0159] Normal consistency loss is calculated using the following formula:

[0160]

[0161] in Represents the normal consistency loss function. and It is a point and The normal vector;

[0162] The overall loss function is:

[0163]

[0164] in This represents the overall loss function during model training. The values ​​are 1.0, 0.2, and 0.05, respectively.

[0165] The model was trained for a total of 300 epochs using the AdamW optimizer, with an initial learning rate of 100%. The weight decays to We employ a OneCycleLR learning rate scheduling strategy and use mixed-precision training and gradient checkpointing techniques to improve training efficiency.

[0166] In this embodiment of the invention, a multi-view encoder module, a cross-slice feature aggregation module, a 3DTransformer global context modeling module, and a hierarchical graph convolutional network decoder module are designed, and multiple geometric loss functions are introduced to enhance the model's feature extraction capabilities and improve model performance. Utilizing a trained deep learning-based brain surface point cloud reconstruction model for brain surface point cloud reconstruction can effectively capture local geometric information while accurately capturing global structural information, thereby reducing information loss during feature extraction and preserving more comprehensive and complete three-dimensional structural information. Furthermore, this invention is more aligned with actual clinical scenarios, making full use of limited intraoperative MRI data to provide a practical solution for neurosurgical planning and navigation.

[0167] This invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program; when the computer program is executed by the processor, the processor performs the steps of the deep learning-based brain surface point cloud reconstruction method as described in any of the above embodiments.

[0168] This invention also provides a computer-readable storage medium storing a computer program / instructions thereon, which, when executed by a processor, implement the steps of the deep learning-based brain surface point cloud reconstruction method as described in any of the above embodiments.

[0169] Although the present invention has been described above with reference to embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of the invention. In particular, as long as there is no structural conflict, the features in the disclosed embodiments can be combined with each other in any manner. The lack of an exhaustive description of these combinations in this specification is merely for the sake of brevity and resource conservation. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for reconstructing point clouds on the surface of the brain, characterized in that, Includes the following steps: S1: Obtain a sample set, which includes multiple pairs of tri-orthogonal MRI slice samples and point cloud labels. The point cloud labels are used to indicate the true surface feature point cloud of the corresponding tri-orthogonal MRI slice sample. S2: Construct a neural network framework, the framework including: The multi-view encoder consists of three parallel branches, each containing a multi-scale feature encoding module. It uses dilated convolutions with different dilation rates to capture multi-scale brain tissue structural features and generate target multi-view features. The cross-slice feature aggregation module includes a cross-view correlation calculation submodule and a channel-spatial attention mechanism, which are used to fuse complementary information from three directions; The 3D Transformer global context modeling module is used to perform global context modeling, establish long-range dependencies between multiple views, and output target global features. The hierarchical graph convolutional network decoder uses edge convolution and residual graph attention mechanisms to generate detailed point clouds of the brain surface layer by layer. S3: The neural network framework is trained using multiple geometric loss functions, including Chamfer distance, improved Hausdorff distance, and normal consistency loss. S4: Input the three-dimensional orthogonal MRI slices to be reconstructed into the trained framework, and output a brain surface point cloud containing 2048 points.

2. The brain surface point cloud reconstruction method according to claim 1, characterized in that, In S2, the multi-scale feature encoding module includes: The dilated convolution residual block contains multiple parallel dilated convolution branches with dilation rates of 1, 2, 3 and 6, forming a hybrid dilated pyramid structure. The multi-level feature extractor consists of three layers of dilated convolutional residual blocks with the number of channels being base_channels×2, base_channels×4, and base_channels×8, respectively, outputting a hierarchical structure of low-level, mid-level, and high-level features.

3. The brain surface point cloud reconstruction method according to claim 1, characterized in that, In S2, the processing procedure of the cross-view correlation calculation submodule is as follows: High-level features of three views ,in These represent the high-level semantic features obtained from sagittal, coronal, and horizontal MRI slices after feature encoding. To fully explore the complementary information between different views, the feature correlation between any two views is calculated using the following formula: in This represents the feature correlation matrix between the i-th view and the j-th view; The high-level semantic features corresponding to the i-th view in the table; This represents the high-level semantic features corresponding to the j-th view; i and j represent the view index, and their values ​​range from 1 to 10. ,and ; T represents the matrix transpose operation; Represents the normalization function; By mapping the features of one view to the feature space of another view using a correlation matrix, we obtain the features of the j-th view after cross-view correlation enhancement. : The original features and the relevance-enhanced features are fused to obtain the optimized view features. : in This represents the concatenation operation of two features along the channel dimension. This indicates a convolutional fusion operation, used for feature mapping and information integration of the concatenated features.

4. The brain surface point cloud reconstruction method according to claim 1, characterized in that, In S2, the channel-space attention mechanism includes: The channel attention submodule performs global average pooling on the stacked three-view features and generates channel weights through a multilayer perceptron. ,in Let F represent the features after channel attention processing, and let F represent the input feature tensor. For global average pooling, It is a multilayer perceptron. for Activation function; The spatial attention submodule uses 3D convolution operations to capture salient regions of features in the spatial dimension. ,in This represents the features after spatial attention enhancement. For 3D convolution operation, σ is Activation function.

5. The brain surface point cloud reconstruction method according to claim 1, characterized in that, In S2, the 3DTransformer global context modeling module includes: The position encoder flattens the fused multi-view feature tensor in the spatial dimension to generate a two-dimensional coordinate position embedding. A view token generator introduces a unique view token for each view as a representative of the view's global semantics; The multi-layer Transformer encoder, which includes multi-head self-attention and feedforward networks, captures complex dependencies between different views and spatial locations through a self-attention mechanism.

6. The brain surface point cloud reconstruction method according to claim 1, characterized in that, In S2, the hierarchical graph convolutional network decoder includes: The edge convolution module constructs a k-nearest neighbor graph and learns local geometric features by aggregating the feature differences between the center point and its neighbors: in This represents the local geometric features of the i-th point after edge convolution aggregation. Point of Nearest neighbor set Represents the difference feature. Implemented by a multilayer perceptron; The residual graph attention mechanism calculates the correlation between points through feature transformations of Query, Key, and Value: in This represents the attention weight matrix between point cloud nodes. This represents the output features of the residual graph attention module, where Q, K, and V are the query, key, and value matrices, respectively. For the number of channels, This represents a one-dimensional convolution operation. This indicates a modified linear unit activation function. Input features; The hierarchical upsampling structure consists of two levels that generate point clouds of 512 and 2048 points respectively.

7. The brain surface point cloud reconstruction method according to claim 1, characterized in that, In S3, the multiple geometric loss function includes: Chamfer distance loss measures the overall matching degree between the predicted point cloud and the target point cloud: ,in Let P represent the predicted point cloud and G represent the ground truth point cloud. Here, p represents any point in the predicted point cloud set P, and g represents any point in the ground truth point cloud set G. |P| and |G| represent the number of points in the predicted point cloud set and the ground truth point cloud set, respectively. Indicates Euclidean distance calculation; Improved Hausdorff distance loss, focusing on outlier regions and local salient features: ,in This represents an improved Hausdorff distance loss function. Indicates from arrive The nearest distance is sorted in descending order and the first one is taken. indivual, Indicates from arrive The nearest distance is sorted in descending order and the first one is taken. indivual, This indicates the operation of calculating the average value; Normal uniformity loss ensures surface smoothness and local geometric properties: ,in Represents the normal consistency loss function. and It is a point and The normal vector; Comprehensive loss function: ,in This represents the overall loss function during model training. The values ​​are 1.0, 0.2, and 0.05, respectively.

8. The brain surface point cloud reconstruction method according to claim 1, characterized in that, It also includes data preprocessing steps: Convert MRI data from DICOM format to tensor format; The center slices of the three orthogonal directions are extracted, normalized to the range of [-1, 1] and adjusted to a resolution of 224×224; A brain surface point cloud containing 2048 points was constructed as a supervision signal through precise surface reconstruction and uniform sampling; The coordinates are aligned to the standard coordinate system and normalized to the unit sphere using a rigid transformation.

9. An electronic device, characterized in that, The method includes a processor; and a memory communicatively connected to the processor, the memory storing computer-executable instructions, the processor executing the computer-executable instructions, which, when executed by the processor, implement the steps of the brain surface point cloud reconstruction method according to any one of claims 1 to 8.

10. A computer-readable storage medium storing computer-executable instructions, characterized in that, When executed by a processor, the computer-executable instructions implement the brain surface point cloud reconstruction method according to any one of claims 1 to 8.

Images