Magnetic resonance image classification method and system based on double-branch network

Abstract

The application discloses a kind of based on double branch network's magnetic resonance image classification method and system, it is related to medical image processing field, the method includes S1.to the magnetic resonance image data obtained is preprocessed and feature extraction obtains the feature data of sample;S2.to the feature data of sample is sequentially carried out invalid feature elimination, feature selection and feature scaling obtains standardization feature data;S3.based on historical magnetic resonance image data training double branch network's magnetic resonance image classification model;S4.combining the feature data of sample and magnetic resonance image classification model obtains magnetic resonance image classification result;The application is extracted from the same source data using double branch or multi-branch network processing and fusion multiple groups of heterogeneous features, with strong universality, can be applied in other image recognition and classification tasks based on multi-feature fusion.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical image processing, in particular to a magnetic resonance image classification method and system based on a double-branch network. BACKGROUND

[0002] Structural magnetic resonance imaging (sMRI) provides rich detailed information for observing brain structure. In the prior art, machine learning or deep learning methods are often used to analyze and classify sMRI images to distinguish brain images in different states.

[0003] For example, some traditional methods use machine learning models such as Support Vector Machine (SVM) to classify features extracted from the whole brain or specific brain regions (such as gray matter and white matter). The accuracy of these methods is limited by the effectiveness of feature extraction and the classification ability of the model. For cases where the differences between different categories of images are subtle and the differences within the same category of images (inter-individual differences) are large, the classification effect is often poor and the accuracy is limited.

[0004] Some other methods use deep learning models such as Convolutional Neural Network (CNN) to directly process two-dimensional or three-dimensional image data. Although deep learning can automatically extract features and improve accuracy to some extent, these methods also have some problems:

[0005] 1. Inadequate use of features: The model usually processes single-modal image data or features, and fails to effectively fuse and use multiple complementary structured feature data (such as subcortical segmentation features and cortical partition features) extracted from sMRI.

[0006] 2. Insensitivity to subtle differences: Traditional classifiers usually process each sample independently and are difficult to learn the relevance between different sample pairs, resulting in insufficient discrimination ability of the model when distinguishing between image categories with extremely subtle feature differences.

[0007] 3. Model complexity and generalization problem: Some deep learning models have complex structures and huge parameter quantities, which not only require a large amount of computing resources, but also are prone to overfitting, resulting in decreased generalization ability on new data sets.

[0008] Therefore, the prior art still has obvious technical bottlenecks and room for improvement in terms of how to efficiently fuse multiple brain structure features and improve the classification accuracy of subtle difference images. SUMMARY

[0009] The purpose of the present application is to provide a magnetic resonance image classification method and system based on a double-branch network. The method can effectively fuse two different sources of brain structure features, enhance the recognition ability of subtle differences between images through a specially designed network structure, and significantly improve the accuracy and robustness of image classification.

[0010] To achieve the above purpose, the present application provides the following solutions:

[0011] In a first aspect, the present application provides a magnetic resonance image classification method based on a double-branch network, comprising:

[0012] S1. Preprocessing and feature extraction are performed on the obtained magnetic resonance image data to obtain feature data of the sample;

[0013] S2. The feature data of the sample is sequentially subjected to invalid feature elimination, feature selection, and feature scaling to obtain standardized feature data;

[0014] S3. A magnetic resonance image classification model of a double-branch network is trained based on historical magnetic resonance image data;

[0015] S4. A magnetic resonance image classification result is obtained by combining the feature data of the sample and the magnetic resonance image classification model;

[0016] The magnetic resonance image classification model comprises:

[0017] an input layer, an adaptive feature selection layer, a first branch, a second branch, a dynamic branch weight layer, a feature fusion layer, and an output layer.

[0018] Further, the feature data of the sample further comprises:

[0019] generating a structure feature set based on the obtained brain structure magnetic resonance image scan data; the structure feature set comprises subcortical segmentation data and cortical partition data;

[0020] The subcortical region of the brain is segmented based on the brain structure to obtain the subcortical segmentation data;

[0021] The subcortical segmentation data includes a plurality of brain structures, and each brain structure includes a plurality of initial features;

[0022] The cortical partition data is obtained by partitioning the cerebral cortex based on the brain region;

[0023] The cortical partition data includes a plurality of brain region structures, and each brain region structure includes a plurality of initial features.

[0024] Further, the standardized feature data further comprises:

[0025] The invalid feature elimination is performed on the subcortical segmentation data and the cortical parcellation data respectively to obtain eliminated subcortical segmentation data and eliminated cortical parcellation data;

[0026] The correlation coefficients in the feature sets corresponding to the eliminated subcortical segmentation data and the eliminated cortical parcellation data are calculated respectively to generate a subcortical segmentation feature matrix and a cortical parcellation feature matrix;

[0027] The subcortical segmentation feature matrix and the cortical parcellation feature matrix are normalized to generate standardized feature data, including a subcortical normalization matrix and a cortical parcellation normalization matrix.

[0028] Further, the training of the magnetic resonance image classification model of the dual-branch network further includes:

[0029] The input layer includes two independent input ports for receiving the subcortical normalization matrix and the cortical parcellation normalization matrix respectively;

[0030] The adaptive feature selection layer includes a gating layer, a soft feature weighting layer, and a sparsity guidance and hard feature selection layer, and is used to generate an optimized subcortical segmentation feature combination and a cortical parcellation feature combination respectively;

[0031] The first branch includes one or more fully connected layers for receiving and processing the subcortical segmentation feature combination;

[0032] The second branch includes one or more fully connected layers for receiving and processing the cortical parcellation feature combination;

[0033] The dynamic branch weight layer includes a branch weight parameterization structure, a weight normalization structure, and a weighted feature fusion structure, and is used to obtain a fusion feature vector of the first branch and a fusion feature vector of the second branch;

[0034] The feature fusion layer is used to splice the output feature vectors of the first branch and the second branch to obtain a final fusion splicing vector;

[0035] The output layer is used to generate a prediction probability of each class based on the final fusion splicing vector.

[0036] Further, the adaptive feature selection layer further includes:

[0037] The gating layer generates corresponding trainable gating parameters based on the feature dimensions of the input branches ;

[0038] The soft feature weighting layer converts the gating parameter vector g into a soft gating weight vector by using a Sigmoid activation function, and the soft gating weight vector the original input feature vector element-wise multiplication to obtain the weighted feature vector ;

[0039] The sparsity guidance and hard feature selection layer applies a regularization penalty to the gating parameter vector g to obtain the optimized feature vector.

[0040] Further, the sparsity guidance and hard feature selection layer further comprises:

[0041] wherein the overall loss function is represented as:

[0042] ;

[0043] wherein, is a classification cross-entropy loss function; is a regularization coefficient; is the i-th parameter in the gating vector.

[0044] Further, the dynamic branch weight layer further comprises:

[0045] The branch weight parameterization layer initializes a trainable scalar weight parameter for each feature processing branch, including: the first branch assigns a weight , and the second branch assigns a weight ;

[0046] The weight normalization layer uses the SoftMax function to normalize the weight parameters of all branches to obtain normalized first branch assignment weight and second branch assignment weight;

[0047] The weighted feature fusion layer weights the normalized weights based on the obtained features vectors output by the branches corresponding to them, and then sends them into the feature fusion layer for splicing to obtain the first branch fusion feature vector and the second branch fusion feature vector.

[0048] Further, the feature fusion layer further comprises:

[0049] The normalized weights are weighted with the feature vectors output by the branches corresponding to them, and splicing is performed based on the feature fusion layer, as follows:

[0050] ;

[0051] wherein, denotes the feature fusion layer splicing operation; are the feature vectors output by the first branch and the second branch, respectively; The normalized weights of the first branch and the second branch are respectively: The final fusion feature vector is obtained by splicing after dynamic weighting.

[0052] Further, the combination of the feature data of the sample and the magnetic resonance image classification model obtains a magnetic resonance image classification result, and further comprises:

[0053] Based on the polynomial attenuation strategy, the trained double-branch network model is dynamically pruned to obtain a final double-branch network model reaching a target sparsity, and the feature data of the sample is detected based on the final double-branch network model to generate a magnetic resonance image classification result.

[0054] The magnetic resonance image classification system based on the double-branch network applies the magnetic resonance image classification method based on the double-branch network described in any one of the above, and comprises:

[0055] The data acquisition module is used for sequentially pre-processing the acquired magnetic resonance image data and outputting the feature data of the sample.

[0056] The data processing module is used for sequentially performing invalid feature elimination, feature selection and feature scaling on the input feature data of the sample to output standardized feature data.

[0057] The model training module inputs the historical magnetic resonance image data for training the double-branch network magnetic resonance image classification model.

[0058] The output module is used for combining the input feature data of the sample and the magnetic resonance image classification model to output a magnetic resonance image classification result.

[0059] According to the magnetic resonance image classification method and system based on the double-branch network provided by the present application, the following technical effects are achieved:

[0060] 1. The classification accuracy is significantly improved: the double-branch network structure proposed by the present application can effectively fuse the complementary feature information of subcortical and cortical features. Experiments show that the accuracy of this method in three classification tasks can reach 93%, which is significantly higher than the method of using only single feature or traditional classification model.

[0061] 2. The recognition ability of subtle differences is stronger: by learning the representation of different feature spaces through two independent branches respectively, and then performing fusion, the model can capture more subtle and complex structural differences between different categories of images, thereby improving the sensitivity and robustness of classification.

[0062] 3. End-to-end adaptive feature selection is achieved: By introducing an adaptive feature selection layer, the feature screening process is seamlessly integrated into network training, achieving end-to-end optimization. Compared with static methods that rely on Pearson correlation coefficients for preprocessing, this method enables the model to automatically learn the importance of features based on task loss, discover better feature combinations, and further improve the intelligence level and performance ceiling of the model. At the same time, the gating weights obtained after training also greatly enhance the model's interpretability.

[0063] 4. The model is efficient and practical: The technical solution of the present application can further combine network pruning (Pruning) technology. Specifically, by introducing a progressive pruning strategy based on weight magnitude and fine-tuning the model, the model's parameter and computational complexity can be reduced by about 60% while maintaining a high classification accuracy of, for example, 93%. This greatly reduces the model's demand for computing resources, making it easier to deploy and apply in conventional hardware environments, and has strong practical value.

[0064] 5. The method is versatile: The core idea of the present application, i.e., using a dual-branch (or multi-branch) network to process and fuse multiple heterogeneous features extracted from the same source data (sMRI), has strong versatility and can be applied to other image recognition and classification tasks based on multi-feature fusion. BRIEF DESCRIPTION OF DRAWINGS

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0066] Figure 1 A flowchart of a magnetic resonance image classification method based on a dual-branch network according to an embodiment of the present application is shown in FIG. 1.

[0067] Figure 2 A block diagram of a magnetic resonance image classification system structure according to an embodiment of the present application is shown in FIG. 2.

[0068] Figure 3 A data preprocessing process diagram according to an embodiment of the present application is shown in FIG. 3.

[0069] Figure 4 A feature selection and feature engineering diagram according to an embodiment of the present application is shown in FIG. 4.

[0070] Figure 5 A dual-branch neural network structure diagram according to an embodiment of the present application is shown in FIG. 5. DETAILED DESCRIPTION

[0071] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0072] The above purposes, features and advantages of the present application will be more apparent and understandable. The present application will be further described in detail below with reference to the drawings and specific embodiments.

[0073] Embodiment one, specific implementation of the dual-branch network model:

[0074] As shown in the figure, the embodiment of the present application provides a magnetic resonance image classification method based on a dual-branch network, which comprises: Figure 1

[0075] S1. Preprocessing and feature extraction are performed on the acquired magnetic resonance image data to obtain feature data of the sample;

[0076] S2. The feature data of the sample is sequentially subjected to invalid feature elimination, feature selection and feature scaling to obtain standardized feature data;

[0077] S3. Training a magnetic resonance image classification model of the dual-branch network based on historical magnetic resonance image data;

[0078] S4. Obtaining a magnetic resonance image classification result in combination with the feature data of the sample and the magnetic resonance image classification model;

[0079] The magnetic resonance image classification model comprises:

[0080] an input layer, an adaptive feature selection layer, a first branch, a second branch, a dynamic branch weight layer, a feature fusion layer and an output layer.

[0081] As shown in the figure, the data preprocessing process is shown, which includes downloading structural magnetic resonance sMRI images from the ADNI database to obtain original sMRI scan images, processing the original sMRI scan images by automatic brain analysis software to obtain structured statistical data of subcortical segmentation and cortical parcellation, respectively. Figure 3

[0082] Figure 4 ​​The feature engineering process is shown. Correlation analysis (PCC matrices) is performed on two sets of statistics, core features are selected, and two reduced dimension feature matrices (e.g., 39x4 and 68x4) are finally generated and rescaled.

[0083] Optionally, the obtained sample feature data further comprises:

[0084] Based on the obtained brain structure magnetic resonance image scanning data, a structure feature set is generated; the structure feature set comprises subcortical segmentation data and cortical parcellation data;

[0085] The subcortical segmentation data is obtained by segmenting the subcortical region of the brain based on the brain structure.

[0086] The subcortical segmentation data includes a plurality of brain structures, and each brain structure includes a plurality of initial features.

[0087] The cortical parcellation data is obtained by partitioning the cerebral cortex based on the brain region.

[0088] The cortical parcellation data includes a plurality of brain region structures, and each brain region structure includes a plurality of initial features.

[0089] Specifically, data preprocessing and feature extraction;

[0090] 1. Data acquisition: Obtain a batch of brain structure magnetic resonance image (sMRI) scanning data (e.g., NIFTI format).

[0091] 2. Structured feature generation: Use an automated brain analysis software (such as FreeSurfer) to process each sMRI image. This processing process mainly generates two types of structured feature data:

[0092] Subcortical segmentation data (SS): Segment the subcortical region of the brain to generate a set of numerical features for about 45 brain structures (such as hippocampus, amygdala, thalamus, etc.). For example, each structure includes 7 initial features: voxel number, volume, normalized average voxel intensity, normalized voxel intensity standard deviation, etc.

[0093] Cortical parcellation data (CP): Partition the cerebral cortex to generate a set of numerical features for each of the 34 brain regions of each hemisphere. For example, each brain region includes 9 initial features: vertex number, surface area, gray matter volume, average thickness, thickness standard deviation, etc.

[0094] Optionally, the obtained standardized feature data further comprises:

[0095] The subcortical segmentation data and the cortical parcellation data are respectively subjected to invalid feature elimination to obtain eliminated subcortical segmentation data and eliminated cortical parcellation data.

[0096] Specifically, the invalid feature elimination includes: analyzing the subcortical segmentation data and the cortical parcellation data of all samples, eliminating brain structures (such as WM-Hypointensities) whose values are always zero or invalid in all samples, so as to simplify the feature set. For example, the structures in the subcortical segmentation data are reduced from 45 to 39.

[0097] The correlation coefficients in the feature sets corresponding to the eliminated subcortical segmentation data and the eliminated cortical parcellation data are respectively calculated to generate a subcortical segmentation feature matrix and a cortical parcellation feature matrix.

[0098] Specifically, the feature selection includes: to reduce data redundancy and retain core information, the eliminated subcortical segmentation data and the eliminated cortical parcellation data are respectively subjected to correlation analysis.

[0099] Pearson correlation coefficients (PCC) between each feature column are calculated to generate a feature correlation heat map.

[0100] From the highly correlated feature group, a representative feature is retained, and the remaining redundant features are eliminated. For example, 4 core features are finally retained for each brain structure or brain region. The double-branch neural network structure is as shown in Figure 5 After this step, each sMRI image sample is finally represented as two independent feature matrices: a subcortical segmentation feature matrix (for example, with a dimension of 39x4) and a cortical parcellation feature matrix (for example, with a dimension of 68x4).

[0101] The subcortical segmentation feature matrix and the cortical parcellation feature matrix are subjected to normalization processing to generate standardized feature data, including a subcortical normalization matrix and a cortical parcellation normalization matrix.

[0102] Specifically, the feature scaling includes: all values in the two filtered feature matrices are scaled to the interval [0, 1] by using the Min-Max Normalization method, so as to facilitate the processing of the neural network model.

[0103] Optionally, the training of the magnetic resonance image classification model of the double-branch network further includes:

[0104] The input layer includes two independent input ports, respectively receiving a subcortical normalization matrix with a dimension of (39, 4) and a cortical partition normalization matrix with a dimension of (68, 4). After input, the data is flattened into one-dimensional vectors with dimensions of 156 (39x4) and 272 (68x4), respectively;

[0105] The adaptive feature selection layer includes: in two branches, the flattened one-dimensional vectors are respectively sent into independent AFS layers. The layer realizes end-to-end feature selection by introducing trainable gating parameters , and applying an L1 regularization penalty to them. The regularization coefficient λ of the L1 regularization penalty can be in the range of 0.001 to 0.1, and in the embodiment, 0.01 is preferred;

[0106] The first branch includes one or more fully connected layers for receiving and processing the subcortical segmentation feature combination;

[0107] Specifically, the first branch: receives and processes the subcortical segmentation feature matrix (after the AFS layer screening). The branch is composed of one or more fully connected layers. For example, it can contain two consecutive fully connected layers, each with 128 neurons, and the activation function uses a rectified linear unit.

[0108] The second branch includes one or more fully connected layers for receiving and processing the cortical partition feature combination;

[0109] Specifically, the second branch: receives and processes the cortical partition feature matrix (after the AFS layer screening). After each fully connected layer, a Dropout layer is set, with a neuron dropout ratio (dropout ratio) set to 0.5 to enhance the generalization ability of the model and prevent overfitting. The weights (parameters) of the two branches are independently trained and not shared.

[0110] The dynamic branch weight layer includes: a branch weight parameterization structure, a weight normalization structure, and a weighted feature fusion structure; for obtaining the fusion feature vector of the first branch and the fusion feature vector of the second branch. The layer learns a trainable scalar weight and for the first branch and the second branch respectively, and uses a SoftMax function to normalize it to and , for dynamically adjusting the contribution of the two branches to the final prediction;

[0111] The feature fusion layer concatenates the output feature vectors (both with a dimension of 128) of the two branches after dynamic weighting to form a final fusion feature vector with a dimension of 256;

[0112] An output layer for generating prediction probabilities of each class based on the final fused concatenation vector.

[0113] Specifically, the output layer: input the 256-dimensional fusion feature vector into a final fully connected layer. This layer contains 3 neurons (corresponding to three classification categories, such as AD, MCI, CN), and uses the SoftMax activation function. The SoftMax layer outputs the prediction probability of each class.

[0114] Optionally, the adaptive feature selection layer further comprises:

[0115] A gating layer for generating corresponding trainable gating parameters based on the feature dimensions of the input branch ;

[0116] A soft feature weighting layer for converting the gating parameter vector g into a soft gating weight vector by using a Sigmoid activation function , and obtaining a weighted feature vector by element-wise multiplication of the soft gating weight vector and the original input feature vector ;

[0117] A sparsity guidance and hard feature selection layer for modifying the gating parameter vector g by applying a regularization penalty to the gating parameter based on the overall loss function to obtain an optimized feature vector.

[0118] Specifically, the adaptive feature selection layer (Adaptive Feature Selector, AFS) aims to solve the technical problems:

[0119] This layer aims to solve the problem of insufficient feature utilization in the background art and overcome the limitations of traditional feature engineering (such as screening methods based on Pearson correlation coefficients). Traditional methods belong to static screening, which cannot be optimized during model training after one-time feature removal, which may lead to suboptimal feature combinations. This layer introduces a trainable gating mechanism to integrate the feature selection process into the end-to-end training of the network model, achieving dynamic and adaptive feature selection.

[0120] Specific implementation: This layer is placed before the input data of each branch enters the fully connected layer, and its core operation is as follows:

[0121] Gating parameter generation: for each feature dimension of the input branch, this layer assigns a trainable gating parameter During training, the model learns these parameters.

[0122] Soft feature weighting: In the model training stage, to ensure that the gradient can be effectively back-propagated, the gating parameter vector g is converted into a soft gating weight vector by a Sigmoid activation function , whose value of each element is between (0, 1). The weight vector is multiplied element by element with the original input feature vector , so as to dynamically weight each feature. The mathematical representation is as follows:

[0123] ;

[0124] ;

[0125] wherein, is the input feature vector of the branch; g is a trainable gating parameter vector with the same dimension as ; is a Sigmoid function; is the generated soft weight vector; and is an element-wise product; is the weighted feature vector.

[0126] Sparse guidance and hard feature selection: To realize automatic feature screening, a regularization penalty is imposed on the gating parameter vector g in the loss function of the model training. The regularization term drives the gating parameter vector g corresponding to the features irrelevant or redundant to the classification task to tend to zero. The overall loss function is represented as:

[0127] ;

[0128] wherein, is a cross-entropy loss function for classification; is a regularization coefficient, which is a hyperparameter; is the i-th parameter in the gating vector. In the model inference stage, the soft weight can be binaryzed to 0 or 1 according to a preset threshold (such as 0.5) to form a hard feature selection mask, so as to explicitly exclude invalid features and enhance the explainability of the model.

[0129] Technical effects: Through the above design, end-to-end adaptive feature selection is realized. The model no longer relies on static preprocessing results, but can automatically learn the optimal feature combination according to the final classification task loss, further improving the classification performance. At the same time, after the training is completed, the size of the gating parameter directly reflects the importance of each feature, greatly enhancing the explainability of the model.

[0130] Optionally, the dynamic branch weight layer further comprises:

[0131] The branch weight parameterization layer initializes a trainable scalar weight parameter for each feature processing branch, including: assigning weights to the first branch. Weight allocation for the second branch ;

[0132] The weight normalization layer uses the SoftMax function to normalize the weight parameters of all branches to obtain the normalized weights assigned to the first and second branches.

[0133] The weighted feature fusion layer weights the obtained normalized weights with the feature vectors output by their respective branches, and then feeds them into the feature fusion layer to concatenate them to obtain the fused feature vectors of the first branch and the fused feature vectors of the second branch.

[0134] Specifically, the Dynamic Branch Weighting (DBW) layer addresses the technical problem that different feature sets (subcortical segmentation features and cortical partitioning features) may contribute differently to the final classification task. Traditional feature fusion methods (such as direct concatenation) assume that the two branches contribute equally, which may not be the optimal strategy. This layer introduces a weight self-learning mechanism, allowing the model to automatically assign appropriate weights to different branches based on the data and the task itself, thereby more intelligently fusing multi-source information.

[0135] Implementation details: This layer is placed before the output features of the two branches are concatenated and fused. Its core operations are as follows:

[0136] Branch weight parameterization: Initialize a trainable scalar weight parameter for each feature processing branch (there are two branches in this example). For example, assign weights to the first branch (processing subcortical segmentation features). Assign weights to the second branch (which processes cortical partition features). .

[0137] Weight normalization: During forward propagation, the SoftMax function is used to normalize the weight parameters of all branches, resulting in a set of "contribution" weights that sum to 1. This ensures that the total contribution of all branches is constant, and the weight values ​​can be intuitively understood as the confidence level of each branch in the final decision. Its mathematical representation is:

[0138] ;

[0139] in:

[0140] ;

[0141] In the formula, These are trainable scalar parameters; are the normalized weights of the first branch and the second branch, respectively, and

[0142] Weighted feature fusion: the normalized weights are respectively weighted with the feature vectors output by the branches, and then sent to the feature fusion layer for splicing.

[0143] Optionally, the feature fusion layer further comprises:

[0144] The normalized weights are respectively weighted with the feature vectors output by the branches, and spliced based on the feature fusion layer, the process being as follows:

[0145]

[0146] In the formula, denotes the splicing operation of the feature fusion layer; are the feature vectors output by the first branch and the second branch, respectively; are the normalized weights of the first branch and the second branch, respectively; is the final fused feature vector after dynamic weighting and splicing.

[0147] Specifically, the feature fusion layer: splices (Concatenate) the output feature vectors of the first branch and the second branch to form a longer and more informative combined feature vector. For example, two 128-dimensional vectors are spliced into a 256-dimensional vector. Technical effect: through this end-to-end learning method, the model can automatically amplify the feature influence of the branch more critical to the classification task, while suppressing the features of the secondary or noise branch. This makes the feature fusion process more intelligent and efficient, thereby improving the decision-making accuracy and robustness of the entire model.

[0148] 3. Model training and optimization:

[0149] The Adam optimizer is used to train the entire network end-to-end. The initial learning rate is set to 0.001, and its value range can be 0.0001 to 0.01. The batch size during training is set to 32, and a total of 100 cycles are trained. The classification cross-entropy is used as the loss function.

[0150] Optionally, the feature data of the combined sample and the magnetic resonance image classification model obtain a magnetic resonance image classification result, further comprising:

[0151] The trained double-branch network model is applied with network pruning technology to reduce the parameter size and computational complexity of the model without significantly reducing the classification accuracy.

[0152] ​​The trained double-branch network model is pruned dynamically based on a polynomial decay strategy to obtain a final double-branch network model reaching a target sparsity, and the feature data of the sample are detected based on the final double-branch network model to generate a magnetic resonance image classification result.

[0153] In a specific implementation, the pruning method based on weight magnitude is adopted, which removes the weights with small absolute values in the model to achieve the effects of model sparsification and compression. The core steps are as follows:

[0154] 1. Defining pruning strategy and parameters: The prune_low_magnitude API in the TensorFlow Model Optimization (tfmot) toolkit is adopted. In order to smoothly transition and maintain the model performance during the pruning process, a dynamic pruning schedule is set. Specifically, the polynomial decay strategy is adopted to gradually increase the sparsity of the model during the training process. For example, the initial sparsity can be set to 50%, and the target sparsity of 80% is reached within a specified number of training steps (for example, 3900 steps).

[0155] 2. Model packaging and recompilation: The trained double-branch network model is packaged using the pruning wrapper. This operation adds a pruning mask to the layers that can be pruned, such as the fully connected layer of the model, to determine which weights should be set to zero during training. After packaging, the model needs to be recompiled using an optimizer such as Adam and a classification cross-entropy loss function.

[0156] 3. Pruning and model fine-tuning: The packaged model is trained additionally. During the fine-tuning process, a key callback function Update Pruning Step is added. This callback function updates the pruning state after each training batch, gradually increasing the proportion of weights that are pruned (i.e., set to zero) according to the pre-set polynomial decay strategy. Through this progressive approach, the model has the ability to adapt to changes in network structure, recovering the accuracy loss caused by pruning through learning while weights are removed.

[0157] 4. Final model export: After fine-tuning, part of the weights in the model have been permanently set to zero, forming a sparse model. At this time, the model with the pruning mask can be exported, and the parameter quantity and computational complexity of the model are significantly reduced compared to the original model.

[0158] Through the above steps, the application can effectively compress the model while maintaining high classification accuracy, making it easier to deploy in resource-constrained environments.

[0159] Further, the Adam optimizer is used to train the entire network end-to-end with the classification cross-entropy as the loss function. After training, the new image feature matrix is input, and the model can output its most likely class attribution.

[0160] Figure 5 The core of the application is shown in the dual-branch neural network model structure. Two feature matrices are input into two independent branches, each branch containing a fully connected layer and a Dropout layer. The outputs of the two branches are spliced, and finally an output layer is used to obtain the final classification prediction result.

[0161] As shown in Figure 2 In one exemplary embodiment, the dual-branch network-based magnetic resonance image classification system applies any of the above dual-branch network-based magnetic resonance image classification methods, including:

[0162] The data acquisition module is used to sequentially pre-process the acquired magnetic resonance image data and output the feature data of the sample.

[0163] The data processing module is used to sequentially perform invalid feature elimination, feature selection, and feature scaling on the input sample feature data to output standardized feature data.

[0164] The model training module inputs historical magnetic resonance image data to train the dual-branch network-based magnetic resonance image classification model.

[0165] The output module is used to combine the input sample feature data and the magnetic resonance image classification model to output the magnetic resonance image classification result.

[0166] Embodiment two, the application of the extension to a multi-branch network model includes:

[0167] The method proposed in the application has strong universality and can be easily applied to scenarios that fuse more than two sets of heterogeneous features. The following is an example of a three-branch network model to illustrate the extension application of the method.

[0168] In this embodiment, in addition to the subcortical segmentation features and the cortical parcellation features extracted from sMRI data in Embodiment One, we introduce a third set of feature data: white matter fiber bundle microstructure features extracted from diffusion tensor imaging (DTI) data. For example, using software to process the DTI image, we obtain statistical features of 48 major white matter fiber bundles (such as the corpus callosum, the cingulum bundle, etc.) in the whole brain, each fiber bundle containing 3 indicators: fractional anisotropy (FA), mean diffusivity (MD), and volume. In this way, each sample obtains an additional feature matrix with a dimension of (48, 3).

[0169] Based on these three sets of heterogeneous feature data, the following three-branch network model can be constructed:

[0170] 1. Input layer: The model is expanded into three independent input ports, which respectively receive the subcortical segmentation feature matrix, the cortical parcellation feature matrix, and the newly added white matter fiber bundle feature matrix.

[0171] 2. Network branches: The structures of the first branch and the second branch remain the same as in Embodiment One, which respectively process the subcortical and cortical features. A third branch is newly added to process the white matter fiber bundle features. This branch is also composed of an independent adaptive feature selection layer (AFS), two fully connected layers each containing 128 neurons, and corresponding Dropout layers. The parameters of the third branch are also independently trained.

[0172] 3. Dynamic branch weight layer (DBW): The function of this layer is correspondingly expanded, initializing a trainable scalar weight for each of the three branches . In forward propagation, through the SoftMax function, the dynamic weights of the three branches are obtained , and .

[0173] 4. Feature fusion layer: This layer concatenates the output feature vectors of the three branches after dynamic weighting to form the final fusion feature vector.

[0174] 5. Output layer and training: The structure and training process of the output layer are similar to those in Embodiment One, which receives a fusion vector with a dimension of 384 and outputs the final classification probability through the SoftMax layer.

[0175] Through this embodiment, it can be seen that the core architecture of the present application can be flexibly expanded from a two-branch to a multi-branch to fuse more complementary information from different sources, further improving the classification performance of the model, and embodying its strong versatility and scalability.

[0176] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0177] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of each method can be included. In the embodiments provided in the present application, any reference to memory, database or other medium can include at least one of non-volatile and volatile memory. Non-volatile memory can include Read Only Memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, Resistive Random Access Memory (ReRAM), Magnetoresistive Random Access Memory (MRAM), Ferroelectric Random Access Memory (FRAM), Phase Change Memory (PCM), graphene memory, etc. Volatile memory can include Random Access Memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), etc.

[0178] The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a blockchain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0179] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, however, as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present application.​

[0180] The principles and implementations of the present application are described in detail with specific examples in this paper, and the above examples are only used to help understand the method and its core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation and application range will be changed. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A magnetic resonance imaging classification method based on a dual-branch network, characterized in that, The method includes: S1. Preprocess and extract features from the acquired magnetic resonance imaging data to obtain the feature data of the sample; S2. Perform invalid feature removal, feature selection, and feature scaling on the sample's feature data in sequence to obtain standardized feature data; S3. A magnetic resonance image classification model with a dual-branch network trained based on historical magnetic resonance image data; S4. Combine the feature data of the samples with the magnetic resonance imaging classification model to obtain the magnetic resonance imaging classification results; The magnetic resonance imaging classification models include: The system consists of an input layer, an adaptive feature selection layer, a first branch, a second branch, a dynamic branch weight layer, a feature fusion layer, and an output layer. The obtained feature data of the sample also includes: A structural feature set is generated based on the acquired brain structural magnetic resonance imaging data; the structural feature set includes: subcortical segmentation data and cortical partition data; Among them, subcortical segmentation data is obtained by segmenting the subcortical region of the brain based on brain structure; The subcortical segmentation data includes multiple brain structures, and each brain structure includes multiple initial features; Cortical partition data is obtained by dividing the cerebral cortex into regions based on brain areas; The cortical partition data includes multiple brain region structures, and each brain region structure contains multiple initial features; The magnetic resonance imaging classification model for training a dual-branch network also includes: The input layer includes two independent input ports, used to receive the subcortical normalization matrix and the cortical partition normalization matrix, respectively. The adaptive feature selection layer includes a gating layer, a soft feature weighting layer, and a sparsity-guided and hard feature selection layer; used to generate optimized subcortical segmentation feature combinations and cortical partition feature combinations, respectively. The first branch includes: one or more fully connected layers for receiving and processing combinations of subcortical segmentation features; The second branch includes one or more fully connected layers for receiving and processing combinations of cortical partition features; The dynamic branch weight layer includes: a branch weight parameterization structure, a weight normalization structure, and a weighted feature fusion structure; used to obtain the fused feature vector of the first branch and the fused feature vector of the second branch; The feature fusion layer is used to concatenate the output feature vectors of the first branch and the second branch to obtain the final fused concatenated vector; The output layer is used to generate the predicted probability for each category based on the final fused spliced ​​vector.

2. The magnetic resonance image classification method based on a dual-branch network according to claim 1, characterized in that, The process of obtaining standardized feature data also includes: The subcortical segmentation data and the cortical partition data were obtained by removing invalid features from the subcortical segmentation data and the cortical partition data, respectively. The correlation coefficients of the feature sets corresponding to the removed subcortical segmentation data and the removed cortical partition data are calculated to generate the subcortical segmentation feature matrix and the cortical partition feature matrix, respectively. Standardized feature data is generated by normalizing the subcortical segmentation feature matrix and the cortical partition feature matrix, including: subcortical normalized matrix and cortical partition normalized matrix.

3. The magnetic resonance image classification method based on a dual-branch network according to claim 1, characterized in that, The adaptive feature selection layer includes: The gating layer generates trainable gating parameters based on the feature dimensions of the input branches. ; The soft feature weighting layer transforms the gating parameter vector g into a soft-gating weight vector using the sigmoid activation function. By using soft-gated weight vectors Compared with the original input feature vector Element-wise multiplication yields the weighted feature vector. ; The sparsity-guided and hard feature selection layer applies a global loss function to the gating parameter vector g. Regularization penalty on trainable gating parameters The optimized feature vector is obtained by making corrections.

4. The magnetic resonance image classification method based on a dual-branch network according to claim 3, characterized in that, The sparse guidance and hard feature selection layer also include: The overall loss function is expressed as: ； In the formula, The cross-entropy loss function is used for classification. The regularization coefficient is used. It is the i-th parameter in the gating vector.

5. The magnetic resonance image classification method based on a dual-branch network according to claim 3, characterized in that, The dynamic branch weight layer also includes: The branch weight parameterization layer initializes a trainable scalar weight parameter for each feature processing branch, including: assigning weights to the first branch. Weight allocation for the second branch ; The weight normalization layer uses the SoftMax function to normalize the weight parameters of all branches to obtain the normalized weights assigned to the first and second branches. The weighted feature fusion layer weights the obtained normalized weights with the feature vectors output by their respective branches, and then feeds them into the feature fusion layer to concatenate them to obtain the fused feature vectors of the first branch and the fused feature vectors of the second branch.

6. The magnetic resonance image classification method based on a dual-branch network according to claim 3, characterized in that, The feature fusion layer further includes: The normalized weights are weighted by the feature vectors output from their respective branches, and then concatenated based on the feature fusion layer, as shown in the following equation: ； In the formula, This indicates the feature fusion layer splicing operation; These are the feature vectors output by the first branch and the second branch, respectively. These are the normalized weights of the first and second branches, respectively. It is the final fused feature vector after dynamic weighting and splicing.

7. The magnetic resonance image classification method based on a dual-branch network according to claim 1, characterized in that, The process of combining the feature data of the samples with the magnetic resonance image classification model to obtain the magnetic resonance image classification result also includes: The trained bi-branch network model is dynamically pruned using a multinomial decay strategy to obtain a final bi-branch network model that achieves the target sparsity. Based on the final bi-branch network model, the feature data of the samples are detected to generate magnetic resonance image classification results.

8. A magnetic resonance imaging classification system based on a dual-branch network, characterized in that, The magnetic resonance image classification method based on a dual-branch network according to any one of claims 1-7 includes: The data acquisition module is used to preprocess the acquired magnetic resonance imaging data and extract features from the output sample feature data. The data processing module sequentially performs invalid feature removal, feature selection, and feature scaling on the feature data of the input samples to output standardized feature data. The model training module takes historical magnetic resonance imaging data as input to train a dual-branch network magnetic resonance imaging classification model. The output module is used to combine the feature data of the input samples with the magnetic resonance image classification model to output the magnetic resonance image classification result.

Images