Brain MRI image classification method based on double-branch hierarchical contrast learning

Abstract

The invention discloses a brain MRI image classification method based on double-branch hierarchical contrast learning, and the method comprises the following steps: 1, obtaining brain MRI image data, constructing a Transform double-branch structure model, and carrying out the feature extraction of the MRI image data of two visual angles; 2, according to MRI image features of two visual angles, on one hand, fusing the two features in a channel dimension for classified learning, and on the other hand, respectively projecting the two features to a low-dimensional space of the same dimension for comparative learning; and 3, designing a hierarchical comparison loss function which is used for constraining similarity ranking between the positive sample pairs and the negative sample pairs with different difficulties, and updating model parameters through combined action of the hierarchical comparison loss function and the classification target loss function so as to realize classification of the MRI images. Through understanding of the constraint model on data structure consistency and semantic consistency in the process of learning and predicting the MRI image, the discrimination of the features extracted by the model is remarkably enhanced, so that a more accurate reasoning result is generated.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence deep learning brain MRI medical image classification technology, and in particular to a brain MRI image classification method based on dual-branch hierarchical contrastive learning. Background Technology

[0002] Magnetic resonance imaging (MRI) is a medical imaging technique that uses magnetic fields, radio frequency pulses, and computer technology to generate detailed images of human structures. Currently, many studies are using artificial intelligence algorithms to analyze MRI data in order to develop non-invasive (preoperative) and precise methods for detecting brain abnormalities.

[0003] In recent years, based on the success of deep neural networks (DNNs) in the field of natural images, many studies have aimed to transfer their pre-trained knowledge through supervised learning, demonstrating the ability to capture complex patterns and features in medical images. Unlike natural images, the acquisition of brain MRI images relies on specialized and expensive scanning equipment, making it difficult to collect abundant data on rare brain diseases. In MRI, system noise and slight motion artifacts are unavoidable, leading to random fluctuations in grayscale values ​​that may be incorrectly used as the basis for computer-aided diagnosis. Furthermore, the high similarity of brain scans among different subjects complicates the detection of subtle changes in brain structure. Therefore, supervised learning methods may lead to overfitting of DNNs, causing them to memorize noise rather than understand disease patterns during feature learning.

[0004] One effective solution is multi-task learning, which incorporates textual modal information such as patient age, gender, clinicopathological features, and radiological subtypes of lesions during DNN training. However, due to privacy concerns and high annotation costs, this additional information is often unavailable. A more intuitive solution is to optimize the learning process of DNNs for single image modal features, enhancing the model's explicit constraints on "feature invariance." However, traditional contrastive learning only utilizes the different intensity transformation features of a single image viewpoint for semantic comparison, leading to the neglect of structural consistency learning and an overemphasis on semantic consistency learning. Furthermore, traditional contrastive learning uses a binary partition of "positive for similar, negative for dissimilar," failing to consider the difficulty levels of negative samples, resulting in insufficient discrimination of nearest-neighbor dissimilar samples and limiting the accuracy of semantic consistency learning. Summary of the Invention

[0005] This invention addresses the shortcomings of existing technologies by proposing a brain MRI image classification method based on dual-branch hierarchical contrastive learning. The aim is to significantly enhance the discriminative power of the features extracted by the model by constraining the model's understanding of structural and semantic consistency during the learning and prediction of MRI images, thereby achieving more accurate MRI image recognition and reasoning.

[0006] To achieve the above-mentioned objectives, the present invention adopts the following technical solution: The present invention provides a brain MRI image classification method based on dual-branch hierarchical contrastive learning, characterized by the following steps: Step S1, obtain the contents Types Brain MRI image dataset ,in, Indicates the first The first of the categories MRI images, and , Indicates the first The first of the categories Zhang's original brain MRI image, Indicates the first The first of the categories Zhang's skull and brain MRI images; among them... Indicates the view type. , Indicates the original brain view type. This indicates the type of craniotomy view of the brain, where T represents the frame number of the brain MRI image. and These represent the height and width of a brain MRI image, respectively, with C indicating the number of channels in the brain MRI image. Indicates the first The total number of different types of brain MRI images; Step S2: Construct a Transformer dual-branch network, including: an image encoder from the original brain MRI branch. and linear projection layer Image encoder of cranial MRI branch and linear projection layer and classifier ; Step S3, based on ,get eigenvectors ,based on ,get eigenvectors and will and After fusion, the first The first of the categories Mixed average characteristics ; Step S4, based on right Processing is performed to obtain Category probability distribution Therefore, based on and Constructing the classification target loss ;in, express The predicted class probability; Step S5, based on ,get Average contrast feature vector Thus, the average contrast feature sequence of the original brain was obtained. ; based on ,get Average contrast feature vector Thus, the average contrast feature sequence of the craniotomy brain was obtained. ; Step S6, using and Construct positive and negative sample pairs, and set hierarchical similarity intervals for different negative sample pairs based on category and view type to calculate the hierarchical contrastive loss function. ; Step S7: Construct the total loss function The Transformer dual-branch network is trained using gradient descent, and all parameter weights of the network are updated through gradient backpropagation until the total loss function is reached. The process continues until convergence, thus obtaining the optimal brain MRI image classification model, which is used to classify brain MRI medical images and output the predicted brain category.

[0007] The brain MRI image classification method based on dual-branch hierarchical contrastive learning described in this invention is also characterized in that step S3 includes: Step S3.1, will After unfolding, a set of 2D original view slice images of T frames is obtained. ,in, Indicates the first The first of the categories The first in Zhang's original brain MRI image Frame 2D original view slice image; Will After unfolding, a set of 2D decranialized slice images of T frames is obtained. ,in, Indicates the first The first of the categories The first in Zhang Qu's skull and brain MRI images Frame 2D desquamated view slice image; Step S3.2, will Convert to original view embedding sequence ;Will Convert to descalp view embedding sequence ; Step S3.3: Use self-attention mechanism to respectively... and Perform feature extraction, and accordingly The final MRI image feature vector and The final MRI image feature vector ; Step S3.4, using formula (9) to... and By splicing along the feature dimension, a form is formed. : (9) In equation (9), This indicates splicing along the last dimension.

[0008] Furthermore, step S3.2 includes: Step S3.2.1, will Divided into Non-overlapping sub-blocks { |m=1,2,…,M}, where, express The m-th sub-block in Indicates the total number of non-overlapping sub-blocks; Using equation (1) to obtain Embedded blocks Reusing equation (2) Add position embedding vector Finally, the original view embedding sequence is obtained using equation (3). ,in, Representing the embedding dimension: (1) (2) (3) In equations (1)-(3), Represents the flattening function. The projection matrix of the original view. The offset vector of the original view; Step S3.2.2, will Divided into Non-overlapping sub-blocks { |m=1,2,…,M}, where, express The m-th sub-block in Indicates the total number of non-overlapping sub-blocks; Using equation (4) to obtain Embedded blocks Reusing equation (2) Add position embedding vector Finally, the descaling view embedding sequence is obtained using equation (3). : (4) (5) (6) In equations (4)-(6), The projection matrix of the skull-free view. This is the bias vector for the deskull view.

[0009] Furthermore, step S3.3 includes: Step S3.3.1, use Self-attention module right Processing is performed to obtain global features ,use Self-attention module in right Processing is performed to obtain global features ; Step S3.3.2: Using equations (7) and (8) respectively, we obtain... The final MRI image feature vector and The final MRI image feature vector : (7) (8).

[0010] Furthermore, step S4 includes: Step S4.1, Classifier Using formula (10) Projected to the Types : (10) In equation (10), This is the classification weight matrix. For classification bias vector, It is the total number of categories; Step S4.2, Classifier Using equation (11) to obtain Category probability distribution ,in, express The probability of predicting it as the k-th category; (11) In equation (11), Indicates the activation function; Step S4.3: Calculate the classification target loss using equation (12). ; (12) Furthermore, step S5 includes: Step S5.1: Obtain using equation (13) Average contrast feature vector ; (13) In equation (13), express The weight matrix of the first linear transformation in the process. express The weight moments of the second-level linear transformation in the middle, express The bias of the first linear transformation in the process. express The bias of the second-level linear transformation in the process. This represents the activation function. To compare and learn the intermediate dimension of the feature space, To learn the final dimension of the feature space through comparison; For standardized functions; Step S5.2, using equation (14) to obtain Average contrast feature vector ; (14) In equation (14), express The weight matrix of the first linear transformation in the process. express The weight matrix of the second-level linear transformation in the process. for The bias of the first linear transformation in the process. for The bias of the second-level linear transformation in the model.

[0011] Furthermore, step S6 includes: Step S6.1, with Average contrast feature vector As anchor points, calculate their cross-view similarity with themselves. Average similarity of similar views Average similarity across views of the same type Average similarity of dissimilar views Average similarity across different views ; Step S6.2, use equation (20) to define the first Category-specific validity indicator functions and heterogeneous validity indicator function ; (20) Step S6.3, based on and Calculate anchor points Single-view hierarchical contrast loss function ; Step S6.4, with Average contrast feature vector Using the anchor point, we obtain the following: Single-view hierarchical contrast loss for anchor points ; Step S6.5: Obtain the final hierarchical contrast loss using equation (27). ; (27) Furthermore, step S6.1 includes: Step S6.1.1, will and As a positive sample pair, its cross-view similarity is calculated using equation (15). : (15) In equation (15), This represents the similarity calculation function; Step S6.1.2, will Average contrast feature vector compared to skull-de-skull views of other MRI images in the same category As negative sample pairs, the average similarity of similar views of the same type is calculated using equation (16). ; (16) In equation (16), It is the first Total number of samples by category Represents a small constant that prevents division by zero. Indicates the first The first of the categories The average contrast feature vector of the skull-de-skull view of an MRI image. Indicates an index for other samples of the same type; Step S6.1.3, will Average contrast feature vector compared to the original view of other MRI images of the same category As negative sample pairs, the average similarity across views of the same class is calculated using equation (17). ; (17) In equation (17), Indicates the first The first of the categories The average contrast feature vector of each original view of an MRI image; Step S6.1.4, With all non-first Average contrast features of different types of MRI images with skull views As negative sample pairs, the average similarity of dissimilar same-view images is calculated using equation (18). ; (18) In equation (18), Indicates the first The first of the categories The average contrast feature vector of the skull-de-skull view of an MRI image. Indicates the first Number of MRI images by category; Step S6.1.5, With all non-first Average contrast feature vector of raw views of different types of MRI images As negative sample pairs, the average cross-view similarity of heterogeneous samples is calculated using equation (19). ; (19) In equation (19), Indicates the first The first of the categories The average contrast feature vector of the original view of an MRI image.

[0012] Furthermore, step S6.3 includes: Step S6.3.1: Use equation (21) to define the order of interval loss for negative sample pairs of different difficulties: (twenty one) Step S6.3.2: Calculate using equation (22) and Interval loss : (twenty two) In equation (22), This is the minimum interval threshold between levels; Step S6.3.3: Calculate using equation (23) and Interval loss : (twenty three) Step S6.3.4: Calculate using equation (24) and Interval loss : (twenty four) Step S6.3.5: Calculate using equation (25) and Interval loss : (25) Step S6.3.6: Calculate the single-view hierarchical contrast loss function using equation (26). : (26).

[0013] The present invention provides an electronic device, including a memory and a processor, characterized in that the memory is used to store a program that supports the processor in executing the MRI image classification method, and the processor is configured to execute the program stored in the memory.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. Enhanced structural and semantic collaborative understanding through dual-branch multi-view feature fusion: This invention constructs a Transformer dual-branch structure to extract features from original and decranialized MRI in parallel, and encodes "anatomical structural integrity" and "tissue detail salience" as joint features, which significantly improves the structural robustness and semantic richness of the features.

[0015] 2. The hierarchical loss function precisely characterizes the difficulty levels of negative samples: This invention forces the model to distinguish the difficulty differences of negative samples, which not only strengthens the semantic consistency of similar samples under different perspectives, but also improves the semantic differentiation of neighboring dissimilar samples, making the "pathological semantic distance" in the feature space more in line with clinical practice.

[0016] 3. Enhanced feature discriminativeness through joint optimization of contrastive learning and classification tasks: This invention uses the collaborative updating of contrastive loss and classification loss. Contrastive loss ensures the structural and semantic consistency of the feature space, while classification loss focuses on task-oriented discriminativeness. The two work together to enable the features extracted by the model to capture subtle differences in brain structure and accurately map to disease category labels, ultimately achieving higher accuracy in MRI classification tasks. Attached Figure Description

[0017] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0018] In this embodiment, a brain MRI image classification method based on dual-branch hierarchical contrastive learning mainly overcomes the information limitations of a single perspective and the coarseness of modeling due to homogeneity of negative samples. It achieves joint constraints on the "structural consistency" and "semantic consistency" of MRI images, providing more discriminative feature representations for brain disease classification and enhancing the accuracy of automatic diagnosis of brain diseases. Figure 1 As shown, it includes the following steps: Step S1, obtain the contents Types Brain MRI image dataset The dataset is partitioned as follows: 60% as the training set, 20% as the validation set, and 20% as the test set, forming a dataset partitioning structure that supports five-fold cross-validation; Indicates the first The first of the categories MRI images, and , Indicates the first The first of the categories Zhang's original brain MRI image, Indicates the first The first of the categories Zhang's skull and brain MRI images; among them... Indicates the view type. , Indicates the original brain view type. This indicates the type of craniotomy view of the brain, where T represents the frame number of the brain MRI image. and These represent the height and width of a brain MRI image, respectively, with C indicating the number of channels in the brain MRI image. Indicates the first The total number of different types of brain MRI images; among them, go through The result is obtained after preprocessing.

[0019] Step S2: Construct a Transformer two-branch network, such as... Figure 1 As shown in section (a), it includes: an image encoder for the original brain MRI branch. and linear projection layer Image encoder of cranial MRI branch and linear projection layer and classifier ;in, and All models are loaded with their pre-trained model weights on the large-scale natural image dataset ImageNet, ensuring the model's generality and enabling rapid convergence during training on medical images.

[0020] Step S3, based on ,get eigenvectors ,based on ,get eigenvectors and will and After fusion, the first The first of the categories Mixed average characteristics ; Step S3.1, will After unfolding, a set of 2D original view slice images of T frames is obtained. ,in, Indicates the first The first of the categories The first in Zhang's original brain MRI image Frame 2D original view slice image; Will After unfolding, a set of 2D decranialized slice images of T frames is obtained. ,in, Indicates the first The first of the categories The first in Zhang Qu's skull and brain MRI images Frame 2D desquamated view slice image.

[0021] Step S3.2, will Convert to original view embedding sequence ;Will Convert to descalp view embedding sequence ; Step S3.2.1, will Divided into Non-overlapping sub-blocks { |m=1,2,…,M}, where, express The m-th sub-block in This represents the total number of non-overlapping sub-blocks.

[0022] Using equation (1) to obtain Embedded blocks Reusing equation (2) Add position embedding vector Finally, the original view embedding sequence is obtained using equation (3). ,in, Representing the embedding dimension: (1) (2) (3) In equations (1)-(3), Represents the flattening function. The projection matrix of the original view. This is the offset vector of the original view.

[0023] Step S3.2.2, will Divided into Non-overlapping sub-blocks { |m=1,2,…,M}, where, express The m-th sub-block in Indicates the total number of non-overlapping sub-blocks; Using equation (4) to obtain Embedded blocks Reusing equation (2) Add position embedding vector Finally, the descaling view embedding sequence is obtained using equation (3). : (4) (5) (6) In equations (4)-(6), The projection matrix of the skull-free view. This is the bias vector for the deskull view.

[0024] Step S3.3: Use self-attention mechanism to respectively... and Perform feature extraction, and accordingly The final MRI image feature vector and The final MRI image feature vector ; Step S3.3.1, use Self-attention module right Processing is performed to obtain global features ,use Self-attention module in right Processing is performed to obtain global features .

[0025] Step S3.3.2: Using equations (7) and (8) respectively, we obtain... The final MRI image feature vector and The final MRI image feature vector : (7) (8) Step S3.4, using formula (9) to... and By splicing along the feature dimension, a form is formed. : (9) Step S4, based on right Processing is performed to obtain Category probability distribution Therefore, based on and Constructing the classification target loss ;in, express The predicted category probability.

[0026] Step S4.1, Classifier Using formula (10) Projected to the Types : (10) In equation (10), This is the classification weight matrix. For classification bias vector, It represents the total number of categories.

[0027] Step S4.2, Classifier Using equation (11) to obtain Category probability distribution ,in, express The probability of predicting it as the k-th category; (11) In equation (11), This represents the activation function.

[0028] Step S4.3: Calculate the classification target loss using equation (12). ; (12) Step S5, based on ,get Average contrast feature vector Thus, the average contrast feature sequence of the original brain was obtained. .

[0029] Step S5.1: Obtain using equation (13) Average contrast feature vector ; (13) In equation (13), express The weight matrix of the first linear transformation in the process. express The weight moments of the second-level linear transformation in the middle, express The bias of the first linear transformation in the process. express The bias of the second-level linear transformation in the process. This represents the activation function. To compare and learn the intermediate dimension of the feature space, To learn the final dimension of the feature space through comparison; This is a standardized function.

[0030] Step S5.2, using equation (14) to obtain Average contrast feature vector ; (14) In equation (14), express The weight matrix of the first linear transformation in the process. express The weight matrix of the second-level linear transformation in the process. for The bias of the first linear transformation in the process. for The bias of the second-level linear transformation in the model.

[0031] Step S6, using and Construct positive and negative sample pairs, such as Figure 1 As shown in section (b), the similarity between positive sample pairs is maximized, while the similarity between negative sample pairs is minimized. Hierarchical similarity intervals are set for different negative sample pairs based on category and view type, thereby constraining the similarity differences between different negative sample pairs, which are used to calculate the hierarchical contrastive loss function. Ultimately, the enhanced model extracts and identifies finer-grained features from brain MRI images.

[0032] Step S6.1, with Average contrast feature vector Using anchor points, calculate cross-view similarity. Average similarity of similar views Average similarity across views of the same type Average similarity of dissimilar views Average similarity across different views Among them, only Calculated based on positive sample pairs, and , , and Calculated based on negative sample pairs of varying difficulty; Step S6.1.1, will and As a positive sample pair, its cross-view similarity is calculated using equation (15). : (15) In equation (15), This represents the similarity calculation function.

[0033] Step S6.1.2, will Average contrast feature vector compared to skull-de-skull views of other MRI images in the same category As negative sample pairs, the average similarity of similar views of the same type is calculated using equation (16). ; (16) In equation (16), It is the first Total number of samples by category Represents a small constant that prevents division by zero. Indicates the first The first of the categories The average contrast feature vector of the skull-de-skull view of an MRI image. Indicates the index of other samples of the same type.

[0034] Step S6.1.3, will Average contrast feature vector compared to the original view of other MRI images of the same category As negative sample pairs, the average similarity across views of the same class is calculated using equation (17). ; (17) In equation (17), Indicates the first The first of the categories The average contrast feature vector of the original view of an MRI image.

[0035] Step S6.1.4, With all non-first Average contrast features of different types of MRI images with skull views As negative sample pairs, the average similarity of dissimilar same-view images is calculated using equation (18). ; (18) In equation (18), Indicates the first The first of the categories The average contrast feature vector of the skull-de-skull view of an MRI image. Indicates the first Number of MRI images of different types.

[0036] Step S6.1.5, With all non-first Average contrast feature vector of raw views of different types of MRI images As negative sample pairs, the average cross-view similarity of heterogeneous samples is calculated using equation (19). ; (19) In equation (19), Indicates the first The first of the categories The average contrast feature vector of the original view of an MRI image.

[0037] Step S6.2, use equation (20) to define the first Category-specific validity indicator functions and heterogeneous validity indicator function ; (20) Step S6.3, based on and Calculate anchor points Single-view hierarchical contrast loss function ; Step S6.3.1: Use equation (21) to define the order of interval loss for negative sample pairs of different difficulties: (twenty one) Step S6.3.2: Calculate using equation (22) and Interval loss : (twenty two) In equation (22), This is the minimum interval threshold between levels.

[0038] Step S6.3.3: Calculate using equation (23) and Interval loss : (twenty three) Step S6.3.4: Calculate using equation (24) and Interval loss : (twenty four) Step S6.3.5: Calculate using equation (25) and Interval loss : (25) Step S6.3.6: Calculate the single-view hierarchical contrast loss function using equation (26). : (26).

[0039] Step S6.4, with Average contrast feature vector Using the anchor point, we obtain the following: Single-view hierarchical contrast loss for anchor points .

[0040] Step S6.5: Obtain the final hierarchical contrast loss using equation (27). ; (27) Step S7: Construct the total loss function The Transformer dual-branch network is trained using gradient descent, and all parameter weights of the network are updated through gradient backpropagation until the total loss function is reached. The process continues until convergence, thus obtaining the optimal brain MRI image classification model, which is used to classify brain MRI medical images and output the predicted brain category.

[0041] In this embodiment, an electronic device includes a memory and a processor. The memory stores a program that supports the processor in executing the medical image classification method, and the processor is configured to execute the program stored in the memory.

[0042] In this embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which is executed by a processor to perform the steps of the medical image classification method.

[0043] To verify the effectiveness of this invention, this implementation selected six advanced contrastive learning classification methods: SimCLR (Simple Framework for Contrastive Learning), BCL (Balanced Contrastive Learning), ProCo (Probabilistic Contrastive Learning), ECL (Class-Enhancement Contrastive Learning), BPaCo (Balanced Parametric Contrastive Learning), and CoCa (Consistency Calibration). The models were trained on three brain MRI medical image datasets to evaluate the performance of different methods: the PBT dataset, the OASIS dataset, and the ADHD dataset. Accuracy and F1 score were used as evaluation metrics for all three datasets. The experimental results are shown in Table 1.

[0044] Table 1. Experimental results of the method of the present invention and the selected comparative method on three brain MRI medical image datasets.

[0045] The experimental results demonstrate that the method of this invention (Ours) significantly improves the model's performance in brain MRI image understanding and classification tasks by constraining structural and semantic consistency. This method significantly enhances the discriminative power of MRI image features, effectively mitigating the negative impacts of scarce medical image data and noise interference, enabling the model to better capture subtle differences in brain structure between different disease types.

Claims

1. A brain MRI image classification method based on two-branch hierarchical contrastive learning, characterized in that, The method comprises the following steps: Step S1, obtain the contents Types Brain MRI image dataset ,in, Indicates the first The first of the categories MRI images, and , Indicates the first The first of the categories Zhang's original brain MRI image, Indicates the first The first of the categories Zhang's skull and brain MRI images; among them... Indicates the view type. , Indicates the original brain view type. This indicates the type of craniotomy view of the brain, where T represents the frame number of the brain MRI image. and These represent the height and width of a brain MRI image, respectively, with C indicating the number of channels in the brain MRI image. Indicates the first The total number of different types of brain MRI images; Step S2, constructing a Transformer double-branch network, including: an image encoder of the original brain MRI branch and a linear projection layer , an image encoder of the skull-removed brain MRI branch and a linear projection layer , and a classifier ; Step S3, based on , get characteristic vector , based on , get characteristic vector , and after fusing and , get the first category of the first mixed average feature ; Step S4, based on processing, obtaining category probability distribution ; wherein, based on and constructing classification target loss ; wherein, represents the predicted category probability of ; Step S5, based on , obtaining the average contrast feature vector , so as to obtain the original brain average contrast feature sequence ; Based on , the average contrast feature vector of is obtained , so as to obtain the average contrast feature sequence of the brain without skull ; Step S6, using and Constructing positive and negative sample pairs, setting hierarchical similarity intervals for different negative sample pairs according to categories and view types, and calculating hierarchical contrast loss functions ; Step S7, constructing a total loss function and training the Transformer double-branch network using the gradient descent method, updating all parameter weights of the network through gradient backpropagation until the total loss function converges, thereby obtaining an optimal brain MRI image classification model for classifying brain MRI medical images and outputting a brain prediction category.

2. The brain MRI image classification method based on two-branch hierarchical contrastive learning according to claim 1, characterized in that, The step S3 comprises: Step S3.1, obtaining After unfolding, a set of 2D raw view slice images of T frames is obtained wherein, represents the i-th 2D raw view slice image in the i-th category of the i-th raw brain MRI image of the i-th category of the i-th raw brain MRI image of the i-th category of the i-th raw brain MRI image frame 2D raw view slice image in the i-th raw brain MRI image Will After unfolding, a set of 2D decranialized slice images of T frames is obtained. ,in, Indicates the first The first of the categories The first in Zhang Qu's skull and brain MRI images Frame 2D desquamated view slice image; Step S3.2, converting the original view embedding sequence into a skull-removed view embedding sequence ; converting the original view embedding sequence into a skull-removed view embedding sequence ; Step S3.3, feature extraction is performed on the MRI image and the CT image respectively using self-attention mechanism, and the final MRI image feature vectors and are obtained respectively. The final MRI image feature vectors and are obtained respectively. ; Step S3.4, utilizing formula (9) to form and concatenating in the feature dimension to form : (9) In formula (9), represents splicing along the last dimension.

3. The brain MRI image classification method based on two-branch hierarchical contrastive learning according to claim 2, characterized in that, The step S3.2 comprises: Step S3.2.1, dividing into non-overlapping sub-blocks , where denotes the m-th sub-block in , and denotes the total number of non-overlapping sub-blocks. Using equation (1) to obtain Embedded blocks Reusing equation (2) Add position embedding vector Finally, the original view embedding sequence is obtained using equation (3). ,in, Representing the embedding dimension: (1) (2) (3) in formulas (1) - (3), denotes a flattening function, is a projection matrix of the original view, is a bias vector of the original view; Step S3.2.2, will Divided into Non-overlapping sub-blocks { |m=1,2,…,M}, where, express The m-th sub-block in Indicates the total number of non-overlapping sub-blocks; Using equation (4) to obtain Embedded blocks Reusing equation (2) Add position embedding vector Finally, the descaling view embedding sequence is obtained using equation (3). : (4) (5) (6) in formulas (4) - (6), is a projection matrix for a skull-removed view, is a bias vector for a skull-removed view.

4. The brain MRI image classification method based on two-branch hierarchical contrastive learning according to claim 3, characterized in that, The step S3.3 comprises: Step S3.3.1, using The self-attention module in the middle The input is processed to obtain the global feature of ; the self-attention module in the middle is used to process the input to obtain the global feature of ; and the global feature of is obtained by processing the input using the self-attention module in the middle . Step S3.3.2, obtaining the final MRI image feature vector of formula (7) and formula (8) respectively and and the final MRI image feature vector of formula (7) and formula (8) respectively : (7) (8)。 5. The brain MRI image classification method based on two-branch hierarchical contrastive learning according to claim 1, characterized in that, The step S4 comprises: Step S4.1, classifier Using equation (10) to map to the first category : (10) In formula (10), is a classification weight matrix, is a classification bias vector, is the total number of classes. Step S4.2, the classifier Using equation (11) gives the class probability distribution where, denotes the probability of being predicted as the k-th class. (11) In formula (11), denotes an activation function; Step S4.3, calculate the classification target loss with formula (12) ; (12)。 6. The brain MRI image classification method based on two-branch hierarchical contrastive learning according to claim 1, characterized in that, The step S5 comprises: Step S5.1, obtaining with formula (13) the average contrast feature vector ; (13) In equation (13), express The weight matrix of the first linear transformation in the process, express The weight moments of the second-level linear transformation in the process, express The bias of the first linear transformation in the process. express The bias of the second-level linear transformation in the process. This represents the activation function. To compare and learn the intermediate dimension of the feature space, To learn the final dimension of the feature space through comparison; For standardized functions; Step S5.2, obtaining with formula (14) the average contrast feature vector ; (14) In formula (14), denotes a weight matrix of the first layer linear transformation in denotes a weight matrix of the second layer linear transformation in is a bias of the first layer linear transformation in is a bias of the second layer linear transformation in 7. The brain MRI image classification method based on two-branch hierarchical contrastive learning according to claim 1, characterized in that, The step S6 comprises: Step S6.1, with Average contrast feature vector As anchor points, calculate their cross-view similarity with themselves. Average similarity of similar views Average similarity across views of the same type Average similarity of dissimilar views Average similarity across different views ; Step S6.2, define the first category of homogenous validity indication functions and heterogeneous validity indication functions ; (20) Step S6.3, calculating the anchor point and based on the single-view hierarchical contrastive loss function of the anchor point Step S6.4, taking as the average contrast feature vector as the anchor point, obtaining a single-view hierarchical contrast loss with as the anchor point ; Step S6.5, obtain the final hierarchical contrastive loss with formula (27) ; (27)。 8. The brain MRI image classification method based on two-branch hierarchical contrastive learning according to claim 7, characterized in that, The step S6.1 comprises: Step S6.1.1, calculating with As a positive sample pair, the self-cross view similarity is calculated with formula (15) : (15) In formula (15), represents a similarity calculation function; Step S6.1.2, calculating average contrast feature vector of other MRI images of the same category As negative sample pairs, the average similarity of the same view of the same category is calculated using formula (16) ; (16) In formula (16), is the total number of samples of the category, represents a small constant to prevent division by zero, represents the MRI image of the category, represents the index of other samples of the same category; Step S6.1.3, calculating average contrast feature vector with other MRI image original views of the same category As negative sample pairs, the same category cross-view average similarity is calculated using equation (17) ; (17) In formula (17), representing the first category of the first average contrast feature vector of the MRI image original view of the category Step S6.1.4, calculating with all non-first Average contrast features of MRI images of all non-first As negative sample pairs, the average similarity of different classes of the same view is calculated using formula (18) ; (18) In formula (18), represents the first category of the first average contrast feature vector of the MRI image skull-removed view, represents the first number of MRI images of the first category; Step S6.1.5, calculating with all non-first average contrast feature vector of all classes of MRI image original views As negative sample pairs, the inter-class cross-view average similarity is calculated using equation (19) ; (19) In formula (19), representing the first category of the first average contrast feature vector of the MRI image original view of the category.

9. The brain MRI image classification method based on two-branch hierarchical contrastive learning according to claim 8, characterized in that, The step S6.3 comprises: Step S6.3.1, defining the interval loss order relationship of different difficulty negative sample pairs by formula (21): (21) Step S6.3.2, calculating with formula (22) with the interval loss : (22) In formula (22), is the minimum interval threshold between tiers; Step S6.3.3, calculating with formula (23) with the interval loss : (23) Step S6.3.4, calculating with formula (24) with the interval loss : (24) Step S6.3.5, calculating with formula (25) with the interval loss : (25) Step S6.3.6, calculating monocular hierarchical contrast loss function with formula (26) : (26)。 10. An electronic device comprising a memory and a processor, characterized in that The memory is configured to store a program supporting the processor to execute the MRI image classification method in any one of claims 1-9, and the processor is configured to execute the program stored in the memory.

Images