Brain glioma grading method based on multi-instance causal characterization learning

Abstract

The invention relates to a brain glioma grading method based on multi-instance causal characterization learning, and the method comprises the following steps: S1, data preprocessing, S2, feature extraction through ResNet-50, S3, feature coding of high-dimensional feature representations of instances, S4, decoupling of low-dimensional feature representations of each instance, obtaining causal features and non-causal features, and carrying out classification on the causal features and the non-causal features. S5, constructing an adaptive adjacency matrix A, constructing a sparse adjacency matrix A (k) by adopting a Topk neighbor selection strategy, and carrying out information aggregation through a graph neural network to obtain aggregation features; S6, carrying out local feature aggregation on each scanning plane p, calculating a weight weighted p of each scanning plane by utilizing a full-connection network, and carrying out weighted fusion to obtain an aggregation feature; s7, fusing and grading the aggregated features and the features; the method has the advantage of improving the stability and clinical applicability of the grading model.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing technology, specifically relating to a glioma grading method based on multi-instance causal representation learning. Background Technology

[0002] Gliomas are among the most common malignant tumors of the central nervous system, typically classified into low-grade gliomas (LGG) and high-grade gliomas (HGG) based on their malignancy. Accurate tumor grading is crucial for developing clinical treatment plans and assessing patient prognosis.

[0003] Magnetic resonance imaging (MRI), due to its excellent soft tissue contrast, has become an important tool for the diagnosis and grading of gliomas. Currently, glioma grading largely relies on manual analysis of MRI images by radiologists. This method not only depends on the doctor's professional experience, inherently involves subjectivity and uncertainty, but is also inefficient, failing to meet the growing clinical demands. To address this, researchers have proposed various automated grading methods in recent years, especially deep learning methods based on convolutional neural networks (CNNs), which can automatically extract key features from images, improving the accuracy and efficiency of grading to some extent. However, existing CNN-based methods generally suffer from the following problems: First, CNN models excel at extracting local image features, but struggle to effectively model the spatial relationships between tumor regions and global contextual information, affecting the overall grading judgment; second, gliomas exhibit high heterogeneity in MRI images, with different regions potentially possessing significantly different imaging characteristics. CNNs have limitations in capturing these complex variations, easily leading to unstable grading results.

[0004] To overcome these problems, existing studies have introduced attention mechanisms to automatically focus on key regions in glioma MRI to improve model performance and, to some extent, enhance the model's ability to model global structures and improve grading results. However, current attention-based methods cannot adequately address the high heterogeneity and complexity of glioma MRI and rely on relatively sufficient and accurate labeled data. When labels are scarce or incomplete, the model's robustness and generalization ability are easily affected. Furthermore, while attention mechanisms can visualize the region of interest, the lack of a clear causal relationship between the selected region of interest and the final grading result leads to insufficient model interpretability, making it difficult to meet the clinical need for understandable and reliable AI-assisted diagnostic tools.

[0005] Therefore, existing automated grading methods for gliomas still need improvement in handling the high heterogeneity and complexity of MRI images, as well as in enhancing robustness and interpretability. Summary of the Invention

[0006] The purpose of this invention is to overcome the problems of poor robustness, insufficient generalization ability and lack of interpretability in existing automatic grading methods for gliomas, and to provide a glioma grading method based on multi-instance causal representation learning, so as to effectively deal with the high heterogeneity and complexity of glioma MRI images and improve the stability and clinical applicability of the grading model.

[0007] The technical solution of the present invention is as follows:

[0008] A glioma grading method based on multi-instance causal representation learning includes the following steps:

[0009] S1: Obtain four modal MRIs of a glioma patient and preprocess them. Then, slice the whole brain along three scanning planes using the four modal MRIs. Following the requirements of the multi-instance learning framework, all the patient's slices are packaged into a single package, with each slice serving as an independent instance.

[0010] S2: The package is used to extract features using a pre-trained ResNet-50 to obtain a high-dimensional feature representation for each instance;

[0011] S3: Encode the high-dimensional feature representation of the instance to compress the high-dimensional features into a low-dimensional feature representation of the latent space;

[0012] S4: Decouple the low-dimensional feature representation of each instance to obtain causal and non-causal features, and calculate the causal attention matrix to distinguish key features related to tumor grade in glioma MRI of each package;

[0013] S5: Utilize the causal sensing feature h to analyze the low-dimensional features of each distinguished instance. causal With attention weight attn, an adaptive adjacency matrix A is constructed. A Topk neighbor selection strategy is adopted, retaining only the k most relevant neighbors. A Topk neighbor set is constructed based on the neighbors, and a sparse adjacency matrix A is constructed based on the Topk neighbor set. (k) Then, information is aggregated from the low-dimensional features of each embodiment using a graph neural network to obtain aggregated features;

[0014] S6: Perform local feature aggregation on each scan plane p, where p = 1, 2, 3, to obtain feature h. plane,p Then, a fully connected network is used to calculate the weights for each scan plane. p Then, the features of each scanning phase are weighted and fused to finally obtain the feature h. fuse ;

[0015] S7. Combine the aggregated features with feature h fuseThe features are fused, and a counterfactual generator is introduced to enhance the fused features. The enhanced features are then weighted and fused using global attention to generate feature h. final ;

[0016] S8. Applying a fully connected layer to feature h final The classification is performed to obtain the classification results.

[0017] Furthermore, a total loss function is set up, and multi-step optimization is implemented based on the total loss function.

[0018] Furthermore, the feature extraction in step S2 specifically involves the pre-trained ResNet-50 mapping all instances within the packet to a high-dimensional space through its convolutional layers, pooling layers, and fully connected layers to obtain a high-dimensional feature representation.

[0019] f(x i ) = ResNet-50(x i );

[0020] The high-dimensional features include local features, global features, and multi-view features of gliomas.

[0021] Furthermore, the data encoding in step S3 specifically includes constructing a high-dimensional feature representation with shape... The tensor is given by B, where B is the batch size, N is the number of instances in the bag, and d is the feature dimension. The feature encoding process is as follows:

[0022] h = LeakyReLU(W enc x i +b enc )

[0023] and d represents the weights and biases of the encoding, and d′ represents the dimension of the encoded features.

[0024] After encoding, the high-dimensional features are compressed into low-dimensional feature representations in the latent space.

[0025] Furthermore, step S4 specifically includes the following steps:

[0026] S41. Decouple the low-dimensional features of each instance through causal and noncausal projection to obtain causal features (causa) and noncausal features (noncausal):

[0027] causal = tanh(W) causal h+b causal )

[0028] noncausal=σ(W noncausal h+b noncausal )

[0029] Where tanh(·) and σ(·) represent the hyperbolic tangent and sigmoid activation functions, respectively, and W causal and W noncausal Let b be a causal projection matrix and a non-causal projection matrix. causal and b noncausal For the corresponding bias;

[0030] S42. An independent adversarial network is introduced to process non-causal features. The adversarial network consists of two fully connected layers, activated by ReLU in the middle, and the mean of the output values ​​is used as the adversarial regularization loss L. adv Then L adv The negative weights are incorporated into the total loss function:

[0031]

[0032] Where AdvNet(·) represents the forward propagation process of the adversarial network, B is the number of packets obtained in the data preprocessing stage, N is the number of instances in each packet, and noncausal i This is the i-th non-causal feature;

[0033] S43. Weighted fusion of causal and non-causal features:

[0034] h causal =causal + 0.1·noncausal

[0035] S44, By analyzing the weighted fusion causal perception features h causal Two linear transformations are performed to obtain Q and K, from which the attention matrix is ​​calculated, and weights are assigned to each instance:

[0036] Q,K=W q h causal W k h causal

[0037]

[0038] Among them, W q and W k These are the linear transformation weights used to generate Q and K, and e′ is h. causal Dimensions Used as a scaling factor, Q·K T The result of the dot product of Q and K is used to represent the similarity or association between different nodes;

[0039] S35. Based on the weight of each instance, focus on key features of the tumor region to distinguish key features.

[0040] Furthermore, the construction of the adaptive adjacency matrix A in step S5 includes:

[0041] The similarity between instances i and j is calculated using a Gaussian kernel and incorporated into the causal perceptual attention score to construct an adjacency matrix;

[0042]

[0043] Where, α i Let be the causal perceptual attention score of instance i, given by Calculations show that The causal feature representation of node i is given. α represents the causal feature representation of node j. i and α j A represents the causal weights of nodes i and j. ij This represents the edge weights between nodes i and j in the adjacency matrix.

[0044] Furthermore, the Topk neighbor selection strategy in step S5 includes using... Let A represent the set of Topk neighbors of instance i, and construct a sparse adjacency matrix A. (k) ;

[0045]

[0046] in, This represents the weights of the k most relevant neighbors j of each node i, with the rest set to 0.

[0047] Furthermore, the graph neural network aggregation in step S5 includes processing the sparse adjacency matrix A using a graph neural network. (k) Information is aggregated from the data, and each instance i has features at layer l. initial value The update process is as follows:

[0048]

[0049] Among them, W (l) and b (l) σ(·) represents the weights and biases of the l-th layer of the graph neural network, respectively, and σ(·) represents the non-linear activation function ReLU.

[0050] After l layers of aggregation, the aggregated features are obtained. Where d ” The aggregated feature dimensions.

[0051] Furthermore, step S7 specifically includes the following:

[0052] S71, Combine aggregated features with feature hfuse By fusing the features, we obtain the feature h. comb Then Gaussian noise is injected into it to generate counterfactual features:

[0053]

[0054] in, This represents the added Gaussian noise, and cf_hen(·) is the counterfactual generator.

[0055] S72, The feature h enhanced by counterfactuality cf After further weighted fusion using global attention, the final feature j is generated. final :

[0056] h final =GlobalAttention(h cf ).

[0057] Compared with the prior art, the beneficial effects of the present invention are:

[0058] 1. This invention introduces a multi-instance learning framework, treating glioma MRI slices as multiple independent instances. It combines causal reasoning with graph neural networks to propose a causal perception attention mechanism and a causal perception dynamic aggregation mechanism. This enables dynamic weight allocation and deep structural modeling of multi-instance features, significantly improving the robustness and generalization ability of the classification. Furthermore, this invention enhances the interpretability of the model through causal relationship modeling, effectively alleviating the "black box" problem of artificial intelligence models in medical image analysis. This makes the basis for the model's classification decisions clearer and more transparent, providing clinicians with a more reliable auxiliary diagnostic tool.

[0059] In summary, this invention has the advantages of improving the stability and clinical applicability of the grading model. Attached Figure Description

[0060] Figure 1 This is a flowchart of the data preprocessing process of the present invention;

[0061] Figure 2 This is a schematic diagram of the overall architecture of the MCRNet of the present invention;

[0062] Figure 3 This is a performance comparison chart of the MCRNet of this invention with five advanced models on the validation set;

[0063] Figure 4 This is a performance comparison chart of the MCRNet of this invention with five advanced models on the test set. Detailed Implementation

[0064] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0065] Example 1

[0066] like Figures 1 to 4 As shown, a glioma grading method based on multi-instance causal representation learning includes the following steps:

[0067] I. Data Preprocessing

[0068] To effectively overcome the high heterogeneity and complexity of glioma MRI, targeted data preprocessing is required. Figure 1 );

[0069] S11. Obtain four modal MRIs of patients with gliomas and standardize the four modal MRIs of each patient with gliomas to eliminate intensity differences between different modalities. Use the maximum-minimum normalization method to uniformly map the intensity values ​​of all modal MRIs to the range of [0,1] to eliminate intensity deviations caused by different scanning equipment, changes in patient position, etc., and ensure stable model training.

[0070] S12. For each patient, four modal MRI (T1WI, T2WI, T1ce, FLAIR) are used to perform whole brain slices along three scanning planes (axial, sagittal, and coronal) to ensure comprehensive capture of tumor characteristics.

[0071] S13. In order to further eliminate the difference in imaging resolution, a bilinear interpolation algorithm is used to standardize the spatial resolution of all slices, so that the pixel size of each slice is uniform and the consistency of the model input data is guaranteed.

[0072] S14. In accordance with the requirements of the multi-instance learning framework, all slides of each patient are combined into a package, and each slide in the package is treated as an independent instance for subsequent processing.

[0073] In step S14, the patient's tumor grade (HGG or LGG) is used as a label for the package, inheriting the patient's original pathological diagnosis information. Through weakly supervised learning, the model can infer the overall tumor grade from multiple slice instances, thereby improving the robustness and generalization ability of the grading.

[0074] II. Glioma Grading Network Based on Multi-Instance Causal Representation Learning (MCRNet)

[0075] It mainly includes four stages: feature extraction, feature encoding, feature aggregation, and hierarchical classification. Its overall architecture is as follows: Figure 2 As shown.

[0076] S2, Feature Extraction

[0077] The pre-processed glioma MRI images were used to extract features using a pre-trained ResNet-50. ResNet-50 is a convolutional neural network based on residual learning, which can effectively extract fine-grained features. In particular, the pre-trained ResNet-50 extracts low-level features through convolution operations and processes deeper high-level features through residual blocks, realizing the mapping from the original data to the latent space, and thus obtaining a high-dimensional feature representation for each instance.

[0078] Specifically, each package contains instances x1, x2, ..., x n ,in Let H represent the i-th instance, and H, W, and C represent the height, width, and number of channels of the input data, respectively. ResNet-50 maps these instances to a high-dimensional space through its convolutional layers, pooling layers, and fully connected layers to obtain latent feature representations.

[0079] f(x i ) = ResNet-50(x i )

[0080] Ultimately, this stage yielded local, global, and multi-perspective features of gliomas, ensuring the representational power of each instance's features and providing a rich set of features for subsequent work, laying a solid foundation.

[0081] S3, Feature Coding

[0082] The extracted high-dimensional feature representations are then encoded, which involves compressing, standardizing, and normalizing them using linear transformations and the LeakyReLU activation function for use by subsequent modules. The purpose of feature encoding is to ensure that the features of each instance are suitable for the needs of subsequent modules.

[0083] Specifically, the extracted high-dimensional features are represented to form a shape as The tensor is defined by B, where B is the batch size, N is the number of instances in the batch, and d is the feature dimension. The feature encoding process is as follows:

[0084] h = LeakyReLU(W enc x i +b enc )

[0085] in, and d represents the weights and biases of the encoding, and d′ represents the dimension of the encoded features. The LeakyReLU activation function helps MCRNet learn more complex feature representations by introducing non-linearity.

[0086] Finally, this stage compresses the high-dimensional feature representation into a low-dimensional feature representation in the latent space to ensure that the features of each instance have a good distribution, which is convenient for subsequent processing by the causal perception attention mechanism.

[0087] S4, Causal Perception Attention Mechanism

[0088] The causal awareness attention mechanism is one of the core modules of MCRNet. It decouples the low-dimensional feature representations of each instance to obtain causal and non-causal features, and computes a causal attention matrix to effectively distinguish key features related to tumor grading in glioma MRI. The key to the causal awareness attention mechanism is to focus on the tumor region by learning causal features, avoiding interference from irrelevant regions.

[0089] Specifically:

[0090] S41. Decouple the features of each instance through causal projection and noncausal projection to obtain causal features (causa) and noncausal features (noncausal):

[0091] causal = tanh(W) causal h+b causal )

[0092] noncausal=σ(W noncausal h+b noncausal )

[0093] Where tanh(·) and σ(·) represent the hyperbolic tangent and sigmoid activation functions, respectively, and W causal W noncausal Let b be the causal and non-causal projection matrix. causal b noncausal For the corresponding bias;

[0094] Through this transformation, MCRNet is able to capture feature changes in the data caused by causal relationships and other potential factors.

[0095] S42. To further reduce the interference of non-causal features on the network's final representation ability, an independent adversarial network is introduced to process the non-causal features. This adversarial network consists of two fully connected layers with ReLU activation in between. Its aim is to extract potential noise signals from the non-causal features, and the mean of the output value is calculated as the adversarial regularization loss L. adv :

[0096]

[0097] Where AdvNet(·) represents the forward propagation process of the adversarial network, B can also be understood as the number of packets obtained in the data preprocessing stage (one packet per patient), N is the number of instances in each packet (the number of slices in the patient's MRI), and noncausal i This is the i-th non-causal feature.

[0098] Finally, L adv By incorporating negative weights into the total loss of MCRNet, we encourage MCRNet to reduce its reliance on non-causal features and thus rely more on causal features.

[0099] S43 and MCRNet networks enhance their focus on tumor regions by calculating a causal attention matrix. Prior to this, to appropriately incorporate some non-causal information into the causal features (to prevent excessive information sparsity), the decoupled causal and non-causal features obtained above need to be weighted and fused:

[0100] h causal =causal + 0.1·noncausal

[0101] S44, By analyzing the weighted fusion causal perception features h causal Two linear transformations are performed to obtain Q and K, from which the attention matrix is ​​calculated:

[0102] Q,K=W q h causal W k h causal

[0103]

[0104] Among them, W q and W k These are the linear transformation weights used to generate Q and K, and e′ is h. causal Dimensions Used as a scaling factor to prevent excessively large values ​​when calculating the attention matrix, which could lead to vanishing gradients or instability. The dot product of the query and key represents the similarity or correlation between different nodes.

[0105] S45. The causal perception attention mechanism assigns weights to the features of each instance so that neighbors can be better weighted when performing feature aggregation, thereby enabling the model to focus on the key features of the tumor region and achieve the distinction of key features. The above content improves the robustness and generalization ability of the classification.

[0106] S5, Causal Perception Dynamic Aggregation Mechanism

[0107] To fully utilize the contextual information between instances, step S5 designs a causal-aware dynamic aggregation mechanism to aggregate instance features. This mechanism consists of three steps: constructing an adjacency matrix, selecting Topk neighbors, and aggregation via a graph neural network.

[0108] S51. Adjacency matrix construction, utilizing causal sensing feature h causal An adaptive adjacency matrix A is constructed using the attention weight attn. Specifically, the adjacency matrix is ​​constructed by calculating the similarity between instances i and j using a Gaussian kernel and incorporating causal perceptual attention scores. The formula is as follows:

[0109]

[0110] Where, α i Let be the causal perceptual attention score of instance i, given by Calculations show that The causal feature representation of node i is given. α represents the causal feature representation of node j. i and α j A represents the causal weights of nodes i and j. ij This represents the edge weights between nodes i and j in the adjacency matrix.

[0111] S52. Topk Neighbor Selection: Since a fully connected graph increases computational overhead, a Topk neighbor selection strategy is adopted for each instance i, retaining only its k most relevant neighbors.

[0112] Specifically, use Let A represent the set of Topk neighbors of instance i, and construct a sparse adjacency matrix A. (k) :

[0113]

[0114] This represents the weights of the k most relevant neighbors j of each node i, with the rest set to 0, ultimately constructing a sparse graph.

[0115] This strategy reduces redundant information while ensuring that the most representative neighbor information is fully utilized.

[0116] S53. Graph neural network aggregation to obtain the sparse adjacency matrix A (k) Next, graph neural networks are needed for information aggregation, specifically for the features of each instance i at layer l. (initial value) The update process is as follows:

[0117]

[0118] Among them, W (l) and b (l) σ and φ represent the weights and biases of the l-th layer of the graph neural network, respectively, and σ(·) represents the non-linear activation function ReLU.

[0119] After l layers of aggregation, the aggregated features are obtained. (d ” (The aggregated feature dimension) ensures that the features of each instance include both its own information and the contextual information of its most relevant neighbors, thereby better capturing the spatial heterogeneity of gliomas on different slices.

[0120] S6, Differential Feature Adaptive Fusion Module

[0121] Since the number of slices in glioma MRI varies greatly in different scanning planes (axial, sagittal, and coronal), in order to prevent the network from being biased towards the features of a certain scanning plane, step S6 designed an adaptive fusion module for differential features.

[0122] Specifically, firstly, local feature aggregation is performed on each scan plane p (p = 1, 2, 3) to obtain feature h. plane,p Then, a fully connected network is used to calculate the weights for each scan plane. p Then, the features of each scanning phase are weighted and fused to finally obtain the feature h. fuse :

[0123]

[0124] This module can adaptively balance the feature contributions of different scanning phases, ensuring that multi-view information is fully utilized.

[0125] S7. Counterfactual Generation and Feature Enhancement

[0126] To further enhance the network's generalization ability, a counterfactual generator is introduced into the network to strengthen the aggregated features.

[0127] S71, Aggregate feature h agg The feature h obtained by fusing with the scanning plane fuse By fusing the features, we obtain the feature h. comb Then Gaussian noise is injected into it to generate counterfactual features:

[0128]

[0129] in, This represents the added Gaussian noise, and cf_gen(·) is a counterfactual generator to enhance feature diversity and improve the robustness of the network.

[0130] S72, The feature h enhanced by counterfactuality cf After further weighted fusion using global attention, the final feature h is generated. final :

[0131] h final =GlobalAttention(h cf )

[0132] Finally, a hierarchical structure is achieved using fully connected layers:

[0133] pred = W clf h final +b clf

[0134] h cf W represents a feature enhanced by counterfactual evidence. clf and b clf These represent the weights and biases of the classifier, respectively.

[0135] Preferably, the grading task for gliomas is not only applicable to binary classification (low-grade glioma LGG and high-grade glioma HGG), but can also be extended to three-level or even multi-level classification (such as WHO grades 1-4). In practical applications, the labels can be reclassified in conjunction with clinical diagnostic criteria, and the output space of the model can be adjusted accordingly to adapt to more refined tumor grading tasks.

[0136] III. Loss Function

[0137] MCRNet employs a multi-task loss function to jointly optimize each module, including the cross-entropy loss L for hierarchical calculations. CE =CrossEntropy(pred,y), which is the counterfactual loss L that constrains the consistency between the counterfactual prediction and the true label by means of mean squared error. cf =MSE(pred,cf_pred) and the adversarial regularization loss L for improving network robustness adv .

[0138] Finally, the total loss function of the network is (where λ cf and λ adv The weights for counterfactual loss and adversarial regularization loss are respectively:

[0139] L = L CE +λ cf ·L cf -λ adv ·L adv .

[0140] To fully validate and evaluate the performance of the proposed MCRNet, a series of comparative experiments were designed. The study used the BraTS2020 dataset and a private dataset from the First Affiliated Hospital of Zhengzhou University. The public dataset came from the 2020 Brain Tumor Segmentation (BraTS) Challenge, organized by the Medical Image Computing and Computer-Assisted Intervention Association, which provided preoperative multimodal MRI scans (obtained in routine clinical practice) of high-grade and low-grade glioma patients from multiple institutions. This dataset contains multimodal MRIs of 369 glioma patients (293 HGG and 76 LGG). The private dataset was collected and compiled by the First Affiliated Hospital of Zhengzhou University between 2016 and 2018, including 74 glioma patients (59 HGG and 15 LGG). This study was approved by the local ethics committee (ethics review number: 2019-KY-231). Considering the retrospective nature of the data and patient anonymity, the step of obtaining written informed consent from patients was omitted. Both datasets were labeled and validated by professional radiologists to extract masks of tumor edema, necrosis, and the core region.

[0141] Specifically, for the private dataset, annotation was performed by two experienced radiologists with 10 years of experience using the ITK-SNAP software. If the two radiologists could not reach a consensus on the annotation, and the difference was less than 5%, the final annotation was determined based on the overlapping area; if the difference was greater than 5%, the senior radiologist made the final decision. Furthermore, the grading labels for the above data were all confirmed by pathological results.

[0142] Specifically, to verify the robustness and generalization ability of MCRNet, comparative experiments were conducted on both the validation and test sets, including four advanced methods for glioma grading (IPTV, ResMT, U-Net, and CSF-Glioma) and an advanced multi-instance learning method, TransMIL. Among them, IPTV is a radiomics-based method, ResMT is a hybrid network for 3D MRI, and CSF-Glioma utilizes causal inference.

[0143] All experiments were conducted on a Linux server equipped with an NVIDIA RTX 3090 graphics processing unit, using the deep learning frameworks PyTorch 1.13.1, CUDA 11.6.0, and Python 3.8. The hyperparameter settings are shown in Table 1.

[0144]

[0145] Table 1 (Hyperparameter settings)

[0146] Based on the results of the comparative experiments (Tables 2, 3 and 4), Figure 3 , 4 As shown in the figure, the proposed MCRNet outperforms other models in both validation and test sets, especially in terms of AUC, which reaches 99.65% (validation set) and 99.55% (test set). Moreover, the difference between the validation set results and the test set results is minimal, indicating that the proposed MCRNet has excellent generalization ability.

[0147] Specifically, MCRNet demonstrates stronger robustness in handling highly complex and heterogeneous glioma MRI, enabling accurate grading and performing on par with the hybrid network ResMT for 3D MRI. Other models such as IPTV, U-Net, CSF-Glioma, and TransMIL generally performed poorly on the validation set, particularly in AUC, failing to achieve high accuracy and robustness. Furthermore, MCRNet exhibited minimal fluctuation between its validation and test set results, with an accuracy decrease of 0.16%, an F1 score decrease of 0.17%, and an AUC decrease of 0.05%, fully validating its robustness and generalization ability. This is attributed to the combined effect of modules such as multi-instance learning, causal-aware attention mechanisms, and causal-aware dynamic aggregation mechanisms.

[0148]

[0149]

[0150] Table 2 compares the performance of MCRNet with five state-of-the-art models on the validation set (bold text indicates the maximum value of the metric).

[0151]

[0152] Table 3 compares the performance of MCRNet with five state-of-the-art models on the test set (bold indicates the maximum value of the metric).

[0153] Example 2

[0154] A glioma grading method based on multi-instance causal representation learning differs from Example 1 in that the specific convolutional neural network architecture in the feature extraction module of glioma MRI slices is replaced with ResNet, VGG, or DenseNet. For edge computing devices or scenarios with low computational resource requirements, lightweight networks (such as MobileNet and EfficientNet) can also be used as feature extractors, thereby reducing the computational overhead of the model and improving the flexibility of the application.

[0155] Example 3

[0156] A glioma grading method based on multi-instance causal representation learning differs from Example 1 in that the causal perception attention mechanism can be implemented using different modeling methods, such as self-attention, multi-head attention, or channel attention. In specific implementation, the attention mechanism structure can be flexibly selected according to data characteristics and task requirements to adapt to different medical image tasks and glioma types.

[0157] Example 4

[0158] A glioma grading method based on multi-instance causal representation learning is presented. The difference in Example 1 lies in the fact that the graph neural network used in the feature aggregation process is not limited to GCN (Graph Convolutional Network). Variants such as GraphSAGE, GAT (Graph Attention Network), or Transformer-based GNN can also be used to better model the complex relationships between instances. Different GNN structures can be selected based on the number of slices and the feature distribution between slices in the glioma MRI to optimize performance and interpretability.

[0159] Example 5

[0160] A glioma grading method based on multi-instance causal representation learning is presented. The difference in Example 1 is that the proposed multi-instance causal representation learning method is also applicable to other types of medical image analysis tasks, such as the benign / malignant differentiation of lung nodules and the detection of breast tumor lesions. By simply adjusting the input structure and feature processing module according to the specific task, this invention can be extended to various intelligent clinical image diagnosis scenarios.

[0161] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A glioma grading method based on multi-instance causal representation learning, characterized in that: Includes the following steps: S1: Obtain four modal MRIs of a glioma patient and preprocess them. Then, slice the whole brain along three scanning planes using the four modal MRIs. Following the requirements of the multi-instance learning framework, all the patient's slices are packaged into a single package, with each slice serving as an independent instance. S2: The package is used to extract features using a pre-trained ResNet-50 to obtain a high-dimensional feature representation for each instance; S3: Encode the high-dimensional feature representation of the instance to compress the high-dimensional features into a low-dimensional feature representation of the latent space; S4: Decouple the low-dimensional feature representation of each instance to obtain causal and non-causal features, and calculate the causal attention matrix to distinguish key features related to tumor grade in glioma MRI of each package; S5: Utilize the causal sensing feature h to analyze the low-dimensional features of each distinguished instance. causal With attention weight attn, an adaptive adjacency matrix A is constructed. A Topk neighbor selection strategy is adopted, retaining only the k most relevant neighbors. A Topk neighbor set is constructed based on the neighbors, and a sparse adjacency matrix A is constructed based on the Topk neighbor set. (k) Then, information is aggregated from the low-dimensional features of each embodiment using a graph neural network to obtain aggregated features; S6: Perform local feature aggregation on each scan plane p, where p = 1, 2, 3, to obtain feature h. plane,p Then, a fully connected network is used to calculate the weights for each scan plane. p Then, the features of each scanning phase are weighted and fused to finally obtain the feature h. fuse ; S7. Combine the aggregated features with feature h fuse The features are fused, and a counterfactual generator is introduced to enhance the fused features. The enhanced features are then weighted and fused using global attention to generate feature h. final ; S8. Applying a fully connected layer to feature h final The classification is performed to obtain the classification results.

2. The glioma grading method based on multi-instance causal representation learning according to claim 1, characterized in that: It also sets a total loss function and implements multi-step optimization based on the total loss function.

3. The glioma grading method based on multi-instance causal representation learning according to claim 1, characterized in that: The feature extraction in step S2 specifically involves the pre-trained ResNet-50 mapping all instances within the packet to a high-dimensional space through its convolutional layers, pooling layers, and fully connected layers to obtain a high-dimensional feature representation. f(x i )=ResNet-50(x i ); in High-dimensional feature representation includes local features, global features, and multi-view features of gliomas.

4. The glioma grading method based on multi-instance causal representation learning according to claim 1, characterized in that: The data encoding in step S3 specifically includes representing high-dimensional features into a shape as follows: The tensor is given by B, where B is the batch size, N is the number of instances in the bag, and d is the feature dimension. The feature encoding process is as follows: h=LeakyReLU(W enc x i +b enc ) and d represents the weights and biases of the encoding, and d′ represents the dimension of the encoded feature. After encoding, the high-dimensional features are compressed into low-dimensional feature representations in the latent space.

5. A glioma grading method based on multi-instance causal representation learning according to claim 2, characterized in that: Step S4 specifically includes the following steps: S41. Decouple the low-dimensional features of each instance through causal and noncausal projection to obtain causal features (causa) and noncausal features (noncausal): causal=tanh(W causal h+b causal ) noncausal=σ(W noncausal h+b noncausal ) Where tanh(·) and σ(·) represent the hyperbolic tangent and sigmoid activation functions, respectively, and W causal and W noncausal Let b be a causal projection matrix and a non-causal projection matrix. causal and b noncausal For the corresponding bias; S42. An independent adversarial network is introduced to process non-causal features. The adversarial network consists of two fully connected layers, activated by ReLU in the middle, and the mean of the output values ​​is used as the adversarial regularization loss L. adv Then L adv The negative weights are incorporated into the total loss function: Where AdvNet(·) represents the forward propagation process of the adversarial network, B is the number of packets obtained in the data preprocessing stage, N is the number of instances in each packet, and noncausal i This is the i-th non-causal feature; S43. Weighted fusion of causal and non-causal features: h causal =causal+0.1·noncausal S44, By analyzing the weighted fusion causal perception features h causal Two linear transformations are performed to obtain Q and K, from which the attention matrix is ​​calculated, and weights are assigned to each instance: Q,K=W q h causal ,W k h causal Among them, W q and W k These are the linear transformation weights used to generate Q and K, and e′ is h. causal Dimensions Used as a scaling factor, Q·K T The result of the dot product of Q and K is used to represent the similarity or association between different nodes; S35. Based on the weight of each instance, focus on key features of the tumor region to distinguish key features.

6. The glioma grading method based on multi-instance causal representation learning according to claim 5, characterized in that: The construction of the adaptive adjacency matrix A in step S5 includes: The similarity between instances i and j is calculated using a Gaussian kernel and incorporated into the causal perceptual attention score to construct an adjacency matrix; Where, α i Let be the causal perceptual attention score of instance i, given by Calculations show that The causal feature representation of node i is given. α represents the causal feature representation of node j. i and α j A represents the causal weights of nodes i and j. ij This represents the edge weights between nodes i and j in the adjacency matrix.

7. A glioma grading method based on multi-instance causal representation learning according to claim 6, characterized in that: The Topk neighbor selection strategy in step S5 includes using Let A represent the set of Topk neighbors of instance i, and construct a sparse adjacency matrix A. (k) ; in, This represents the weights of the k most relevant neighbors j of each node i, with the rest set to 0.

8. A glioma grading method based on multi-instance causal representation learning according to claim 7, characterized in that: The graph neural network aggregation in step S5 includes processing the sparse adjacency matrix A using a graph neural network. (k) Information is aggregated from the data, and each instance i has features at layer l. initial value The update process is as follows: Among them, W (l) and b (l) σ(·) represents the weights and biases of the l-th layer of the graph neural network, respectively, and σ(·) represents the non-linear activation function ReLU. After l layers of aggregation, the aggregated features are obtained. Where d″ is the aggregated feature dimension.

9. A glioma grading method based on multi-instance causal representation learning according to claim 1, characterized in that: Step S7 specifically includes the following: S71, Combine aggregated features with feature h fuse By fusing the features, we obtain the feature h. comb Then Gaussian noise is injected into it to generate counterfactual features: in, This indicates the addition of Gaussian noise, and cf_gen(·) is a counterfactual generator; S72, The feature h enhanced by counterfactuality cf After further weighted fusion using global attention, the final feature h is generated. final : h final =GlobalAttention(h cf )。

Images