A kidney cancer MRI image classification method based on contrastive learning

Abstract

This invention relates to a renal cell carcinoma MRI image classification method based on contrastive learning, comprising: acquiring a renal cell carcinoma subtype dataset and corresponding kidney region annotations; inputting the training set into a segmentation network for kidney region segmentation and ROI extraction; performing data augmentation; constructing a dual-domain contrastive learning network model; obtaining the trained dual-domain contrastive learning network model; inputting the MRI image to be classified, after data augmentation, into the trained dual-domain contrastive learning network model to obtain the final renal cell carcinoma classification result. This invention effectively aligns features of different modalities in the same case and enhances the differences between features of different categories, improving classification accuracy. It can mitigate the impact of differences between datasets to a certain extent and enhance the model's generalization ability. Through multimodal image fusion, contrastive learning constraints, and enhanced sample generation strategies, it effectively improves the accuracy and stability of renal tumor classification tasks, and has strong practical application value.

Description

Technical Field

[0001] This invention relates to the field of image classification technology, and in particular to a method for classifying renal cell carcinoma MRI images based on contrastive learning. Background Technology

[0002] Renal cell carcinoma (RCC) is a common tumor of the urinary system, and its incidence has been steadily increasing in recent years, posing a serious health threat to patients. In the current RCC treatment paradigm, subtype classification is a key factor in determining the need for surgery, and different subtypes have significantly different prognoses. However, in clinical practice, the gold standard for RCC subtype identification usually relies on postoperative histopathological evaluation, which cannot be used for treatment planning and prognostic assessment. Magnetic resonance imaging (MRI), due to its widespread use in routine examinations and clinical diagnosis, has become an important tool for preoperative RCC subtype diagnosis. However, the quality of MRI images is significantly affected by scanning parameters and equipment performance, and its interpretation depends on the physician's experience and professional competence, making it susceptible to subjective factors and human error.

[0003] In recent years, the integration of artificial intelligence and medical imaging has shown significant promise in predicting renal cell carcinoma subtypes. While traditional radiomics methods can extract useful quantitative features from medical images, they often neglect the contrast information between the lesion and surrounding kidney tissue. Furthermore, these methods typically require pixel-level annotation of the tumor region and manual intervention by radiologists, making the process both time-consuming and labor-intensive. With the development of deep learning, neural networks have been widely applied to MRI image classification. Deep learning models can automatically learn and extract rich features from raw images, avoiding the need for manual feature extraction. However, the prediction of renal cell carcinoma subtypes still faces the challenge of multi-source heterogeneity. Pixel heterogeneity, caused by factors such as contrast agents and acquisition parameters, leads to inconsistent tumor features, affecting model prediction. Modal heterogeneity refers to the differences in features and resolution between different imaging sequences, making single-modality diagnosis unreliable, and multimodal fusion challenging. Objective representational heterogeneity manifests as differences in morphology and size among similar tumors, as well as the similarity between different tumor categories, making it difficult for models to accurately identify task-relevant features. Summary of the Invention

[0004] To address the shortcomings of existing technologies where multi-source heterogeneity leads to poor MRI image classification results, the present invention aims to provide a contrastive learning-based MRI image classification method for renal cancer that improves classification accuracy, enhances the model's generalization ability, and effectively improves the accuracy and stability of renal tumor classification tasks.

[0005] To achieve the above objectives, the present invention employs the following technical solution: a renal cell carcinoma MRI image classification method based on contrastive learning, the method comprising the following sequential steps:

[0006] (1) Obtain the renal cell carcinoma subtype dataset and the corresponding kidney region annotations, and use the kidney region annotations as the segmentation label Y. s The renal cell carcinoma subtype dataset was divided into a training set, a validation set, and a test set.

[0007] (2) Input the training set into the segmentation network to perform kidney region segmentation and ROI extraction to obtain the ROI region of interest; the segmentation network includes an encoder and a decoder;

[0008] (3) The extracted ROI region of interest is augmented using a linear diffusion strategy to obtain an enhanced MRI image;

[0009] (4) Construct a dual-domain contrastive learning network model, which includes a two-stream network and a dual-domain contrastive learning loss function;

[0010] (5) Input the enhanced MRI image into the dual-domain contrast learning network model for training to obtain the trained dual-domain contrast learning network model.

[0011] (6) The MRI images to be classified are data augmented and then input into the trained dual-domain contrast learning network model to obtain the final renal cell carcinoma classification result.

[0012] Step (2) specifically includes the following steps in sequence:

[0013] (2a) Input the MRI images from the training set into the segmentation network to obtain the kidney region segmentation mask prediction result P. s The segmentation network used is the MedicalNet segmentation network, which includes an encoder and a decoder.

[0014] (2b) Based on the segmentation label Y s Kidney region segmentation mask prediction results P s Construct the cross-entropy loss function;

[0015] (2c) Update all parameters of the MedicalNet segmentation network using the backpropagation mechanism based on the cross-entropy loss function;

[0016] (2d) Use the validation set to evaluate the MedicalNet segmentation network and save the model parameter weights with the best performance, which are the optimal model weights.

[0017] (2e) Repeat steps (2a) to (2d) until training is complete;

[0018] (2f) Load the optimal model weights for the MedicalNet segmentation network;

[0019] (2g) Input all MRI images from the training set, validation set, and test set into the MedicalNet segmentation network to obtain the segmentation mask;

[0020] (2h) Calculate the minimum bounding box of the kidneys in all segmentation masks;

[0021] (2i) Increase the minimum bounding box by 5px along the x, y, z directions to extract the region of interest (ROI).

[0022] Step (3) specifically includes the following steps in sequence:

[0023] (3a) Initialize the T1-weighted MRI images and T2-weighted MRI images of all cases in the training set, and set the ROI of the image to the starting image x0 of the diffusion process;

[0024] (3b) Set the total time step T and the maximum noise value β for the diffusion process;

[0025] (3c) Generate and scale noise based on the current time step t;

[0026] (3d) By scaling the noise δ t Added to the initial image x0, the diffused image x is obtained. t , where the noise δ t The expression is:

[0027]

[0028] The diffused image x t The expression is:

[0029] x t =x0+δ t ε

[0030] Where ε~N(0,I) is standard Gaussian noise;

[0031] (3e) Increment the current time step t by 1;

[0032] (3f) Repeat steps (3c) to (3e) until the current time step t equals the total time step T, the diffusion is complete, and a series of enhanced MRI images are obtained.

[0033] Step (4) specifically includes the following steps in sequence:

[0034] (4a) The dual-stream network includes a T1-weighted image branch, a T2-weighted image branch, and a third fully connected layer. The T1-weighted image branch and the T2-weighted image branch have the same structure. A pre-trained encoder is obtained. The T1-weighted image branch is composed of a pre-trained encoder for processing T1-weighted MRI images, a first adaptive average pooling layer, and a first fully connected layer. The T2-weighted image branch is composed of a pre-trained encoder for processing T2-weighted MRI images, a second adaptive average pooling layer, and a second fully connected layer. The first fully connected layer, the second fully connected layer, and the third fully connected layer have the same structure.

[0035] (4b) For patient i, features were extracted from T1-weighted MRI images using a dual-stream network. Feature extraction from T2-weighted MRI images

[0036] (4c) Feature extraction is performed using adaptive average pooling. Perform pooling;

[0037] (4d) The pooled features are mapped to a low-dimensional space through the first fully connected layer and the second fully connected layer, respectively, to form a learnable prototype representation. and

[0038] (4e) Using prototype representation and Construct the intra-case consistency constraint loss function L intra-case :

[0039]

[0040] Where N is the number of samples and τ is the temperature constant;

[0041] (4f) Representing the prototype and The features are concatenated to form a fused feature vector g. i ;

[0042] (4g) Using the fused feature vector g i Constructing a case-specific constraint loss function L inter-case :

[0043]

[0044] In the formula, δ [·] As an indicator function, δ is defined when the condition in [·] is true. [·] Equal to 1, otherwise δ [·] Equals 0; i, j, k all represent the index values ​​of the samples, y i y is the class label of sample i. jG is the class label of sample j. j and g k These represent the fused feature vectors of samples j and k, respectively.

[0045] (4h) Constructing the dual-domain contrastive learning loss function L DCL =L intra-case +L inter-case ;

[0046] (4i) fuse the feature vector g i The input is fed into the third fully connected layer to calculate the final renal cell carcinoma classification result.

[0047] Step (5) specifically includes the following steps in sequence:

[0048] (5a) Obtain the enhanced MRI images and the corresponding renal cell carcinoma subtype label Y. Select the ROI regions of interest from the T1-weighted MRI images and T2-weighted MRI images of one group of cases, and input them into the dual-domain contrast learning network model to obtain the classification results.

[0049] (5b) Classification results The renal cell carcinoma subtype label Y is input into the loss function L, which includes the cross-entropy loss function L. CLS Comparison with dual-domain learning loss function L DCL :

[0050] L = L CLS +L DCL

[0051] in, y i ∈Y, N is the number of samples, C is the number of subtypes, τ is the temperature constant, δ [·] This is an indicator function that equals 1 when the condition in [·] is true, and 0 otherwise; y i For the true label of the i-th case, Let i be the classification prediction label for the i-th case;

[0052] (5c) Update all parameters of the dual-domain contrastive learning network model through backpropagation based on the loss function L;

[0053] (5d) Use the validation set to evaluate the dual-domain contrastive learning network model and save the parameter weights of the model with the best performance.

[0054] (5e) Repeat steps (5a) to (5d) until training is complete.

[0055] As can be seen from the above technical solution, the beneficial effects of the present invention are as follows: First, the present invention comprehensively utilizes T1-weighted and T2-weighted MRI images, effectively segments the kidney region through a segmentation network, and accurately characterizes the discriminative features of tumor appearance; Second, the present invention adopts a contrastive learning scheme based on intra-case consistency and inter-case specificity constraints. This scheme effectively aligns the features of different modalities in the same case and enhances the differences between features of different categories. Specifically, by introducing consistency constraints, it ensures that images of different modalities can align their corresponding features in the same case, while specificity constraints ensure that the features between different cases can maintain sufficient differences, thereby improving classification accuracy; Third, the present invention generates information-rich augmented samples through a linear diffusion strategy, thereby effectively combating the network performance degradation problem that may be caused by cross-dataset applications. By generating augmented samples and combining them with the linear diffusion strategy, the impact of differences between datasets can be mitigated to a certain extent, enhancing the generalization ability of the model; Fourth, the present invention effectively improves the accuracy and stability of kidney tumor classification tasks through multimodal image fusion, contrastive learning constraints, and augmented sample generation strategies, and has strong practical application value. Attached Figure Description

[0056] Figure 1 This is a flowchart of the method of the present invention;

[0057] Figure 2 This is a diagram showing the kidney region segmentation and ROI extraction of the present invention;

[0058] Figure 3 This is a diagram of the linear diffusion enhancement process of the present invention;

[0059] Figure 4 This is a schematic diagram of the structure of the dual-domain contrastive learning network model in this invention;

[0060] Figure 5 This is the evaluation metric for the renal cell carcinoma MRI image classification task of this invention. Detailed Implementation

[0061] like Figure 1 As shown, a renal cell carcinoma MRI image classification method based on contrastive learning includes the following sequential steps:

[0062] (1) Obtain the renal cell carcinoma subtype dataset and the corresponding kidney region annotations, and use the kidney region annotations as the segmentation label Y. s The renal cell carcinoma subtype dataset is divided into a training set, a validation set, and a test set. In this invention, the renal cell carcinoma subtype dataset and corresponding kidney region annotations are obtained from the Cancer Hospital of the Chinese Academy of Medical Sciences.

[0063] (2) Input the training set into the segmentation network to perform kidney region segmentation and ROI extraction to obtain the ROI region of interest; the segmentation network includes an encoder and a decoder;

[0064] (3) The extracted ROI region of interest is augmented using a linear diffusion strategy to obtain an enhanced MRI image;

[0065] (4) Construct a dual-domain contrastive learning network model, which includes a two-stream network and a dual-domain contrastive learning loss function;

[0066] (5) Input the enhanced MRI image into the dual-domain contrast learning network model for training to obtain the trained dual-domain contrast learning network model.

[0067] (6) The MRI images to be classified are data augmented and then input into the trained dual-domain contrast learning network model to obtain the final renal cell carcinoma classification result.

[0068] like Figure 2 As shown, step (2) specifically includes the following steps in sequence:

[0069] (2a) Input the MRI images from the training set into the segmentation network to obtain the kidney region segmentation mask prediction result P. s The segmentation network used is the MedicalNet segmentation network, which includes an encoder and a decoder.

[0070] (2b) Based on the segmentation label Y s Kidney region segmentation mask prediction results P s Construct the cross-entropy loss function;

[0071] (2c) Update all parameters of the MedicalNet segmentation network using the backpropagation mechanism based on the cross-entropy loss function;

[0072] (2d) Use the validation set to evaluate the MedicalNet segmentation network and save the model parameter weights with the best performance, which are the optimal model weights.

[0073] (2e) Repeat steps (2a) to (2d) until training is complete;

[0074] (2f) Load the optimal model weights for the MedicalNet segmentation network;

[0075] (2g) Input all MRI images from the training set, validation set, and test set into the MedicalNet segmentation network to obtain the segmentation mask;

[0076] (2h) Calculate the minimum bounding box of the kidneys in all segmentation masks;

[0077] (2i) Increase the minimum bounding box by 5px along the x, y, z directions to extract the region of interest (ROI).

[0078] like Figure 3 As shown, step (3) specifically includes the following steps in sequence:

[0079] (3a) Initialize the T1-weighted MRI images and T2-weighted MRI images of all cases in the training set, and set the ROI of the image to the starting image x0 of the diffusion process;

[0080] (3b) Set the total time step T and the maximum noise value β for the diffusion process;

[0081] (3c) Generate and scale noise based on the current time step t;

[0082] (3d) By scaling the noise δ t Added to the initial image x0, the diffused image x is obtained. t , where the noise δ t The expression is:

[0083]

[0084] The diffused image x t The expression is:

[0085] x t =x0+δ t ε

[0086] Where ε~N(0,I) is standard Gaussian noise;

[0087] (3e) Increment the current time step t by 1;

[0088] (3f) Repeat steps (3c) to (3e) until the current time step t equals the total time step T, the diffusion is complete, and a series of enhanced MRI images are obtained.

[0089] Step (4) specifically includes the following steps in sequence:

[0090] (4a) such as Figure 4As shown, the dual-stream network includes a T1-weighted image branch, a T2-weighted image branch, and a third fully connected layer. The T1-weighted image branch and the T2-weighted image branch have the same structure. A pre-trained encoder is obtained for processing T1-weighted MRI images. The T1-weighted image branch consists of a pre-trained encoder for processing T1-weighted MRI images, a first adaptive average pooling layer, and a first fully connected layer. The T2-weighted image branch consists of a pre-trained encoder for processing T2-weighted MRI images, a second adaptive average pooling layer, and a second fully connected layer. The first fully connected layer, the second fully connected layer, and the third fully connected layer have the same structure.

[0091] (4b) For patient i, features were extracted from T1-weighted MRI images using a dual-stream network. Feature extraction from T2-weighted MRI images

[0092] (4c) Feature extraction is performed using adaptive average pooling. Perform pooling;

[0093] (4d) The pooled features are mapped to a low-dimensional space through the first fully connected layer and the second fully connected layer, respectively, to form a learnable prototype representation. and

[0094] (4e) Using prototype representation and Construct the intra-case consistency constraint loss function L intra-case :

[0095]

[0096] Where N is the number of samples and τ is the temperature constant;

[0097] (4f) Representing the prototype and The features are concatenated to form a fused feature vector g. i ;

[0098] (4g) Using the fused feature vector g i Constructing a case-specific constraint loss function L inter-case :

[0099]

[0100] In the formula, δ [·] As an indicator function, δ is defined when the condition in [·] is true. [·] Equal to 1, otherwise δ [·] Equals 0; i, j, k all represent the index values ​​of the samples, y i y is the class label of sample i. jG is the class label of sample j. j and g k These represent the fused feature vectors of samples j and k, respectively.

[0101] (4h) Constructing the dual-domain contrastive learning loss function L DCL =L intra-case +L inter-case ;

[0102] (4i) fuse the feature vector g i The input is fed into the third fully connected layer to calculate the final renal cell carcinoma classification result.

[0103] Step (5) specifically includes the following steps in sequence:

[0104] (5a) Obtain the enhanced MRI images and the corresponding renal cell carcinoma subtype label Y. Select the ROI regions of interest from the T1-weighted MRI images and T2-weighted MRI images of one group of cases, and input them into the dual-domain contrast learning network model to obtain the classification results.

[0105] (5b) Classification results The renal cell carcinoma subtype label Y is input into the loss function L, which includes the cross-entropy loss function L. CLS Comparison with dual-domain learning loss function L DCL :

[0106] L = L CLS +L DCL

[0107] in, y i ∈Y, N is the number of samples, C is the number of subtypes, τ is the temperature constant, δ [·] This is an indicator function that equals 1 when the condition in [·] is true, and 0 otherwise; y i For the true label of the i-th case, Let i be the classification prediction label for the i-th case;

[0108] (5c) Update all parameters of the dual-domain contrastive learning network model through backpropagation based on the loss function L;

[0109] (5d) Use the validation set to evaluate the dual-domain contrastive learning network model and save the parameter weights of the model with the best performance.

[0110] (5e) Repeat steps (5a) to (5d) until training is complete.

[0111] like Figure 5As shown, the dual-domain contrastive learning network model can achieve accurate classification in the MRI image renal cell carcinoma subtype classification task and has a high evaluation index.

[0112] In summary, this invention comprehensively utilizes T1-weighted and T2-weighted MRI images to effectively segment the kidney region through a segmentation network and accurately characterize the discriminative features of tumor appearance. This invention employs a contrastive learning scheme based on intra-case consistency and inter-case specificity constraints. This scheme effectively aligns features of different modalities within the same case and enhances the differences between features of different categories. Specifically, by introducing consistency constraints, it ensures that images of different modalities can align their corresponding features within the same case, while specificity constraints ensure that features between different cases maintain sufficient differences, thereby improving classification accuracy. This invention generates information-rich augmented samples through a linear diffusion strategy, effectively combating the potential network performance degradation caused by cross-dataset applications. By generating augmented samples and combining them with the linear diffusion strategy, the impact of differences between datasets can be mitigated to some extent, enhancing the model's generalization ability. Through multimodal image fusion, contrastive learning constraints, and augmented sample generation strategies, this invention effectively improves the accuracy and stability of kidney tumor classification tasks and has strong practical application value.

Claims

1. A method for classifying renal cell carcinoma MRI images based on contrastive learning, characterized in that: The method includes the following steps in sequence: (1) Obtain the renal cell carcinoma subtype dataset and the corresponding kidney region annotations, and use the kidney region annotations as the segmentation label Y. s The renal cell carcinoma subtype dataset was divided into a training set, a validation set, and a test set. (2) Input the training set into the segmentation network to perform kidney region segmentation and ROI extraction to obtain the ROI region of interest; the segmentation network includes an encoder and a decoder; (3) The extracted ROI region of interest is augmented using a linear diffusion strategy to obtain an enhanced MRI image; (4) Construct a dual-domain contrastive learning network model, which includes a two-stream network and a dual-domain contrastive learning loss function; (5) Input the enhanced MRI image into the dual-domain contrast learning network model for training to obtain the trained dual-domain contrast learning network model. (6) The MRI images to be classified are data-enhanced and then input into the trained dual-domain contrast learning network model to obtain the final renal cell carcinoma classification result. Step (4) specifically includes the following steps in sequence: (4a) The dual-stream network includes a T1-weighted image branch, a T2-weighted image branch, and a third fully connected layer. The T1-weighted image branch and the T2-weighted image branch have the same structure. A pre-trained encoder is obtained. The T1-weighted image branch is composed of a pre-trained encoder for processing T1-weighted MRI images, a first adaptive average pooling layer, and a first fully connected layer. The T2-weighted image branch is composed of a pre-trained encoder for processing T2-weighted MRI images, a second adaptive average pooling layer, and a second fully connected layer. The first fully connected layer, the second fully connected layer, and the third fully connected layer have the same structure. (4b) For patient i, features were extracted from T1-weighted MRI images using a dual-stream network. Feature extraction from T2-weighted MRI images (4c) Feature extraction is performed using adaptive average pooling. Perform pooling; (4d) The pooled features are mapped to a low-dimensional space through the first fully connected layer and the second fully connected layer, respectively, to form a learnable prototype representation. and (4e) Using prototype representation and Construct the intra-case consistency constraint loss function L intra-case : Where N is the number of samples and τ is the temperature constant; (4f) Representing the prototype and The features are concatenated to form a fused feature vector g. i ; (4g) Using the fused feature vector g i Constructing a case-specific constraint loss function L inter-case : In the formula, δ [·] As an indicator function, δ is defined when the condition in [·] is true. [·] Equal to 1, otherwise δ [·] Equals 0; i, j, k all represent the index values ​​of the samples, y i y is the class label of sample i. j G is the class label of sample j. j and g k These represent the fused feature vectors of samples j and k, respectively. (4h) Constructing the dual-domain contrastive learning loss function L DCL =L intra-case +L inter-case ; (4i) fuse the feature vector g i The input is fed into the third fully connected layer to calculate the final renal cell carcinoma classification result.

2. The renal cell carcinoma MRI image classification method based on contrastive learning according to claim 1, characterized in that: Step (2) specifically includes the following steps in sequence: (2a) Input the MRI images from the training set into the segmentation network to obtain the kidney region segmentation mask prediction result P. s The segmentation network used is the MedicalNet segmentation network, which includes an encoder and a decoder. (2b) Based on the segmentation label Y s Kidney region segmentation mask prediction results P s Construct the cross-entropy loss function; (2c) Update all parameters of the MedicalNet segmentation network using the backpropagation mechanism based on the cross-entropy loss function; (2d) Use the validation set to evaluate the MedicalNet segmentation network and save the model parameter weights with the best performance, which are the optimal model weights. (2e) Repeat steps (2a) to (2d) until training is complete; (2f) Load the optimal model weights for the MedicalNet segmentation network; (2g) Input all MRI images from the training set, validation set, and test set into the MedicalNet segmentation network to obtain the segmentation mask; (2h) Calculate the minimum bounding box of the kidneys in all segmentation masks; (2i) Increase the minimum bounding box by 5px along the x, y, z directions to extract the region of interest (ROI).

3. The renal cell carcinoma MRI image classification method based on contrastive learning according to claim 1, characterized in that: Step (3) specifically includes the following steps in sequence: (3a) Initialize the T1-weighted MRI images and T2-weighted MRI images of all cases in the training set, and set the ROI of the image to the starting image x0 of the diffusion process; (3b) Set the total time step T and the maximum noise value β for the diffusion process; (3c) Generate and scale noise based on the current time step t; (3d) By scaling the noise δ t Added to the initial image x0, the diffused image x is obtained. t , where the noise δ t The expression is: The diffused image x t The expression is: x t =x0+δ t e Where ε~N(0,I) is standard Gaussian noise; (3e) Increment the current time step t by 1; (3f) Repeat steps (3c) to (3e) until the current time step t equals the total time step T, the diffusion is complete, and a series of enhanced MRI images are obtained.

4. The renal cell carcinoma MRI image classification method based on contrastive learning according to claim 1, characterized in that: Step (5) specifically includes the following steps in sequence: (5a) Obtain the enhanced MRI images and the corresponding renal cell carcinoma subtype label Y. Select the ROI regions of interest from the T1-weighted MRI images and T2-weighted MRI images of one group of cases, and input them into the dual-domain contrast learning network model to obtain the classification results. (5b) Classification results The renal cell carcinoma subtype label Y is input into the loss function L, which includes the cross-entropy loss function L. CLS Comparison with dual-domain learning loss function L DCL : L=L CLS +L DCL in, N is the sample size, C is the number of subtypes, τ is the temperature constant, and δ is the temperature coefficient. [·] This is an indicator function that equals 1 when the condition in [·] is true, and 0 otherwise; y i For the true label of the i-th case, Let i be the classification prediction label for the i-th case; (5c) Update all parameters of the dual-domain contrastive learning network model through backpropagation based on the loss function L; (5d) Use the validation set to evaluate the dual-domain contrastive learning network model and save the parameter weights of the model with the best performance. (5e) Repeat steps (5a) to (5d) until training is complete.

Images