Kidney cancer MRI image classification method based on contrast learning

Abstract

The invention relates to a kidney cancer MRI (Magnetic Resonance Imaging) image classification method based on comparative learning. The method comprises the following steps: acquiring a kidney cancer subtype data set and a corresponding kidney region label; inputting the training set into a segmentation network for kidney region segmentation and ROI extraction; performing data enhancement; constructing a double-domain contrast learning network model; obtaining a trained double-domain contrast learning network model; and performing data enhancement on a to-be-classified MRI image, and inputting the to-be-classified MRI image into the trained double-domain contrast learning network model to obtain a final kidney cancer classification result. According to the method, the features of different modal images in the same case are effectively aligned, the difference between different types of features is enhanced, the classification precision is improved, the influence caused by the difference between data sets can be reduced to a certain extent, and the generalization ability of the model is enhanced; through multi-modal image fusion, contrast learning constraint and enhancement of a sample generation strategy, the accuracy and stability of a kidney tumor classification task are effectively improved, and the method has a relatively high practical application value.

Description

Technical Field

[0001] The invention relates to the technical field of image classification, and in particular to a kidney cancer MRI image classification method based on contrast learning. Background Art

[0002] Renal cancer is a common tumor in the urinary system. Its incidence has been steadily increasing in recent years, posing a serious health threat to patients. In the current treatment paradigm for renal cancer, subtype classification is a key factor in determining whether surgery is needed, and there are significant differences in the prognosis of different subtypes. However, in clinical practice, the gold standard for renal cancer subtypes usually relies on postoperative histopathological evaluation, which cannot be used for treatment planning and prognosis assessment. Magnetic resonance imaging (MRI) has become an important tool for preoperative diagnosis of renal cancer subtypes due to its wide application in routine examinations and clinical diagnosis. However, the quality of MRI images is significantly affected by scanning parameters and equipment performance, and its interpretation depends on the doctor's experience and professionalism, and is easily interfered by subjective factors and human errors.

[0003] In recent years, the combination of artificial intelligence and medical imaging has performed significantly in the task of renal cancer subtype prediction. Although traditional radiomics methods can extract useful quantitative features from medical images, they usually ignore the contrast information between lesions and surrounding renal tissues. In addition, these methods usually require pixel-level annotation of tumor regions and require manual intervention by radiologists, making this process time-consuming and laborious. With the development of deep learning, neural networks have been widely used in MRI image classification. Deep learning models can automatically learn and extract rich features from raw images, avoiding the process of manual feature extraction. However, the prediction of renal cancer subtypes still faces the challenge of multi-source heterogeneity. Pixel heterogeneity is caused by factors such as contrast agents and acquisition parameters, resulting in inconsistent tumor features and affecting model prediction. Modality heterogeneity refers to the differences in features and resolution between different imaging sequences, making it difficult to reliably diagnose with a single modality and challenging multimodal fusion. Objective representation heterogeneity is manifested as differences in morphology and size of similar tumors and similarities between different tumor categories, making it difficult for the model to accurately identify task-related features. Summary of the invention

[0004] In order to solve the defect of poor MRI image classification due to multi-source heterogeneity differences in the prior art, the purpose of the present invention is to provide a renal cancer MRI image classification method based on contrastive learning, which improves classification accuracy, enhances the generalization ability of the model, and effectively improves the accuracy and stability of renal tumor classification tasks.

[0005] To achieve the above object, the present invention adopts the following technical solution: a method for classifying renal cancer MRI images based on contrast learning, the method comprising the following steps in order:

[0006] (1) Obtain a renal cancer subtype dataset and the corresponding kidney region annotations, and use the kidney region annotations as the segmentation label Y s , divide the renal cancer subtype dataset into training set, validation set and test set;

[0007] (2) inputting the training set into a segmentation network to perform kidney region segmentation and ROI extraction to obtain a ROI region of interest; the segmentation network includes an encoder and a decoder;

[0008] (3) Performing data enhancement on the extracted ROI region of interest using a linear diffusion strategy to obtain an enhanced MRI image;

[0009] (4) constructing a dual-domain contrastive learning network model, which includes a dual-stream network and a dual-domain contrastive learning loss function;

[0010] (5) inputting the enhanced MRI image into the dual-domain contrast learning network model for training to obtain a trained dual-domain contrast learning network model;

[0011] (6) After data enhancement, the MRI image to be classified is input into the trained dual-domain contrast learning network model to obtain the final renal cancer classification result.

[0012] Step (2) specifically includes the following steps in order:

[0013] (2a) Input the MRI images in the training set into the segmentation network to obtain the kidney region segmentation mask prediction result P s ; The segmentation network adopts the MedicalNet segmentation network, and the segmentation network MedicalNet includes an encoder and a decoder;

[0014] (2b) According to the segmentation label Y s And the kidney region segmentation mask prediction result P s Construct cross entropy loss function;

[0015] (2c) All parameters of the MedicalNet segmentation network are updated through the back-propagation mechanism according to the cross-entropy loss function;

[0016] (2d) Use the validation set to evaluate the MedicalNet segmentation network and save the model parameter weights with the best indicators, i.e. the optimal model weights;

[0017] (2e) Repeat steps (2a) to (2d) until the training is completed;

[0018] (2f) Load the optimal model weights of the MedicalNet segmentation network;

[0019] (2g) Input all MRI images in the training set, validation set, and test set into the MedicalNet segmentation network to obtain segmentation masks;

[0020] (2h) Calculate the minimum bounding box of the kidney in all segmentation masks;

[0021] (2i) Increase the minimum bounding box by 5px along the x, y, and z directions to extract the ROI region of interest.

[0022] Step (3) specifically includes the following steps in order:

[0023] (3a) Initialize the T1-weighted MRI images and T2-weighted MRI images of all cases in the training set, and set the ROI of the image to the starting image x0 of the diffusion process;

[0024] (3b) Set the total time step T and the maximum noise value β of the diffusion process;

[0025] (3c) Noise generation and scaling according to the current time step t;

[0026] (3d) By adding the scaled noise δ t Add to the starting image x0 to get the diffused image x t , where the noise δ t The expression is:

[0027]

[0028] The diffused image x t The expression is:

[0029] x t =x0+δ t ε

[0030] Among them, ε~N(0,I) is standard Gaussian noise;

[0031] (3e) The current time step t is incremented by 1;

[0032] (3f) Repeat steps (3c) to (3e) until the current time step t is equal to the total time step T, the diffusion is completed, and a series of enhanced MRI images are obtained.

[0033] Step (4) specifically includes the following steps in order:

[0034] (4a) The dual-stream network includes a T1 weighted image branch, a T2 weighted image branch and a third fully connected layer, the T1 weighted image branch and the T2 weighted image branch have the same structure, a pre-trained encoder is obtained, the pre-trained encoder for processing T1 weighted MRI images, the first adaptive average pooling layer, and the first fully connected layer constitute the T1 weighted image branch, and the pre-trained encoder for processing T2 weighted MRI images, the second adaptive average pooling layer, and the second fully connected layer constitute the T2 weighted image branch; the first fully connected layer, the second fully connected layer, and the third fully connected layer have the same structure;

[0035] (4b) For patient i, a two-stream network is used to extract features from the T1-weighted MRI image. Feature extraction from T2-weighted MRI images

[0036] (4c) Extract features through adaptive average pooling operation Perform pooling;

[0037] (4d) The pooled features are mapped to a low-dimensional space through the first fully connected layer and the second fully connected layer to form a learnable prototype representation. and

[0038] (4e) Using prototype representation and Construct the intra-case consistency constraint loss function L intra-case :

[0039]

[0040] Where N is the number of samples, τ is the temperature constant;

[0041] (4f) The prototype is represented as and Splice to form a fusion feature vector g i ;

[0042] (4g) Use the fused feature vector g i Construct case-specific constraint loss function L inter-case :

[0043]

[0044] In the formula, δ [·] is an indicator function. When the conditions in [·] are met, δ [·] is equal to 1, otherwise δ [·] is equal to 0; i, j, k all represent the index value of the sample, y i is the category label of sample i, y jis the category label of sample j, g j and g k Represent the fused feature vectors of samples j and k respectively;

[0045] (4h) Constructing the dual-domain contrastive learning loss function L DCL =L intra-case +L inter-case ;

[0046] (4i) The fused feature vector g i Input into the third fully connected layer to calculate the final kidney cancer classification result.

[0047] Step (5) specifically includes the following steps in order:

[0048] (5a) Obtain the enhanced MRI image and the corresponding renal cancer subtype label Y, select the ROI region of interest of the T1-weighted MRI image and the T2-weighted MRI image of one group of cases, and input them into the dual-domain contrast learning network model to obtain the classification results.

[0049] (5b) The classification results and renal cancer subtype label Y are input into the loss function L, which includes the cross entropy loss function L CLS And the dual-domain contrast learning loss function L DCL :

[0050] L=L CLS +L DCL

[0051] in, y i ∈Y, N is the number of samples, C is the number of subtype categories, τ is the temperature constant, δ [·] is an indicator function, which is equal to 1 when the condition in [·] is met, and equal to 0 otherwise; i is the true label of the ith case, Predict the classification label for the i-th case;

[0052] (5c) updating all parameters of the dual-domain contrastive learning network model through the back-propagation mechanism according to the loss function L;

[0053] (5d) Use the validation set to evaluate the dual-domain contrastive learning network model and save the model parameter weights with the best index;

[0054] (5e) Repeat steps (5a) to (5d) until the training is completed.

[0055] It can be seen from the above technical scheme that the beneficial effects of the present invention are as follows: first, the present invention comprehensively utilizes T1-weighted and T2-weighted MRI images, effectively segments the kidney area through a segmentation network, and accurately characterizes the discriminant features of the tumor appearance; second, the present invention adopts a contrastive learning scheme based on intra-case consistency and inter-case specific constraints, which effectively aligns the features of different modal images in the same case and enhances the differences between different category features. Specifically, by introducing consistency constraints, it is ensured that images of different modalities can align their corresponding features in the same case, and through specific constraints, the features between different cases can maintain sufficient differences, thereby improving the classification accuracy; third, the present invention generates information-rich enhanced samples through a linear diffusion strategy, thereby effectively counteracting the network performance attenuation problem that may be caused by cross-dataset applications. By generating enhanced samples and combining the linear diffusion strategy, the impact of differences between data sets can be alleviated to a certain extent, and the generalization ability of the model can be enhanced; fourth, the present invention effectively improves the accuracy and stability of kidney tumor classification tasks through multimodal image fusion, contrastive learning constraints and enhanced sample generation strategies, and has strong practical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a flow chart of the method of the present invention;

[0057] Figure 2 It is the kidney region segmentation and ROI extraction map of the present invention;

[0058] Figure 3 It is a linear diffusion enhancement process diagram of the present invention;

[0059] Figure 4 Schematic diagram of the structure of the dual-domain contrast learning network model in the present invention;

[0060] Figure 5 It is the evaluation indicator of the present invention in the task of renal cancer MRI image classification. DETAILED DESCRIPTION

[0061] like Figure 1 As shown, a method for classifying renal cancer MRI images based on contrastive learning comprises the following steps in order:

[0062] (1) Obtain a renal cancer subtype dataset and the corresponding kidney region annotations, and use the kidney region annotations as the segmentation label Y s , the renal cancer subtype dataset is divided into a training set, a validation set and a test set; in the present invention, the renal cancer subtype dataset and the corresponding kidney region annotations are obtained from the Cancer Hospital of the Chinese Academy of Medical Sciences.

[0063] (2) inputting the training set into a segmentation network to perform kidney region segmentation and ROI extraction to obtain a ROI region of interest; the segmentation network includes an encoder and a decoder;

[0064] (3) Performing data enhancement on the extracted ROI region of interest using a linear diffusion strategy to obtain an enhanced MRI image;

[0065] (4) constructing a dual-domain contrastive learning network model, which includes a dual-stream network and a dual-domain contrastive learning loss function;

[0066] (5) inputting the enhanced MRI image into the dual-domain contrast learning network model for training to obtain a trained dual-domain contrast learning network model;

[0067] (6) After data enhancement, the MRI image to be classified is input into the trained dual-domain contrast learning network model to obtain the final renal cancer classification result.

[0068] like Figure 2 As shown, step (2) specifically includes the following steps in order:

[0069] (2a) Input the MRI images in the training set into the segmentation network to obtain the kidney region segmentation mask prediction result P s ; The segmentation network adopts the MedicalNet segmentation network, and the segmentation network MedicalNet includes an encoder and a decoder;

[0070] (2b) According to the segmentation label Y s And the kidney region segmentation mask prediction result P s Construct cross entropy loss function;

[0071] (2c) All parameters of the MedicalNet segmentation network are updated through the back-propagation mechanism according to the cross-entropy loss function;

[0072] (2d) Use the validation set to evaluate the MedicalNet segmentation network and save the model parameter weights with the best indicators, i.e. the optimal model weights;

[0073] (2e) Repeat steps (2a) to (2d) until the training is completed;

[0074] (2f) Load the optimal model weights of the MedicalNet segmentation network;

[0075] (2g) Input all MRI images in the training set, validation set, and test set into the MedicalNet segmentation network to obtain segmentation masks;

[0076] (2h) Calculate the minimum bounding box of the kidney in all segmentation masks;

[0077] (2i) Increase the minimum bounding box by 5px along the x, y, and z directions to extract the ROI region of interest.

[0078] like Figure 3 As shown, step (3) specifically includes the following steps in order:

[0079] (3a) Initialize the T1-weighted MRI images and T2-weighted MRI images of all cases in the training set, and set the ROI of the image to the starting image x0 of the diffusion process;

[0080] (3b) Set the total time step T and the maximum noise value β of the diffusion process;

[0081] (3c) Noise generation and scaling according to the current time step t;

[0082] (3d) By adding the scaled noise δ t Add to the starting image x0 to get the diffused image x t , where the noise δ t The expression is:

[0083]

[0084] The diffused image x t The expression is:

[0085] x t =x0+δ t ε

[0086] Among them, ε~N(0,I) is standard Gaussian noise;

[0087] (3e) The current time step t is incremented by 1;

[0088] (3f) Repeat steps (3c) to (3e) until the current time step t is equal to the total time step T, the diffusion is completed, and a series of enhanced MRI images are obtained.

[0089] Step (4) specifically includes the following steps in order:

[0090] (4a) Figure 4As shown, the dual-stream network includes a T1 weighted image branch, a T2 weighted image branch and a third fully connected layer, the structures of the T1 weighted image branch and the T2 weighted image branch are the same, a pre-trained encoder is obtained, and the pre-trained encoder for processing T1 weighted MRI images, the first adaptive average pooling layer, and the first fully connected layer constitute the T1 weighted image branch, and the pre-trained encoder for processing T2 weighted MRI images, the second adaptive average pooling layer, and the second fully connected layer constitute the T2 weighted image branch; the structures of the first fully connected layer, the second fully connected layer, and the third fully connected layer are the same;

[0091] (4b) For patient i, a two-stream network is used to extract features from the T1-weighted MRI image. Feature extraction from T2-weighted MRI images

[0092] (4c) Extract features through adaptive average pooling operation Perform pooling;

[0093] (4d) The pooled features are mapped to a low-dimensional space through the first fully connected layer and the second fully connected layer to form a learnable prototype representation. and

[0094] (4e) Using prototype representation and Construct the intra-case consistency constraint loss function L intra-case :

[0095]

[0096] Where N is the number of samples, τ is the temperature constant;

[0097] (4f) The prototype is represented as and Splice to form a fusion feature vector g i ;

[0098] (4g) Use the fused feature vector g i Construct case-specific constraint loss function L inter-case :

[0099]

[0100] In the formula, δ [·] is an indicator function. When the conditions in [·] are met, δ [·] is equal to 1, otherwise δ [·] is equal to 0; i, j, k all represent the index value of the sample, y i is the category label of sample i, y jis the category label of sample j, g j and g k Represent the fused feature vectors of samples j and k respectively;

[0101] (4h) Constructing the dual-domain contrastive learning loss function L DCL =L intra-case +L inter-case ;

[0102] (4i) The fused feature vector g i Input into the third fully connected layer to calculate the final kidney cancer classification result.

[0103] Step (5) specifically includes the following steps in order:

[0104] (5a) Obtain the enhanced MRI image and the corresponding renal cancer subtype label Y, select the ROI region of interest of the T1-weighted MRI image and the T2-weighted MRI image of one group of cases, and input them into the dual-domain contrast learning network model to obtain the classification results.

[0105] (5b) The classification results and renal cancer subtype label Y are input into the loss function L, which includes the cross entropy loss function L CLS And the dual-domain contrast learning loss function L DCL :

[0106] L=L CLS +L DCL

[0107] in, y i ∈Y, N is the number of samples, C is the number of subtype categories, τ is the temperature constant, δ [·] is an indicator function, which is equal to 1 when the condition in [·] is met, and equal to 0 otherwise; i is the true label of the ith case, Predict the classification label for the i-th case;

[0108] (5c) updating all parameters of the dual-domain contrastive learning network model through the back-propagation mechanism according to the loss function L;

[0109] (5d) Use the validation set to evaluate the dual-domain contrastive learning network model and save the model parameter weights with the best index;

[0110] (5e) Repeat steps (5a) to (5d) until the training is completed.

[0111] like Figure 5As shown in the figure, the dual-domain contrast learning network model can achieve accurate classification in the task of renal cancer subtype classification in MRI images and has high evaluation indicators.

[0112] In summary, the present invention comprehensively utilizes T1-weighted and T2-weighted MRI images, effectively segments the kidney area through a segmentation network, and accurately characterizes the discriminant features of the tumor appearance; the present invention adopts a contrastive learning scheme based on intra-case consistency and inter-case specific constraints, which effectively aligns the features of different modal images in the same case and enhances the differences between different category features. Specifically, by introducing consistency constraints, it is ensured that images of different modalities can align their corresponding features in the same case, and through specific constraints, the features between different cases can maintain sufficient differences, thereby improving the classification accuracy; the present invention generates information-rich enhanced samples through a linear diffusion strategy, thereby effectively counteracting the problem of network performance attenuation that may be caused by cross-dataset applications. By generating enhanced samples and combining with the linear diffusion strategy, the impact of differences between data sets can be alleviated to a certain extent, and the generalization ability of the model can be enhanced; the present invention effectively improves the accuracy and stability of kidney tumor classification tasks through multimodal image fusion, contrastive learning constraints and enhanced sample generation strategies, and has strong practical application value.

Claims

1. A method for classifying renal cancer MRI images based on contrastive learning, characterized by: The method comprises the following steps in order: (1) Obtain a renal cancer subtype dataset and the corresponding kidney region annotations, and use the kidney region annotations as the segmentation label Y s , divide the renal cancer subtype dataset into training set, validation set and test set; (2) inputting the training set into a segmentation network to perform kidney region segmentation and ROI extraction to obtain a ROI region of interest; the segmentation network includes an encoder and a decoder; (3) Performing data enhancement on the extracted ROI region of interest using a linear diffusion strategy to obtain an enhanced MRI image; (4) constructing a dual-domain contrastive learning network model, which includes a dual-stream network and a dual-domain contrastive learning loss function; (5) inputting the enhanced MRI image into the dual-domain contrast learning network model for training to obtain a trained dual-domain contrast learning network model; (6) After data enhancement, the MRI image to be classified is input into the trained dual-domain contrast learning network model to obtain the final renal cancer classification result.

2. The method for renal cancer MRI image classification based on contrastive learning according to claim 1, characterized in that: Step (2) specifically includes the following steps in order: (2a) Input the MRI images in the training set into the segmentation network to obtain the kidney region segmentation mask prediction result P s ; The segmentation network adopts the MedicalNet segmentation network, and the segmentation network MedicalNet includes an encoder and a decoder; (2b) According to the segmentation label Y s And the kidney region segmentation mask prediction result P s Construct cross entropy loss function; (2c) All parameters of the MedicalNet segmentation network are updated through the back-propagation mechanism according to the cross-entropy loss function; (2d) Use the validation set to evaluate the MedicalNet segmentation network and save the model parameter weights with the best indicators, i.e. the optimal model weights; (2e) Repeat steps (2a) to (2d) until the training is completed; (2f) Load the optimal model weights of the MedicalNet segmentation network; (2g) Input all MRI images in the training set, validation set, and test set into the MedicalNet segmentation network to obtain segmentation masks; (2h) Calculate the minimum bounding box of the kidney in all segmentation masks; (2i) Increase the minimum bounding box by 5px along the x, y, and z directions to extract the ROI region of interest.

3. The method for renal cancer MRI image classification based on contrastive learning according to claim 1, characterized in that: Step (3) specifically includes the following steps in order: (3a) Initialize the T1-weighted MRI images and T2-weighted MRI images of all cases in the training set, and set the ROI of the image to the starting image x0 of the diffusion process; (3b) Set the total time step T and the maximum noise value β of the diffusion process; (3c) Noise generation and scaling according to the current time step t; (3d) By adding the scaled noise δ t Add to the starting image x0 to get the diffused image x t , where the noise δ t The expression is: The diffused image x t The expression is: x t =x0+δ t e Among them, ε~N(0,I) is standard Gaussian noise; (3e) The current time step t is incremented by 1; (3f) Repeat steps (3c) to (3e) until the current time step t is equal to the total time step T, the diffusion is completed, and a series of enhanced MRI images are obtained.

4. The method for renal cancer MRI image classification based on contrastive learning according to claim 1, characterized in that: Step (4) specifically includes the following steps in order: (4a) The dual-stream network includes a T1 weighted image branch, a T2 weighted image branch and a third fully connected layer, the T1 weighted image branch and the T2 weighted image branch have the same structure, a pre-trained encoder is obtained, the pre-trained encoder for processing T1 weighted MRI images, the first adaptive average pooling layer, and the first fully connected layer constitute the T1 weighted image branch, and the pre-trained encoder for processing T2 weighted MRI images, the second adaptive average pooling layer, and the second fully connected layer constitute the T2 weighted image branch; the first fully connected layer, the second fully connected layer, and the third fully connected layer have the same structure; (4b) For patient i, a two-stream network is used to extract features from the T1-weighted MRI image. Feature extraction from T2-weighted MRI images (4c) Extract features through adaptive average pooling operation Perform pooling; (4d) The pooled features are mapped to a low-dimensional space through the first fully connected layer and the second fully connected layer to form a learnable prototype representation. and (4e) Using prototype representation and Construct the intra-case consistency constraint loss function L intra-case : Where N is the number of samples, τ is the temperature constant; (4f) The prototype is represented as and Splice to form a fusion feature vector g i ; (4g) Use the fused feature vector g i Construct case-specific constraint loss function L inter-case : In the formula, δ [·] is an indicator function. When the conditions in [·] are met, δ [·] is equal to 1, otherwise δ [·] is equal to 0; i, j, k all represent the index value of the sample, y i is the category label of sample i, y j is the category label of sample j, g j and g k Represent the fused feature vectors of samples j and k respectively; (4h) Constructing the dual-domain contrastive learning loss function L DCL =L intra-case +L inter-case ; (4i) The fused feature vector g i Input into the third fully connected layer to calculate the final kidney cancer classification result.

5. The method for renal cancer MRI image classification based on contrastive learning according to claim 1, characterized in that: Step (5) specifically includes the following steps in order: (5a) Obtain the enhanced MRI image and the corresponding renal cancer subtype label Y, select the ROI region of interest of the T1-weighted MRI image and the T2-weighted MRI image of one group of cases, and input them into the dual-domain contrast learning network model to obtain the classification results. (5b) The classification results and renal cancer subtype label Y are input into the loss function L, which includes the cross entropy loss function L CLS And the dual-domain contrast learning loss function L DCL : L=L CLS +L DCL in, N is the number of samples, C is the number of subtype categories, τ is the temperature constant, δ [·] is an indicator function, which is equal to 1 when the condition in [·] is met, and equal to 0 otherwise; i is the true label of the ith case, Predict the classification label for the i-th case; (5c) updating all parameters of the dual-domain contrastive learning network model through the back-propagation mechanism according to the loss function L; (5d) Use the validation set to evaluate the dual-domain contrastive learning network model and save the model parameter weights with the best index; (5e) Repeat steps (5a) to (5d) until the training is completed.

Images