A breast cancer image multi-task classification method and system based on unbalanced data

Abstract

The application discloses a breast cancer image multi-task classification method and system based on unbalanced data, which firstly acquires ultrasound breast cancer images and CT images, constructs a data set, and performs data enhancement. Secondly, a balanced loss function is used to process different numbers of breast cancer categories, and the number of breast cancer images of different categories is input as a weight into the loss function. Then, a feature extraction module is constructed, the binary classification and six classification networks are jointly trained and decoupled, the features extracted by the two training are fused for classification, and the enhanced images are used for testing, and the benign and malignant judgment results of the lesion area in the breast cancer image are output. The application can obtain a better model faster with less data amount, and make the network model more accurate in feature extraction.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of medical image classification, and particularly relates to a breast cancer image multi-task classification method and system based on unbalanced data. BACKGROUND

[0002] Breast cancer has become the most common malignant tumor, and early screening of breast cancer can improve the early diagnosis rate and reduce the mortality rate, so it is urgent to improve the early screening level. Breast cancer can be detected by X-ray examination, ultrasonic examination, magnetic resonance imaging, computed tomography (CT), port emission tomography (PET), breast temperature measurement and other methods. Early diagnosis of breast cancer can reduce mortality and increase survival rate, and early intervention can be carried out through accurate detection and diagnosis of malignant tumors to reduce mortality. However, the sensitivity of imaging examination for more comprehensive evaluation of cancerous regions and identification of cancer subtypes is low. Histopathological imaging by breast biopsy can also accurately identify cancer subtypes and accurately locate lesions. However, this manual examination by pathologists is inefficient and prone to errors. Therefore, it is necessary to automate the method of breast cancer diagnosis.

[0003] At present, researchers have proposed a variety of artificial intelligence-based early screening algorithms for breast cancer. Transferring a pre-trained model on ImageNet to fine-tune a breast lump classification model is a common method for small or medium-sized data. Some researchers convert BUS images into feature maps with BI-RADS features. Feature maps are used to classify breast tumors under semi-supervised learning, and to reconstruct lesion classification guided feature maps under unsupervised learning, or propose a hierarchical feature aggregation branch to fuse the features of the two branches and perform reasoning, and design two identical CNN trunks to identify malignant tumors and solid nodules respectively, and use the class activation maps generated by the two CNN trunks to guide each other. In the above methods, the pre-training based on ImageNet does not take into account the problem of sample balance of the data set, and only uses BI-RADS or two CNNs for training, and the features learned by the model are very single, and the breast cancer image multi-task classification method and system based on unbalanced data solves the above problems.

[0004] In fact, convolutional neural networks encounter several challenges in the process of implementation on medical data sets, such as:

[0005] (1) Small amount of data. Due to the consideration of protecting the privacy of patients when collecting medical data, it is difficult to collect sufficient data, and the data of some institutions are not publicly released. If the amount of data is small, the generalization of the model when training will be poor, resulting in overfitting phenomenon.

[0006] (2) The number of multiple samples is unbalanced, the number of categories of samples is unbalanced, the model performs better on the category with more samples, and the performance on the category with less samples is poor, and finally a result that cannot correctly represent the accuracy is output.

[0007] (3) The data dimension is single, most medical images only have picture information and do not carry text information, and it is very difficult to obtain a pathological report.

[0008] (4) The time and hardware cost used for training the network is too high, the convolutional neural network often has millions of parameters, and it is difficult to popularize to low-level devices. SUMMARY

[0009] The application provides a breast cancer image multi-task classification method based on unbalanced data, to solve the problem of poor generalization caused by unbalanced categories of medical data. A multi-task training strategy is proposed, which simulates the judgment of doctors in reality according to the shape, size and aspect ratio information of the tumor when diagnosing the tumor by image, so that the model can fully learn the correlation between the lesion and the information around the lesion, to improve the accuracy of the model for benign and malignant classification of the tumor. The application can obtain reliable classification results on breast cancer images, effectively assist doctors in tumor judgment, and improve the efficiency of doctor diagnosis.

[0010] The purpose of the application is achieved by the following technical scheme: a breast cancer image multi-task classification method based on unbalanced data, comprising the following steps:

[0011] S1, obtain ultrasound breast cancer images and CT images, and construct a data set. The ultrasound images and CT images provide two label files, one is the benign and malignant label of the tumor obtained by biopsy, and the other is the BI-RADS evaluation label of the tumor level judged by doctors according to experience, the data set is put into 6 folders according to the 6 levels of BI-RADS, called data set 1, and the 0-5 labels are assigned, and the data set is put into two folders according to the two levels of benign and malignant, called data set 2, the benign tumor is assigned 0 label, and the malignant tumor is assigned 1 label;

[0012] S2, each breast cancer image and CT image obtained in step S1 is subjected to fuzzy, horizontal flip, random cutting, and single gray image is changed into three, finally different forms of the same breast cancer picture are obtained, data enhancement is carried out, and the generalization ability of the model is improved;

[0013] S3, use a balanced loss function to process different numbers of breast cancer categories to balance the learning ability of the model to each category, and the number of different categories of breast cancer images is input into the loss function as a weight to adjust the learning ability of the model to different breast cancer categories;

[0014] S4, construct a feature extraction module, jointly train the two-class and six-class networks, and decouple the two-class and six-class networks, take the joint training as task 1, take the decoupling training as task 2, and finally fuse the features extracted by the two tasks for classification to improve the performance of the two-class classification;

[0015] S5, using the image obtained in S2, training the network model obtained in S4, generating a six-class network labeled with BI-RADS and a two-class network labeled with benign and malignant for breast cancer, and fusing the features of the two models for joint model training, taking the two-class classification in joint training as a generator and the six-class classification as a discriminator, training a generative adversarial network (GAN) model, and connecting the trained joint model and GAN model with a two-class classification head, fine-tuning the classification head (freezing the weight of a two-class classification head) ;

[0016] S6, obtaining a breast cancer image with a label, inputting the network model trained in S5, and outputting the benign and malignant judgment result of the lesion area in the breast cancer image.

[0017] In step S1, in step S1, the specific method of constructing labels according to the model requirements for different patient ultrasound images or CT images is: according to the BI-RADS classification levels 2, 3, 4A, 4B, 4C, 5, the labels assigned to these six levels are 0, 1, 2, 3, 4, and 5. According to the benign and malignant classification, the benign tumor is labeled as 0 and the malignant tumor is labeled as 1.

[0018] In step S2, the specific method of processing each breast cancer image is: according to the lesion center of the breast cancer image, a uniform size grayscale image is obtained by cropping, Gaussian noise is used to deblur the image, then center cropping and random flipping are performed, and the single-channel grayscale image is converted into a three-channel image.

[0019] In step S3, the specific method of preprocessing is: adjusting the model learning ability according to the balance loss function for different patient ultrasound images or CT images is: counting the number of all categories in the data set as the weight of the balance loss function.

[0020] In step S4, the processed lesion area image is sent to the feature extraction module: the feature extraction module is a DenseNet121 network or a SwinTransformer_Tiny network; the parameters of the network model are randomly initialized, the architecture of the two-class model and the six-class model is CNN or Transformer network, the connection Flatten layer, the activation function is LeakyReLU, the dense layer with l2 parameter, the dropout layer, the activation function is softmax, and the output layer with 2 neurons.

[0021] In step S5, taking DenseNet121 as an example, the specific steps are as follows:

[0022] 5.1: Input the folder storing the data based on benign and malignant status and the CSV file storing the image names and labels into the DenseNet121 model for training;

[0023] 5.2: Input the folder containing the BI-RADS dataset and the CSV file containing the image names and labels into the DenseNet121 model for training;

[0024] 5.3: Connect the DenseNet121 model to two classification heads. Classification head 1 is for binary classification, and classification head 2 is for six-class classification. Both classification heads share a DenseNet121 feature extraction model. Send the data and the CSV file containing the data name, data binary classification label, and data six-class classification label to the model for training;

[0025] 5.4: Use the binary classification of the DenseNet121 model in the joint model as the generator, and the six-class classification head of the DenseNet121 model as the discriminator to jointly train a DenseNet121 feature extractor.

[0026] 5.5: Combine the joint model trained in step 3 and the GAN model trained in step 4 to train a binary classification head. During training, freeze the weights of the joint model and the GAN model and only fine-tune the classification head to improve the performance of the binary classification.

[0027] The specific process of fine-tuning the classification head is as follows:

[0028] (1): Send the binary classification data into the model to train the binary classification model, and send the six-category data into the model to train the six-category model;

[0029] (2): Connect the classification head of the binary classification model and the classification head of the six-classification model to the same CNN model, and send the binary classification data and the six-classification data together into the connected model for joint model training;

[0030] (3): The binary classification model in the joint model is used as the generator, and the six-class classification head in the joint model is used as the discriminator to train the GAN model;

[0031] (4): Combine the joint model and the GAN model, freeze the weights and fine-tune the classification head of a binary classification model.

[0032] A breast cancer image multi-task classification system based on unbalanced data, comprising: a data processing module, a balanced loss module, a binary classification module, a six-classification module, a joint training module, a GAN module and a fine-tuning module.

[0033] The data processing module is used for processing the size of the image data to adapt to the input of the model, and pre-processing the image data to improve the generalization performance of the model.

[0034] The balanced loss module uses a balanced loss function to balance the learning ability of the model for different numbers of classes.

[0035] The binary classification module extracts features from the pre-processed image data and performs binary classification.

[0036] The six-classification module extracts features from the pre-processed image data and performs six-classification.

[0037] The joint training module jointly trains the binary classification and six-classification modules to extract more classification information.

[0038] The GAN module decouples the training of the binary classification and six-classification modules to extract more classification information.

[0039] The fine-tuning module fine-tunes a classification head together with the joint training module and the GAN module, combines the feature extraction capabilities of the two, and performs classification output.

[0040] In the joint training module, a network is trained together using the binary classification and six-classification modules to extract more feature information as task one.

[0041] In the GAN module, a network is decoupled and trained together using the binary classification and six-classification modules to extract more feature information as task two.

[0042] In the fine-tuning module, a network is trained together using task one and task two to extract more feature information.

[0043] In the data processing module, the patient's breast cancer image or CT image is cropped, flipped, blurred, and the specific method of data processing is: reading batch size data from the data set, cropping to a uniform size, adding Gaussian noise blur after cropping, and then converting the one-channel grayscale image to three channels.

[0044] For each patient's grayscale image obtained in S2, the number of each class in the binary classification and six-classification is counted as the weight of the balanced loss function, and the balanced loss function is added to the cross-entropy loss function when used.

[0045] Compared with the prior art, the advantages of the present application are:

[0046] (1) The present invention proposes a multi-task classification method and system for breast cancer images based on imbalanced data. By using a balanced loss function to train the DenseNet121 model and the classification head, the common problems of data imbalance and small data volume in medical imaging data are solved. With less data, a better performing model can be obtained through training more quickly.

[0047] (2) The present invention adopts a multi-task training network model, so that the same feature extraction subject can fully learn the relationship between different tasks. Compared with the existing technology that only uses two-category or six-category networks, according to the characteristics of the convolutional neural network architecture, we use multi-task training to avoid the problem of insufficient feature extraction due to a single task, thereby making the network model more accurate in feature extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is an overall flow chart of the method proposed in the present invention;

[0049] Figure 2 This is a partial architecture diagram of the binary classification model feature extractor used in the present invention;

[0050] Figure 3 This is a partial architecture diagram of the six-category model feature extractor used in the present invention;

[0051] Figure 4 This is a partial architecture diagram of the joint model feature extractor used in the present invention;

[0052] Figure 5 This is a partial architecture diagram of the GAN model feature extractor used in this invention. DETAILED DESCRIPTION

[0053] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the present disclosure is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0054] like Figure 1 As shown in FIG, a multi-task classification method for breast cancer images based on imbalanced data includes the following steps:

[0055] S1. Obtain ultrasound breast cancer images and CT images. The datasets are divided into six folders (called Dataset 1) according to the six BI-RADS grades, and assigned labels ranging from 0 to 5. The datasets are divided into two folders (called Dataset 2) according to the two grades of benign and malignant, and assigned labels ranging from 0 to 1.

[0056] S2, using blur, horizontal flip, random cropping, etc. on each breast cancer image obtained in step S1, and changing the single-channel grayscale image into 3 channels, finally obtaining different forms of the same breast cancer image, and improving the generalization ability of the model;

[0057] S3, using a balanced loss function to process different numbers of breast cancer categories to balance the learning ability of the model for each category, and inputting the number of breast cancer images of different categories as weights into the loss function to adjust the learning ability of the model for different breast cancer categories;

[0058] S4, using a CNN or Transformer model to construct a feature extraction module, as shown in Figure 2 and Figure 3 , the two-class and six-class networks are jointly trained, the two-class and six-class networks are decoupled, the joint training is taken as task 1, and the decoupled training is taken as task 2, and finally the features extracted by the two tasks are fused for classification to improve the performance of two-class classification;

[0059] S5, using the images obtained in S2 to train the network model obtained in S4, generating a six-class network with BI-RADS as the label and a two-class network with benign and malignant as the label for breast cancer, and fusing the features of the two models for joint model training, as shown in Figure 4 , the two-class classification in the joint training is taken as a generator, and the six-class classification is taken as a discriminator, and a GAN model is trained, as shown in Figure 5 , the trained joint model and GAN model are connected with a two-class classification head, and the weight is frozen to train a two-class classification head;

[0060] S6, obtaining breast cancer images with labels, inputting the network model obtained in S5, and outputting the benign and malignant judgment results of the lesion area in the breast cancer image.

[0061] A breast cancer image multi-task classification system based on unbalanced data, comprising:

[0062] A data processing module for processing ultrasound breast cancer images and CT images to adapt to the input of the model and pre-process the data to improve the generalization performance of the model;

[0063] A balanced loss module using a balanced loss function to balance the learning ability of the model for different numbers of categories;

[0064] A two-class module using a CNN or Transformer to construct a feature extraction module for two-class classification;

[0065] A six-class module using a CNN or Transformer to construct a feature extraction module for six-class classification;

[0066] Joint training module, combining the two-class classification model with the six-class classification model to extract more classification information;

[0067] The GAN module decouples the binary and six-classification models to extract more classification information;

[0068] Fine-tuning module, which fine-tunes a classification head together with the joint training module and the GAN module to combine the feature extraction capabilities of the two;

[0069] In the joint training module, a network is trained jointly using a binary classification model and a six-class classification model to extract more feature information as task one;

[0070] In the GAN module, the two-class classification and six-class classification models are decoupled and jointly trained into a network to extract more feature information, which serves as Task 2.

[0071] In the fine-tuning module, a network is trained together using the task one and task two models to extract more feature information.

[0072] Example:

[0073] A multi-task classification method for breast cancer images based on imbalanced data includes the following steps:

[0074] S1. Obtain ultrasound breast cancer images and CT images. Place the datasets into six folders (called Dataset 1) based on the six BI-RADS grades and assign labels ranging from 0 to 5. Place the datasets into two folders (called Dataset 2) based on the two grades of benign and malignant, and assign labels ranging from 0 to 1.

[0075] In this embodiment, the original ultrasound data in step S1 is based on the GDPH&SYSUCC dataset, and the original CT data is based on the CBIS-DDSM dataset. GDPH&SYSUCC is a PNG format file, and CBIS-DDSM is a Dicom format file. The relevant labels of the data will be written into a CSV file for use during training.

[0076] Taking the GDPH&SYSUC dataset as an example, this dataset is a grayscale image, so it needs to be expanded to three channels before entering the model. To ensure the generalization of the model, random data enhancement will be performed before training.

[0077] S2. For each breast cancer image obtained in step S1, data enhancement such as blurring, horizontal flipping, and random cropping is performed, and the single-pass grayscale image is converted into a three-pass image. Ultimately, different representations of the same breast cancer image are obtained, improving the generalization ability of the model.

[0078] In this embodiment, the training set data is randomly blurred by Gaussian noise, center cropped, and finally converted into Tensor form to be sent to the model for training. No data augmentation operation is performed when testing with the test set to more objectively test the performance of the model.

[0079] The feature extraction model used in this embodiment is based on DenseNet121. The input data size of the non-convolutional layer of the network model constructed based on this needs to be consistent. For example, the pictures used in the model are all RGB three-channel color images with a size of 224*224 pixels. Therefore, the channel number of the input data needs to be consistent with that of the original model. The original data is a single-channel grayscale image. To input the network, the single channel needs to be copied to other channels to form a three-channel image.

[0080] S3, use a balanced loss function to process different numbers of breast cancer categories to balance the learning ability of the model for each category. The number of different categories of breast cancer images is input as a weight into the loss function to adjust the learning ability of the model for different breast cancer categories.

[0081] In this embodiment, the number of all categories in the data set is counted as the weight of the balanced loss function.

[0082] S4, use CNN or Transformer model to construct feature extraction module, joint training of two classification and six classification network, decoupling training of two classification and six classification network, joint training as task 1, decoupling training as task 2, finally fusion of features extracted by two tasks for classification to improve the performance of two classification.

[0083] In this embodiment, AdamW optimizer is used to optimize the network in the training stage. Only the learning rate is set to 1e-4. Other parameters use the default parameters in the DenseNet121 framework. The loss function is the binary cross entropy function (Binary_Crossentropy) plus the balanced loss function. In addition, early stopping method (Early-Stopping) is used to avoid overtraining. Early stopping method automatically monitors the indicators in network training. If the indicators are not optimized for a certain period of time, early stopping method considers that the network model has been optimized and stops further training. This method well avoids the overfitting problem caused by overtraining. Early stopping method is based on ACC. When ACC is no longer updated, it means that the optimal model has been reached and further training is not needed.

[0084] S5, using the image obtained in S2 to train the network model obtained in S4, generating a six-classification network model labeled with BI-RADS and a two-classification network model labeled with benign and malignant for breast cancer, and training a joint model by fusing the features of the two models, taking the two-classification in the joint training as a generator and the six-classification as a discriminator, training a GAN model, and connecting the trained joint model and GAN model with a two-classification classification head to freeze the weight and train a two-classification classification head.

[0085] In this embodiment, DenseNet121 is taken as an example. The folder storing data based on benign and malignant and the CSV file storing picture names and labels are input into the DenseNet121 model for training. The model is connected with a binary classification head to output various indexes of binary classification, such as ACC, SPE, REN, AUC, PRE, F1-SCORE, ACC (Accuracy), i.e. the percentage of correctly predicted results in total samples; SPE (Specificity), i.e. specificity, which represents the ratio of correctly classified negative samples to all true negative samples, and measures the recall rate of negative samples of the classifier, i.e. the ability to correctly identify negative examples. High specificity indicates that the classifier less incorrectly predicts negative examples as positive examples; REN (recall) is the probability of being predicted as 1 sample in actual 1 sample; AUC (Area Under the Curve) is a commonly used index for evaluating the performance of a binary classification model, which is the area under the ROC curve (Receiver Operating Characteristic curve); PRE (Precision) is the probability of actual 1 sample in all predicted 1 samples; F1-SCORE combines the precision (Precision) and recall (Recall) of the model to measure the classification accuracy and omission rate of the model. The folder storing the data set based on BI-RADS and the CSV file storing picture names and labels are input into the DenseNet121 model for training. The model is connected with a six-class classification head to output various indexes of six-class classification, such as MAP (i.e. the average of AP0-AP5), AP0 (the acc of BI-RADS level 0 category), AP1 (the acc of BI-RADS level 1 category), AP2 (the acc of BI-RADS level 2 category), AP3 (the acc of BI-RADS level 3 category), AP4 (the acc of BI-RADS level 4 category), and AP5 (the acc of BI-RADS level 5 category). The DenseNet121 model is connected with two classification heads, the first classification head is binary classification, and the second classification head is six-class classification. The two classification heads share one DenseNet121 feature extractor model. The data and the CSV file storing data names, binary classification labels and six-class classification labels are input into the model for training. The binary classification of the DenseNet121 model in the joint model is taken as a generator, and the six-class classification head of the DenseNet121 model is taken as a discriminator to jointly train a DenseNet121 feature extractor.The trained joint model in step 3 and the GAN model in step 4 are jointly trained with a binary classification head, and the weights of the joint model and the GAN model are frozen during training, and only the classification head is fine-tuned to improve the binary classification performance.

[0086] S6, obtaining the breast cancer image with the label, inputting the network model obtained in S5, and outputting the benign or malignant judgment result of the lesion area in the breast cancer image.

[0087] Table 1 shows the experimental results.

[0088]

[0089] The above is a further detailed description of the present application in combination with specific / preferred embodiments, and cannot be regarded as limiting the specific implementation of the present application to these descriptions. For ordinary skilled persons in the technical field to which the present application belongs, several alternatives or modifications can be made to the described embodiments without departing from the concept of the present application, and these alternatives or modifications should be regarded as belonging to the protection scope of the present application.

[0090] The part not described in detail in the present application is the technology known to those skilled in the art.

Claims

1. A multi-task classification method for breast cancer images based on imbalanced data, characterized by: Comprise the following steps: S1, obtain ultrasound breast cancer images and CT images, and construct a data set; In the data set, according to the grade of BI-RADS evaluation label, it is divided into six categories, and according to the benign and malignant, it is divided into two categories; S2, data enhancement is carried out on each breast cancer image and CT image obtained; S3, using balanced loss function to process different number of breast cancer categories, and taking the number of different categories of breast cancer images as weight input into the loss function; S4, construct a feature extraction module, jointly train and decouple train the two classification and six classification networks, the joint training as task 1, the decoupling training as task 2, and the features extracted from the two tasks are fused for classification; The feature extraction module is DenseNet121 network or SwinTransformer_Tiny network, the parameters of the network are randomly initialized, the architecture of the two classification model and the six classification model is CNN or Transformer network, the connection Flatten layer, the activation function is LeakyReLU, the dense layer with l2 parameter, the drop layer, the activation function is softmax, and the output layer with 2 neurons; S5, using the images obtained in S2 to train the network model obtained in S4, the specific process is as follows: Using the images obtained in S2 to train the network model obtained in S4, generating a six classification of breast cancer with BI-RADS as label and a two classification of breast cancer with benign and malignant as label network model, and fusing the features of the two models for joint model training, taking the two classification in joint training as generator and the six classification as discriminator, training a GAN model, connecting the trained joint model and GAN model with a two classification classification head, fine tuning the classification head, freezing the weight training a two classification classification head; S6, obtaining breast cancer images with labels, inputting the network model trained in S5, and outputting the benign and malignant judgment result of the lesion area in the breast cancer image.

2. The breast cancer image multi-task classification method based on unbalanced data according to claim 1, characterized in that, In S1, the ultrasound breast cancer images and CT images provide two label files, one is the benign and malignant label of tumor obtained by biopsy, and the other is the BI-RADS evaluation label of tumor level obtained by doctors according to experience, according to the six levels of BI-RADS, the data set is put into six folders, called data set 1, and the labels 0-5 are assigned, according to the two levels of benign and malignant, the data set is put into two folders, called data set 2, the benign tumor is assigned 0 label, and the malignant tumor is assigned 1 label. 3.The breast cancer image multi-task classification method based on unbalanced data according to claim 1, characterized in that, In S2, the data enhancement specifically includes: according to the lesion center in the breast cancer image, the uniform size gray image is obtained by center cropping, the image is deblurred by using Gaussian noise, then center cropping and random flipping are carried out, and the single channel gray image is converted into three channels. 4.The breast cancer image multi-task classification method based on unbalanced data according to claim 1, wherein, In S3, it also includes: for each patient's gray image obtained in S2, the number of each category in the two classification and six classification is counted as the weight of the balanced loss function, and the balanced loss function is added with the cross entropy loss function when used.

5. The breast cancer image multi-task classification method based on unbalanced data according to claim 4, characterized in that, The specific process of fine tuning the classification head is: S5.1: The binary classification data is input into the model to train the binary classification model, and the six classification data is input into the model to train the six classification model; S5.2: The classification head of the binary classification model and the classification head of the six classification model are connected to the same CNN model, and the binary classification data and the six classification data are input into the connected model for joint model training; S5.3: The binary classification model in the joint model is used as the generator, and the classification head of the six classification in the joint model is used as the discriminator for GAN model training; S5.4: The joint model and the GAN model are combined, and the weight is frozen to fine-tune the classification head of a binary classification model.

6. A breast cancer image multi-task classification system based on unbalanced data, for implementing the classification method of any one of claims 1 to 5, characterized in that, The system comprises a data processing module, a balanced loss module, a binary classification module, a six classification module, a joint training module, a GAN module and a fine-tuning module; The data processing module is used for pre-processing the ultrasound breast cancer image and CT image data; The balanced loss module uses a balanced loss function to balance the learning ability of the model for different numbers of classes; The binary classification module extracts features from the pre-processed image data for binary classification; The six classification module extracts features from the pre-processed image data for six classification; The joint training module jointly trains the binary classification and six classification modules to extract classification information; The GAN module decouples the training of the binary classification and six classification modules to extract classification information; The fine-tuning module fine-tunes the joint training module and the GAN module together to output the classification.

Images