A Classification Method for Pathological Tissue Images of Colorectal Cancer Based on Active Learning

Abstract

This invention discloses a method for classifying colorectal cancer tumor pathological tissue images based on active learning. The method includes the following steps: collecting colorectal cancer tissue images and filtering out unqualified images; dividing all images into training, validation, and test sets; performing image-level annotation on a very small portion of colorectal cancer pathological images in the training set; dividing all images into training, validation, and test sets; establishing a deep learning model; initializing labeled and unlabeled pools, and training the network using samples from the labeled pool; using the trained model to predict samples from the unlabeled pool; and selecting samples that meet the query function conditions for annotation. This invention utilizes a deep neural network based on active learning to classify colorectal cancer tumor pathological tissue images, thus solving, to some extent, the problem of computer-aided colorectal cancer pathological diagnosis.

Description

Technical Field

[0001] This invention relates to the application of pathological images in the field of computer-aided medical diagnosis of colorectal cancer, and more specifically, to a method for classifying colorectal cancer pathological tissue images based on active learning. Background Technology

[0002] Colorectal cancer (CRC) is a common malignant tumor of the digestive tract, ranking third in incidence and second in mortality among all cancers. It has become a major public health problem threatening human health. If not diagnosed and treated early, the malignancy of colorectal cancer will continue to worsen, causing immense physical and psychological suffering and ultimately threatening the patient's life. Histopathological diagnosis is the most reliable diagnostic method and is considered the "gold standard" for confirming cancer.

[0003] Traditional pathological diagnosis mainly relies on doctors' visual analysis and experience in tissue sections, which is therefore subjective and uncertain, and the diagnostic results of different doctors may vary greatly.

[0004] Intelligent classification of various tissues in CRC pathological images can quantify the tumor microenvironment, assist in pathological diagnosis, and help pathologists diagnose diseases more objectively and accurately.

[0005] 1. Tumor pathology image tissue classification: In recent years, with the development of deep learning technology, deep learning has been widely used in pathological diagnosis. The purpose of patch-based classification of colorectal cancer tissue images is similar to that of pixel-based segmentation, both aiming to quantify the various components in the tumor immune microenvironment to assist pathologists in diagnosing colorectal cancer. However, the annotation workload behind these training data is enormous, and most studies on tumor pathology image classification and segmentation have not considered the annotation cost.

[0006] 2. Convolutional Neural Network (CNN): A convolutional neural network is a type of deep neural network widely used in image, speech, and natural language processing. Its core is the convolutional layer, which can effectively extract features from data such as images. It has characteristics such as translation invariance and partial invariance. Using CNN to extract features from tumor tissue images and intelligently classify tissue images can quickly and accurately detect various tissue types in images, including normal tissue and tumor tissue. This helps pathologists better understand the condition and improve the accuracy and reliability of diagnosis. Due to the heterogeneity of tumor pathology images, their image features are complex and diverse, which increases the difficulty for the network to extract pathological features, making it difficult to guarantee the generalization ability of the model.

[0007] 3. Active-Learning Algorithms: Active-Learning (AL) is a machine learning method that selects the most representative samples for annotation in each training iteration to improve the model's learning efficiency and performance. Its aim is to train deep neural network models with the lowest possible annotation cost and obtain high-accuracy classification models. In active learning, the model typically uses a query function to select a subset of samples from unlabeled data and submits them to annotators for labeling. Active learning can reduce annotation costs and training time, providing reasonable support for balancing model generalization ability and data requirements. However, current active-learning algorithms for tumor pathological tissues suffer from poor performance due to the following shortcomings: the query function is not designed according to the specific properties of pathological images, and current methods neglect the utilization of unlabeled data information.

[0008] In summary, existing intelligent classification algorithms for pathological tissues based on deep neural networks rely on a large amount of labeled data, resulting in excessively high costs for pathological labeling. Secondly, current active learning algorithms are not suitable for colorectal cancer pathological images and cannot achieve efficient and accurate automatic tissue classification when the number of labels is low. Summary of the Invention

[0009] The purpose of this invention is to overcome the shortcomings and deficiencies of traditional pathological diagnosis and existing computer-aided diagnostic technologies, and to provide a computer-aided diagnostic technology based on deep neural networks. This technology enables accurate and automatic classification of colorectal cancer pathological tissues under conditions of limited label quantity, thereby quantifying the tumor microenvironment and assisting pathologists in diagnosing colorectal cancer. To address the problem that model performance in colorectal cancer pathological tissue image classification tasks is limited by the number of labels, an active learning algorithm is designed. This algorithm selects samples based on feature representation, allowing doctors to label samples in a targeted manner: samples with high training value are manually labeled, while samples easily identifiable by the model are automatically pseudo-labeled based on the model's prediction results. Simultaneously, a diversity query function enriches the diversity of training data distribution, and an information content query function ensures the training value of the training data for the model, thereby reducing the model's need for a large number of labeled samples. Ultimately, this achieves accurate and automatic classification of colorectal cancer pathological tissue images under the premise of limited label quantity.

[0010] To achieve the above objectives, the present invention provides the following technical solution:

[0011] A method for classifying pathological tissue images of colorectal cancer based on active learning, characterized by comprising the following steps:

[0012] S1: Collect pathological tissue images of colorectal cancer, including labeled images and unlabeled images; initialize the unlabeled pool and the labeled pool; put the labeled images into the labeled pool and the unlabeled tissue images into the unlabeled pool; and establish a deep neural network model.

[0013] S2: Train the deep neural network model using the images in the labeled pool;

[0014] S3: Save the trained deep neural network model, filter the unlabeled images according to sample diversity and sample average entropy, manually label the filtered images, remove the manually labeled images from the unlabeled pool and add them to the labeled pool, filter the samples whose model prediction probability confidence is higher than a set threshold, use the class predicted by the model as the pseudo label of the sample, remove the pseudo-labeled images from the unlabeled pool and add them to the labeled pool;

[0015] S4: Update the deep neural network model using the images in the labeled pool;

[0016] S5: When the number of manually labeled images in the labeled pool reaches a preset threshold, the updated deep neural network model is obtained; when the number of manually labeled images in the labeled pool does not reach the preset threshold, return to step S3.

[0017] S6: Classify colorectal cancer pathological images using the updated deep neural network model.

[0018] Furthermore, in step S2, training the deep neural network model using the images in the labeled pool includes the following steps:

[0019] S2.1: First, randomly scale the images in the labeled pool, and then normalize the images in the labeled pool;

[0020] S2.2: Randomly crop a fixed-size tissue image from the labeled pool, and use the cropped tissue image from the labeled pool as input to the deep neural network model to obtain the prediction result;

[0021] S2.3: Use an optimizer to update and calculate the network parameters that affect the training and output of the deep neural network model, so that they approach or reach the optimal values ​​of the network parameters, thereby minimizing the loss function of the deep neural network model and obtaining a trained deep neural network.

[0022] S2.4: The tissue images collected in step S1 are fed into the trained deep neural network model one by one to obtain the segmentation results, prediction results and evaluation metrics of the true labels. Based on this, the accuracy of the current deep neural network model on the validation set is calculated. When the accuracy is the highest, the current deep neural network model is saved as the deep neural network model that has passed the validation.

[0023] Furthermore, in step S3, the unlabeled images are filtered based on sample diversity, the filtered images are manually labeled, and then the manually labeled images are removed from the unlabeled pool and added to the labeled pool, including the following steps:

[0024] S3.01: Input the images in the unlabeled pool into the deep neural network model obtained in step S2, filter out the candidate set according to the prediction results, and obtain the image features of the candidate set;

[0025] S3.02: Extract the image features of the labeled pool using the deep neural network model obtained in step S2, establish a clustering model accordingly, and calculate the center of each cluster in the clustering space through clustering mapping;

[0026] S3.03: Using the clustering model, map the obtained candidate set image features to the clustering space, and calculate the distance from each candidate set image feature to its nearest cluster center;

[0027] S3.04: Select multiple images from the candidate set images that have the largest distance from the image feature map to its nearest cluster center, manually label them, remove the manually labeled images from the unlabeled pool, and add them to the labeled pool.

[0028] Further, in step S3.01, the unlabeled pooled image is input into the deep neural network model obtained in step S2, and the candidate set is selected based on the prediction results, specifically as follows:

[0029]

[0030] P θ (y|x) represents the probability that the deep neural network model with parameter θ will predict x as y. The category with the highest confidence level. For the category with the second highest confidence, when the sample satisfies When the value is less than a fixed threshold τ, the sample x is... i Add it to the candidate set C.

[0031] Furthermore, in step S3.02, the calculation of the center of each cluster in the clustering space through clustering mapping specifically involves:

[0032]

[0033] The coordinates of the tissue image in the cluster space are f(l) j )=(x lj ,y lj Let the category be g (g∈{0,1,2,…,8}, gt j Let lj be the label value of the labeled image, and g be the number of labeled data in category g. g Calculate the coordinates of the center of cluster g;

[0034] Further, in step S3.03, the minimum distance from the sample to each cluster is calculated as the magnitude of the diversity of the tissue image, specifically as follows:

[0035]

[0036] c k The coordinates c under the clustering mapping k This represents a sample in the candidate set C.

[0037] Further, in step S3, unlabeled tissue images are filtered based on the average entropy of the tissue images, the filtered tissue images are manually labeled, and then the manually labeled tissue images are removed from the unlabeled pool and added to the labeled pool, including the following steps:

[0038] S3.11: Perform data augmentation on the tissue image of the unlabeled pool, the data augmentation including rotation, random flipping, random scaling and random cropping to a fixed size and contrast adjustment;

[0039] S3.12: Input the tissue images after data augmentation in the unlabeled pool into the deep neural network model, and calculate the average entropy of each tissue image;

[0040] S3.13: Select the multiple tissue images with the highest average entropy, manually label the multiple tissue images with the highest average entropy, remove them from the unlabeled pool, and add them to the labeled pool;

[0041] Furthermore, in step S3.12, the formula for calculating entropy is as follows:

[0042]

[0043] p(x i ) is the predicted probability for each tissue image, where n is the number of categories.

[0044] Furthermore, step S3 also includes filtering unlabeled images using a threshold.

[0045] Furthermore, the process of using a threshold to filter unlabeled images specifically involves:

[0046] The trained deep neural network model is used to predict the images in the unlabeled pool. Multiple tissue images with a maximum predicted probability distribution greater than τ are selected, and the tissue images are removed from the unlabeled pool and added to the labeled pool. The prediction of the deep neural network model is used as the pseudo-label of the tissue image.

[0047] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0048] To overcome the difficulty of obtaining labels for deep learning model training in deep learning-based colorectal cancer pathological tissue image classification methods, this invention proposes an active learning method that integrates a sample selection strategy that combines information content and diversity. This method aims to select and label the most critical samples for model training, thereby reducing the model's need for labels. Addressing the issue of poor sample quality due to the specific characteristics of pathological images, two active learning query functions are designed based on the characteristics of CRC tumor pathological images. These query functions select representative samples with high information content for labeling and training the deep model, thus improving the quality of samples selected by active learning. This reduces the amount of labeled training data while more accurately classifying colorectal cancer tumor pathological tissue images. To overcome the problem that the performance of the model trained by the active learning algorithm is limited by the amount of labeled data, a semi-supervised module is added to the active learning process. Pseudo-labeling improves the model's utilization of unlabeled data, further optimizing the model and improving its classification accuracy. Attached Figure Description

[0049] Figure 1 This is a flowchart of a method for classifying pathological tissue images of colorectal cancer based on active learning, provided by the present invention.

[0050] Figure 2 This is a flowchart of the training process of the deep neural network model provided in an embodiment of the present invention;

[0051] Figure 3 These are example images of nine types of colorectal cancer pathological tissues provided in embodiments of the present invention;

[0052] Figure 4 This is a diagram of a colorectal cancer pathological tissue image classification framework based on active learning provided in an embodiment of the present invention;

[0053] Figure 5 This is a flowchart of the process for screening samples based on diversity, provided in an embodiment of the present invention.

[0054] Figure 6 This is a flowchart of the process of filtering samples based on the amount of information provided in an embodiment of the present invention. Detailed Implementation

[0055] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent.

[0056] To better illustrate this embodiment, some parts in the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions;

[0057] It will be understood by those skilled in the art that certain well-known structures and their descriptions may be omitted in the accompanying drawings;

[0058] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0059] Example 1

[0060] This embodiment provides a method for classifying colorectal cancer pathological tissue images based on active learning, such as... Figure 1 As shown, it includes the following steps:

[0061] S1: Collect pathological tissue images of colorectal cancer, including labeled images and unlabeled images; initialize the unlabeled pool and the labeled pool; put the labeled images into the labeled pool and the unlabeled tissue images into the unlabeled pool; and establish a deep neural network model.

[0062] S2: Train the deep neural network model using the images in the labeled pool;

[0063] S3: Save the trained deep neural network model, filter unlabeled tissue images according to sample diversity and sample average entropy, manually label the filtered tissue images, remove the manually labeled images from the unlabeled pool and add them to the labeled pool. In addition, filter samples whose model prediction probability confidence is higher than a set threshold, use the class predicted by the model as the pseudo label of the sample, remove the unlabeled images from the unlabeled pool and add them to the labeled pool.

[0064] S4: Update the deep neural network model using the images in the labeled pool;

[0065] S5: When the number of manually labeled images in the labeled pool reaches a preset threshold, the updated deep neural network model is obtained; when the number of manually labeled images in the labeled pool does not reach the preset threshold, return to step S3.

[0066] S6: Classify colorectal cancer pathological tissue images using the updated deep neural network model.

[0067] The pseudocode for the overall iterative process is as follows:

[0068] While m i

[0069] Model = Train(Model, L) i / / Train the prediction model using data from the labeled pool;

[0070] / / 1. Diversity Screening

[0071] D i ​=SelectByDiversity(Model,U i (,n); / / Sample selection based on diversity

[0072] For set D i The samples in the sample were manually labeled;

[0073] U i =U i-1 -D i ,L i =L i-1 +D i / / Update unlabeled pools and labeled pools

[0074] / / 2. Information filtering

[0075] I i =SelectByUncertainty(Model,U i (,n); / / Sample selection based on information content

[0076] For set I i The samples in the sample were manually labeled;

[0077] U i =U i-1 -I i ,L i =L i-1 +I i / / Update unlabeled pools and labeled pools

[0078] / / 3. Pseudo-tag filtering

[0079] T i =SelectByThreshold(Model,U i (,n); / / Simple sample selection based on threshold

[0080] PseudoLabeling(Model,T i / / Use model predictions as pseudo-labels to label T i Samples in

[0081] U i =U i-1 -T i ,L i =L i-1 +T i / / Update unlabeled pools and labeled pools

[0082] End While

[0083] Symbol explanation: i represents the current round, L im represents the labeled pool in round i. i L represents i The number of samples in U i Let represent the unlabeled pool in round i, Model represent the prediction model, B represent the expected total number of labels, p represent the initial number of labels, and n represent the number of samples to be screened.

[0084] In this embodiment, initially there is a very small initial set of labeled samples: Its number is M, and it has a large unlabeled sample pool. The quantity is N; such as Figure 4 As shown, and M < <N,y i For sample x i The classification labels, in each algorithm iteration, are derived from D. u Select a batch of data D q ; Where b represents the top b samples that best meet the selection function criteria. Model is a deep model trained using a labeled set, designed through active learning, and includes a sample query function; subsequently, D... l and D u It will be updated, using D l The model is retrained using samples from the labeled set D; this process iterates until the labeled set D is reached. l The process terminates when the size reaches the stated budget B.

[0085] In this iterative process, this invention designs two query functions based on the characteristics of CRC pathological images to filter samples for annotation and model training; respectively corresponding to Figure 5 and Figure 6 Diversity screening and information content screening; In each round of active learning iteration, the goal is to select a batch of unlabeled samples of size n according to the screening strategy and manually label these samples. In the i-th round, these selected samples will be taken out from the unlabeled pool U of this round and added to the labeled pool L.

[0086] Example 2

[0087] This embodiment, based on Embodiment 1, continues to disclose the following content:

[0088] Training the deep neural network model using tissue images from the labeled pool includes the following steps:

[0089] S2.1: First, randomly scale the images in the labeled pool, and then normalize the images in the labeled pool;

[0090] S2.2: Randomly crop a fixed-size image from the labeled pool, and use the cropped image from the labeled pool as the input to the deep neural network model to obtain the prediction result;

[0091] S2.3: Use an optimizer to update and calculate the network parameters that affect the training and output of the deep neural network model, so that they approach or reach the optimal values ​​of the network parameters, thereby minimizing the loss function of the deep neural network model and obtaining a trained deep neural network.

[0092] S2.4: The tissue images collected in step S1 are fed into the trained deep neural network model one by one to obtain the segmentation results, prediction results and evaluation metrics of the true labels. Based on this, the accuracy of the current deep neural network model on the validation set is calculated. When the accuracy is the highest, the current deep neural network model is saved as the deep neural network model that has passed the validation.

[0093] The following example uses a dataset containing at least 10,000 CRC tumor pathology tissue images to illustrate this point:

[0094] Figure 2 The dataset consists of at least 10,000 CRC tumor pathological tissue images collected for this embodiment.

[0095] Step (1): First, collect at least 10,000 CRC tumor pathology tissue images as a dataset;

[0096] Step (2): Screen out unqualified tissue images, and screen out images in which the tissue type cannot be identified due to objective conditions;

[0097] Step (3) involves image-level annotation. Image-level labeling refers to determining the type of tissue contained in a given CRC tumor pathology image. The annotation method involves placing images of the same category in the same folder and naming the folder after the image category. Training, validation, and test sets are created. The training set contains approximately 8 / 10 images, the validation set approximately 1 / 10, and the test set approximately 1 / 10. The images in the validation and test sets are labeled, while the images in the training set are unlabeled. 100 images from the training set are randomly selected for manual annotation.

[0098] Step (4): Initialize the labeled pool and the unlabeled pool. Add labeled images to the labeled pool and unlabeled images to the unlabeled pool. The labeled pool and the labeled pool are used to record labeled samples and unlabeled samples, respectively.

[0099] Step (5): Build a deep neural network model, using any popular deep learning model, such as the Desnet121 network model.

[0100] Step (6), the training and validation phase, such as Figure 3 As shown in the figure, this is the entire training and validation process. The steps to optimize the network during one round of training are as follows:

[0101] Step (a): In one round of training, the model is trained using data from the labeled pool;

[0102] Step (b) involves performing online data augmentation on this batch of images. First, the images are randomly scaled, then normalized, then randomly cropped to a fixed size, and then cropped again to the same position and size as the last sampled image.

[0103] Step (c) involves using these images as input to the model to obtain the prediction results;

[0104] Step (d) uses the optimizer SGD to update and compute the network parameters that affect model training and model output, making them approach or reach the optimal value, thereby minimizing the loss function and finding the optimal model;

[0105] Step (e): Validation set validation. Each image in the validation set is fed into the trained model to obtain the segmentation results. The evaluation metrics of the prediction results and the true labels are calculated. The ACC of the current model on the validation set is calculated. If the current ACC is the highest, the current model is saved.

[0106] Step (f) involves iteratively optimizing the deep neural network model, continuously minimizing the cross-entropy loss, and improving the network's classification performance.

[0107] Example 3

[0108] This embodiment, based on Embodiment 1, continues to disclose the following content:

[0109] like Figure 5 As shown, unlabeled tissue images are filtered based on sample diversity, the filtered tissue images are manually labeled, and then the manually labeled tissue images are removed from the unlabeled pool and added to the labeled pool. The process includes the following steps:

[0110] S3.01: Input the images in the unlabeled pool into the deep neural network model obtained in step S2, filter out the candidate set according to the prediction results, and obtain the image features of the candidate set;

[0111] S3.02: Extract the organizational features of the labeled pool using the deep neural network model obtained in step S2, establish a clustering model accordingly, and calculate the center of each cluster in the clustering space through clustering mapping;

[0112] S3.03: Using the clustering model, map the obtained candidate set image features to the clustering space, and calculate the distance from each image feature in the candidate set to its nearest cluster center;

[0113] S3.04: Select the 200 images with the largest distance from the image feature map to its nearest cluster center from the candidate set of organized images, manually label them, remove the manually labeled images from the unlabeled pool, and add them to the labeled pool;

[0114] The following section will use a dataset containing at least 10,000 CRC tumor pathology tissue images as an example to illustrate this point.

[0115] (i) Input the unlabeled pool samples into the model obtained in step (6), filter out the candidate set based on the prediction results, and obtain the features of the candidate set samples;

[0116] (ii) Extract sample features from the labeled pool using the model, and establish a UMAP clustering model accordingly. Calculate the center of each cluster in the clustering space through cluster mapping;

[0117] (iii) Using the UMAP model established in step (ii), map the features of the candidate set samples obtained in step (i) to the clustering space, and calculate the distance of each image feature in the candidate set to its nearest cluster center.

[0118] (iv) Based on the calculation in step (iii), manually label the 200 samples with the largest distance from the image feature mapping to the nearest cluster center in the candidate set samples, remove them from the unlabeled pool, and add them to the labeled pool.

[0119] Example 4

[0120] This embodiment, based on Embodiment 1, continues to disclose the following content:

[0121] The unlabeled pooled tissue image is input into the deep neural network model obtained in step S2, and the candidate set is selected based on the prediction results, specifically as follows:

[0122]

[0123] This represents the probability that the deep neural network model with parameter θ will predict x as y. The category with the highest confidence level. For the category with the second highest confidence, when the sample satisfies When the value is less than a fixed threshold of 0.25, the sample x is... i Add to the candidate set C;

[0124] The calculation of the center of each cluster in the cluster space through cluster mapping is specifically as follows:

[0125]

[0126] The coordinates of the tissue image in the cluster space are f(l) j )=(x lj ,y lj ),like Figure 3 As shown, let the category be g (g∈{0,1,2,…,8}, gt j For the labeled image l j The label value, the number of labeled data in category g is n g Calculate the coordinates of the center of cluster g;

[0127] The minimum distance from a sample to each cluster is calculated as the magnitude of the diversity of the tissue image, specifically as follows:

[0128]

[0129] c k The coordinates c under the clustering mapping k This represents a sample in the candidate set C.

[0130] like Figure 6 As shown, unlabeled tissue images are filtered based on the average entropy of the tissue images, the filtered tissue images are manually labeled, and then the manually labeled tissue images are removed from the unlabeled pool and added to the labeled pool. The process includes the following steps:

[0131] S3.11: Perform data augmentation on the image of the unlabeled pool, the data augmentation including rotation and random flipping, random scaling and random cropping to a fixed size, and contrast adjustment;

[0132] S3.12: Input the samples in the unlabeled pool and their data-enhanced images into the deep neural network model to calculate the average entropy of each tissue image;

[0133] S3.13: Select the images with the highest average entropy, manually label the 200 images with the highest average entropy, remove them from the unlabeled pool, and add them to the labeled pool.

[0134] Example 5

[0135] This embodiment, based on Embodiment 1, continues to disclose the following content:

[0136] Figure 6 As shown, two data augmentation methods were used on the original image and the sample information was measured more comprehensively and reliably by calculating their average entropy: (1) flipping and random flipping (2) random scaling and random cropping to a fixed size (3) contrast adjustment;

[0137] Each sample x is input into the unlabeled pool U i For the two data-enhanced samples, the corresponding predicted probability distributions are obtained, and the entropy values ​​are calculated and averaged as a measure of information content. Assuming the total number of categories is n, the entropy calculation formula is:

[0138]

[0139] Then we can obtain the information content of each sample in all unlabeled pools, select the top 200 samples with the highest average entropy from these samples as the key samples with the highest information content in this round of screening, remove these samples from the unlabeled pool, manually label these samples, and add them to the labeled pool;

[0140] In step S3.12, the formula for calculating entropy is as follows:

[0141]

[0142] p(x i ) is the predicted probability for each tissue image, where n is the number of categories.

[0143] Step S3 further includes using a threshold to filter unlabeled tissue images.

[0144] The process of using a threshold to filter unlabeled tissue images specifically involves:

[0145] The trained deep network model is used to predict the tissue images in the unlabeled pool. No more than 400 tissue images with a maximum predicted probability distribution greater than 0.95 are selected, and the tissue images are removed from the unlabeled pool and added to the labeled pool. The predicted category of the deep neural network model is used as the pseudo-label of the tissue image.

[0146] Unlabeled images are filtered based on sample diversity, the principle of which is as follows: Figure 5As shown. The model is trained using images from the unlabeled pool to obtain prediction results. Samples meeting the criteria are placed into a candidate pool as a candidate set. Samples from the labeled pool are input into a feature extractor to obtain feature maps. The flatten function is used to obtain corresponding feature vectors. These feature vectors are used to train a UMAP clustering model to establish a clustering map. The candidate set is input into a feature extractor to obtain feature maps. The flatten function is used to obtain corresponding feature vectors. The trained UMAP clustering model is used to find the multiple samples farthest from the nearest cluster center. These samples are labeled and moved from the unlabeled pool to the labeled pool.

[0147] Unlabeled images are filtered based on the average entropy of the samples, the principle of which is as follows: Figure 6 As shown, unlabeled images undergo three operations: rotation and random flipping, random scaling and random cropping to a fixed size, and contrast adjustment. These are then input into a feature extractor to obtain feature maps. The flatten function is used to obtain the corresponding feature vectors, which are then passed through a convolutional layer to obtain a probability distribution. The average entropy is calculated from the probability distribution, and the images with the highest average entropy are selected for labeling and moved from the unlabeled pool to the labeled pool.

[0148] This invention conducts experiments on the publicly available colorectal cancer pathological image dataset NCT-CRC-HE-100K using the Densenet121 deep network model. The ACC index for image classification is used to evaluate the experimental results. The experimental results of this invention compared with other active learning methods are shown in Table 1, and the ablation experiments are shown in Table 2.

[0149] Table 1 Experimental Results

[0150]

[0151]

[0152] Table 2 Ablation Experiment Results

[0153]

[0154] The same or similar labels correspond to the same or similar parts;

[0155] The terms used to describe positional relationships in the accompanying drawings are for illustrative purposes only and should not be construed as limiting this patent.

[0156] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for classifying colorectal cancer histopathological images based on active learning, characterized in that, The method comprises the following steps: S1: collecting colorectal cancer pathological tissue images, the colorectal cancer pathological tissue images comprising labeled images and unlabeled images, initializing an unlabeled pool and a labeled pool, putting the labeled images into the labeled pool and the unlabeled images into the unlabeled pool, and establishing a deep neural network model; S2: training the deep neural network model with the images in the labeled pool; S3: saving the trained deep neural network model, screening the unlabeled images according to sample diversity and sample average entropy, manually labeling the screened images, removing the manually labeled images from the unlabeled pool and adding the manually labeled images to the labeled pool; S4: updating the deep neural network model with the images in the labeled pool; S5: when the number of manually labeled images in the labeled pool reaches a preset threshold, obtaining the updated deep neural network model; when the number of manually labeled images in the labeled pool does not reach the preset threshold, returning to step S3; S6: classifying colorectal cancer pathological images by using the updated deep neural network model; In step S3, the unlabeled images are screened according to sample diversity, the screened images are manually labeled, and the manually labeled images are removed from the unlabeled pool and added to the labeled pool, comprising the following steps: S3.01: inputting the images in the unlabeled pool into the deep neural network model obtained in step S2, screening a candidate set according to the prediction result, and obtaining image features of the candidate set; S3.02: extracting tissue features of the labeled pool by using the deep neural network model obtained in step S2, establishing a clustering model according to the tissue features, and calculating the center of each cluster in the clustering space through clustering mapping; S3.03: mapping the obtained image features of the candidate set to the clustering space through the clustering model, and calculating the minimum distance of each candidate set image feature to the nearest cluster center; S3.04: manually labeling multiple images with the largest distance from the nearest cluster center from the candidate set image features, removing the manually labeled images from the unlabeled pool, and adding the manually labeled images to the labeled pool.

2. The method of claim 1, wherein the method is based on active learning. In step S2, the deep neural network model is trained with the images in the labeled pool, comprising the following steps: S2.1: first randomly scaling the images in the labeled pool, and then normalizing the images in the labeled pool; S2.2: randomly cropping the images in the labeled pool to a fixed size, taking the cropped images in the labeled pool as inputs of the deep neural network model, and obtaining a prediction result; S2.3: updating and calculating network parameters affecting the training and output of the deep neural network model by using an optimizer, so as to approximate or reach an optimal value of the network parameters, thereby minimizing a loss function of the deep neural network model, and further obtaining a trained deep neural network. S2.4: Put the images collected in step S1 into the trained deep neural network model one by one to obtain the evaluation indexes of segmentation results, prediction results and true labels, and calculate the accuracy of the current deep neural network model on the validation set according to the evaluation indexes, and when the accuracy is the highest, save the current deep neural network model as the validated deep neural network model. 3.The method of claim 1, wherein, In step S3.01, the images in the unlabeled pool are input into the deep neural network model obtained in step S2, and the candidate set is selected according to the prediction results, specifically: the deep neural network model with parameters will predict the probability that the sample is . the deep neural network model with parameters will predict the probability that the sample is . is the class with the largest confidence, is the class with the second largest confidence, and when a sample satisfies is less than a fixed threshold , the sample is added to the candidate set .

4. The method of claim 1, wherein the method is based on active learning. In step S3.02, the center of each cluster in the clustering space is calculated according to the clustering mapping, specifically: label values of the annotated images label values of the annotated images coordinates of the class cluster centers.

5. The method of claim 4, wherein the method is based on active learning. In step S3.04, the distance of each image in the candidate set from its nearest cluster center is calculated as a measure of image diversity, specifically: representing coordinates under the cluster mapping, representing a candidate set of samples.

6. The method of claim 1, wherein the method is based on active learning. In step S3, the unlabeled images are screened according to the average entropy of the images, the screened images are manually labeled, and then the manually labeled images are removed from the unlabeled pool and added to the labeled pool, including the following steps: S3.11: Data augmentation is performed on the images in the unlabeled pool, and the data augmentation includes rotation, random flipping, random scaling, random cropping to a fixed size, and contrast adjustment; S3.12: The images in the unlabeled pool after the data augmentation are input into the deep neural network model, and the average entropy of each image is calculated; S3.13: A plurality of images with the largest average entropy are selected, the plurality of images with the largest average entropy are manually labeled, and the plurality of images are removed from the unlabeled pool and added to the labeled pool.

7. The method of claim 3, wherein the method is based on active learning. In step S3.12, the formula for calculating the entropy is specifically: For each image, the prediction probability, n is the number of classes.

8. The method of claim 1 to 7, wherein, In step S3, it also includes using a threshold to screen the unlabeled images.

9. The method of claim 8, wherein the method is based on active learning. The use of a threshold to screen the unlabeled tissue images is specifically: predicting the images of the unlabeled pool with the trained deep network model, filtering out no more than 2n images whose predicted probability distribution maximum value is greater than adding the images to the labeled pool, and using the prediction of the deep neural network model as a pseudo label for the image.

Images