Colorectal cancer pathological image processing and classifying method based on transfer learning

Abstract

The invention discloses a colorectal cancer pathological image processing and classification method based on transfer learning. The method comprises the following steps: S1, constructing a preliminary model; s2, performing data processing on the original pathological section image to obtain a clinical data set; s3, carrying out second transfer learning and model testing on a clinical data set obtained after processing and the preliminary model to obtain a final model; and S4, inputting data, performing automatic processing on the data, performing classification and detection on the image by utilizing the final model, and performing data result display on the bottom layer model. According to the scheme, transfer learning is performed on the model by using data acquired by hospitals in an actual clinical environment, and the generalization ability of the network is enhanced; through training on real data, the method can better adapt to a clinical application scene, and improves the applicability and robustness in an actual environment.

Description

Technical Field

[0001] The present invention relates to the field of clinical diagnosis technology, and in particular to a colorectal cancer pathology image processing and classification method based on transfer learning. Background Art

[0002] Colorectal cancer (CRC) is one of the common malignant tumors in the gastrointestinal tract, the third most common cancer in the world, and the second leading cause of cancer-related death. In China, the incidence of CRC has risen rapidly and has become the third leading cause of cancer-related death. From 2015 to 2020, the number of cases and deaths was close to 550,000, accounting for about 30% of the total cancer deaths. When colorectal cancer is diagnosed in the early stage, the 5-year survival rate is about 90%, while the survival rate is only 13% when diagnosed in the late stage. Therefore, early diagnosis plays a vital role in the prevention and treatment of CRC. The main ways of CRC screening include colonoscopy, pathological diagnosis and imaging diagnosis. However, these traditional screening methods are often cumbersome and time-consuming, and the diagnostic results are easily affected by differences in observer experience and variable factors between different observers, resulting in inconsistent diagnostic results. Among these screening methods, pathological diagnosis is considered to be the key link in the diagnosis of CRC. However, the global shortage of pathologists is in sharp contrast to the demand for pathologists. This situation has become an important factor restricting the efficiency of early diagnosis of CRC in small and medium-sized hospitals. Many small hospitals need to rely on the professional support of large hospitals for pathological section preparation and diagnosis, which not only increases the workload of pathologists in large hospitals, but also affects the timely diagnosis of CRC, further delaying the treatment and recovery of patients. Therefore, improving the automation and accuracy of pathological diagnosis and achieving early diagnosis of CRC in resource-constrained medical environments have become key issues that need to be addressed.

[0003] In recent years, deep learning has made significant progress in the medical field, especially in medical image processing. At present, although many classification methods have achieved good classification performance on the CRC public dataset, they have not been applied to the clinical field. The reasons are as follows: (1) The complexity of the classification model is relatively high, and it is difficult to run on mobile devices or even ordinary devices; (2) The obvious difference between the public dataset and clinical data makes it difficult for the classification model to maintain good classification performance. Summary of the invention

[0004] In response to the above technical problems, the present invention provides a colorectal cancer pathology image processing and classification method based on transfer learning to improve the efficiency of early diagnosis of CRC.

[0005] The present invention is implemented by adopting the following technical scheme: a colorectal cancer pathology image processing and classification method based on transfer learning, comprising the following steps: Step S1: construct a preliminary model; Step S2: Processing the original pathological slice image to obtain a clinical data set; Step S3: The processed clinical data set is used to perform a second transfer learning and model test with the preliminary model to obtain the final model; Step S4: After inputting the data and automatically processing the data, the final model is used to classify and detect the image, and the data results are displayed in the underlying model.

[0006] Specifically, the step S1 includes the following sub-steps: Step S11: data division, selecting a public colorectal cancer tissue pathology section data set, and dividing the data into a training set, a validation set, and a test set according to a preset ratio; Step S12: Network construction, based on the PyTorch deep learning framework, using MobileNet-V1 as the backbone network to build the model, and using Kather-5K to pre-train the model to obtain a pre-trained model; Step S13: Network training: the first transfer learning of the pre-model is performed on the data set. The model is iteratively trained on the training set and the validation set. When the loss function value on the validation set reaches the optimal value, the early stopping mechanism guides the model to end the training and save the optimal model parameters. Step S14: Model testing, importing the test set into the trained model for evaluation, calculating the accuracy index, completing a complete model training based on the public data set, and obtaining a preliminary model.

[0007] Specifically, the data processing in step S2 includes: fully automatically cropping and sampling the original pathological slice image containing the diseased tissue, matching it with the image of the public data set in a unified format, and performing data enhancement operations.

[0008] Specifically, the second transfer learning in step S3 includes: loading the preliminary model, freezing the parameters of the low-level feature extraction layer in the model, unfreezing the parameters of the high-level feature extraction layer, and adjusting them according to the characteristics of the new data during the training process.

[0009] Specifically, the step S3 model test specifically includes: Cases were selected from the clinical data set for testing. Based on the known pathological diagnosis information of the patients, the original images were cropped and sampled, and sub-images were generated and then imported into the model. The model predicts the tissue category of each sub-graph and determines whether the patient has colorectal cancer by summarizing the prediction results; The prediction results were compared with the actual lesion conditions of the patients, and the detection indicators based on the patient level were calculated to verify the performance of the model on clinical data.

[0010] Specifically, the automated processing of input data in step S4 specifically includes: Step A1: remove most of the black pixel areas in the image, and then perform the first sampling to clean up the remaining black pixels; Step A2: automatically determine the actual size of the image and crop the original pathological slice image into multiple pathological slice small blocks; Step A3: Mark the traces of the second sampling in the first sampling image, save it as a copy, and return the detection result; save the preprocessed image file to the specified path to provide input data for subsequent classification and recognition tasks.

[0011] Specifically, the pathological slice small block size of the image cropped in step A2 is 224×224, and also includes: the parts of the last column and the last row that cannot be rounded are sampled forward starting from the rightmost or bottommost side to obtain a complete 224×224 small block.

[0012] Specifically, the image classification and detection specifically includes: taking the file path of the generated pathological slice small blocks as input, calling a pre-trained deep learning classification model to predict and classify each small block, and the classification model classifies each slice small block according to tissue type.

[0013] Specifically, the tissue types include: adipose tissue ADI, lymphocytes LYM, normal colon mucosa NORM, background BACK, mucus tissue MUC, tumor-associated stromal tissue STR, debris DEB, smooth muscle tissue MUS and colorectal adenocarcinoma epithelial tissue TUM.

[0014] Specifically, the prediction classification includes the following sub-steps: Step B1: The image is resized and rescaled to the fixed size required by the network to fit the size of the input layer; Step B2: pixel standardization, standardizing the mean and variance of each channel; Step B3: The preprocessed image is sent to the model for forward propagation. The model uses the feature representation learned during the training process to predict the classification task of the input image. Based on the features of the input image, a series of convolution operations, pooling operations, and fully connected layer calculations are performed, and finally the prediction score for each category is output; Step B4: Take the category with the highest probability as the predicted category and return the corresponding predicted probability.

[0015] The beneficial effect of the present invention is that the present invention uses the data collected by the hospital in the actual clinical environment to transfer learning to the model and enhance the generalization ability of the network. Through training on real data, the model can better adapt to clinical application scenarios, thereby improving its applicability and robustness in the actual environment; it can adaptively crop images and perform patch sampling, thereby reducing the huge workload of doctors in the data processing link and improving the accuracy of detection. This method is also applicable to digital images generated by microscope scanning, and has high versatility and adaptability; this method incorporates the clinical data collected by the hospital into the transfer learning process, and performs secondary optimization on the model that has been transferred from the NCT-CRC-HE-100K data set. After the training is completed, by comparing the performance of the test set in the transfer learning of clinical data sets using the pre-trained model VGG16, ResNet-50, GoogleNet, the MobileNet-V1 model trained only on NCT-CRC-HE-100K, and the MobileNet-V1 model after two transfer learning, the results show that the transfer learning strategy of this experiment significantly improves the applicability of the model on clinical data, verifying the feasibility and effectiveness of the method. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0017] Figure 1 It is a flow chart of a colorectal cancer pathology image processing and classification method based on transfer learning in an embodiment of the present invention; Figure 2 Schematic diagram of the fine-tuning strategy for transfer learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0019] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0020] The following is combined with Figure 1 , some embodiments of the present invention are described in detail. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0021] The present invention proposes a colorectal cancer pathology image processing and classification method based on transfer learning. In a preferred embodiment, the method is as follows: Figure 1 As shown, the following steps are included: Step S1: construct a preliminary model; Step S2: Processing the original pathological slice image to obtain a clinical data set; Step S3: The processed clinical data set is used to perform a second transfer learning and model test with the preliminary model to obtain the final model; Step S4: After inputting the data and automatically processing the data, the final model is used to classify and detect the image, and the data results are displayed in the underlying model.

[0022] In this embodiment, step S1 specifically includes the following sub-steps: Step S11: data division, selecting a public colorectal cancer tissue pathology section data set, and dividing the data into a training set, a validation set, and a test set according to a preset ratio; Step S12: Network construction, based on the PyTorch deep learning framework, using MobileNet-V1 as the backbone network to build the model, and using Kather-5K to pre-train the model to obtain a pre-trained model; Step S13: Network training: the first transfer learning of the pre-model is performed on the data set. The model is iteratively trained on the training set and the validation set. When the loss function value on the validation set reaches the optimal value, the early stopping mechanism guides the model to end the training and save the optimal model parameters. Step S14: Model testing, importing the test set into the trained model for evaluation, calculating the accuracy index, completing a complete model training based on the public data set, and obtaining a preliminary model.

[0023] In this embodiment, step S2 of data processing includes: fully automatic cropping and sampling of the original pathological slice image containing the diseased tissue, matching with the image of the public data set in a unified format, and performing data enhancement operations.

[0024] In this embodiment, the second transfer learning in step S3 includes: loading the preliminary model, freezing the parameters of the low-level feature extraction layer in the model, unfreezing the parameters of the high-level feature extraction layer, and adjusting them according to the characteristics of the new data during the training process.

[0025] In this embodiment, step S3 model testing specifically includes: Cases were selected from the clinical data set for testing. Based on the known pathological diagnosis information of the patients, the original images were cropped and sampled, and sub-images were generated and then imported into the model. The model predicts the tissue category of each sub-graph and determines whether the patient has colorectal cancer by summarizing the prediction results; The prediction results were compared with the actual lesion conditions of the patients, and the detection indicators based on the patient level were calculated to verify the performance of the model on clinical data.

[0026] In this embodiment, the automated processing of input data in step S4 specifically includes: Step A1: remove most of the black pixel areas in the image, and then perform the first sampling to clean up the remaining black pixels; Step A2: automatically determine the actual size of the image and crop the original pathological slice image into multiple pathological slice small blocks; Step A3: Mark the traces of the second sampling in the first sampling image, save it as a copy, and return the detection result; save the preprocessed image file to the specified path to provide input data for subsequent classification and recognition tasks.

[0027] In this embodiment, the pathological slice small block size of the image cropped in step A2 is 224×224, and also includes: the parts of the last column and the last row that cannot be rounded are sampled forward starting from the rightmost or bottommost side to obtain a complete 224×224 small block.

[0028] In this embodiment, image classification and detection specifically include: taking the file path of the generated pathological slice small blocks as input, calling a pre-trained deep learning classification model to predict and classify each small block, and the classification model classifies each slice small block according to tissue type.

[0029] In this embodiment, the tissue types include: adipose tissue ADI, lymphocytes LYM, normal colon mucosa NORM, background BACK, mucus tissue MUC, tumor-associated stromal tissue STR, debris DEB, smooth muscle tissue MUS and colorectal adenocarcinoma epithelial tissue TUM.

[0030] In this embodiment, the prediction classification includes the following sub-steps: Step B1: The image is resized and rescaled to the fixed size required by the network to fit the size of the input layer; Step B2: pixel standardization, standardizing the mean and variance of each channel; Step B3: The preprocessed image is sent to the model for forward propagation. The model uses the feature representation learned during the training process to predict the classification task of the input image. Based on the features of the input image, a series of convolution operations, pooling operations, and fully connected layer calculations are performed, and finally the prediction score for each category is output; Step B4: Take the category with the highest probability as the predicted category and return the corresponding predicted probability.

[0031] In another embodiment, colorectal cancer pathology image processing and classification based on transfer learning includes the following: 1. Preliminary model construction (1) Data division. We selected the publicly available colorectal cancer tissue pathology slide dataset NCT-CRC-HE-100K (containing 9 tissue categories and a total of 10,000 images) and divided the data into training, validation, and test sets at a ratio of 8:1:1. The image content was used as the model input, and the image file name was used as the category label to annotate the corresponding tissue.

[0032] (2) Network construction. Based on the PyTorch deep learning framework, MobileNet-V1 was selected as the backbone network to build the model. Kather-5K was used to pre-train the model to obtain a pre-trained model.

[0033] (3) Network training. The first transfer learning of the pre-model was performed on the NCT-CRC-HE-100K dataset. The cross entropy loss function was used to evaluate the fitting effect of the model, and the Adam optimizer was used to update the network parameters. The cosine annealing learning rate scheduling strategy was introduced to dynamically adjust the learning rate. This strategy dynamically adjusts the learning rate during the training process, which helps the model to converge stably in the later stage of training. The early stopping mechanism is then used to supervise the training process to prevent overfitting. The initial learning rate is set to 0.0001, the training round (epochs) is set to 100, and the tolerance range of the early stopping mechanism is 20 epochs. The model is iteratively trained on the training set and the validation set. When the loss function value on the validation set reaches the optimal value, the early stopping mechanism guides the model to end training and save the optimal model parameters.

[0034] (4) Model testing. The test set is imported into the trained model for evaluation, and objective indicators such as accuracy are calculated to complete a complete model training based on the public data set, thereby obtaining a preliminary model.

[0035] Compared with other mainstream large networks, MobileNet-V1 has a lighter size and a certain degree of efficiency and flexibility. At the same time, this network also achieves a balance between accuracy and parameter quantity. The following table shows the indicators of each model in the public dataset: Table 1 Performance of different models on NCT-CRC-HE100K

[0036] As shown in Table 1, FLOPs represents the amount of computation, Para represents the number of parameters, Acc represents the accuracy, Pre represents the precision, Rec represents the recall, and F1-score represents the F1 score. MobileNet-V1 (unpre) represents the MobileNet-V1 model that has not undergone transfer learning, and MobileNet-V1 (pre) represents the model that has not undergone transfer learning from the clinical dataset but has undergone transfer learning from the NCT-CRC-HE-100K dataset. VGG16, ResNet-50, and GoogleNet represent the models that have not undergone transfer learning.

[0037] As can be seen from the table, the accuracy of the MobileNEt-V1 (pre) model is 0.35% lower than that of GoogleNet, but the latter's computational workload and parameter count are 4.83 times and 6.75 times that of the former, respectively. The accuracy of the MobileNEt-V1 (pre) model is 1.33% higher than that of MobileNEt-V1 (unpre) and 0.13% higher than that of VGG16, but the computational workload and parameter count of the VGG16 model are 26.31 times and 41.58 times that of MobileNEt-V1 (pre), respectively. The accuracy of the MobileNEt-V1 (pre) model is 0.02% higher than that of ResNet-50, and the latter's computational workload and parameter count are 7 times and 7.28 times that of the former. Overall, the MobileNet-V1 (pre) model achieves a good balance between accuracy and computational efficiency, and is a more efficient choice, especially suitable for practical application scenarios with limited resources.

[0038] 2. Enhanced Model (1) Data processing. The original pathological slice images (including lesion tissue) taken by the microscope provided by the hospital were automatically cropped and sampled to generate sub-images (patches) of size 224×224, which were matched with the images of the public dataset in a unified format and data enhancement operations (including rotation, translation, flipping and adding noise) were performed. Finally, 8542 sub-images were obtained, of which the training set, validation set and test set contained 6833, 855 and 854 images respectively, providing data support for subsequent transfer learning.

[0039] (2) Transfer learning. At this stage, this solution loaded the MobileNet-V1 model that had been trained on the Kather-5K and NCT-CRC-HE-100K datasets. According to the results in Table 1, the model has good preliminary feature extraction capabilities. Based on this, this solution freezes the parameters of the low-level feature extraction layer in the model to ensure that it is not updated during the training process. For the high-level feature extraction layer, this solution chooses to unfreeze its parameters so that it can be adjusted according to the characteristics of the new data during the training process, so that the model can better adapt to the needs of new tasks. This strategy can effectively retain the general feature extraction capabilities while optimizing the task-specific high-level feature representation, thereby improving the task adaptability and performance of the model.

[0040] Table 2 Comparison of different fine-tuning of transfer learning

[0041] As shown in Table 2, Train_time represents the model training time, FLOPs represents the amount of calculation, Para represents the parameter amount (Parameter), Acc represents the accuracy (Accuracy), Pre represents the precision (Precision), Rec represents the recall rate (Recall), F1-score represents the F1 score, UnFreeze_All represents freezing all layers, UnFreeze_First represents freezing only the low-level feature extraction layers of the first few layers, and UnFreeze_No represents unfreezing all layers.

[0042] In Table 2, the training time of the UnFreeze_First model is 198.72 minutes longer than that of the UnFreeze_All model, but the classification accuracy of the former is 4.04% higher than that of the latter. In addition, the training time of the UnFreeze_First model is 139.45 minutes shorter than that of the UnFreeze_No model, and its classification accuracy is 1.85% higher than that of the latter. These results show that the fine-tuning strategy adopted in this experiment has significant feasibility and superiority in balancing training efficiency and model performance, saving computing resources while improving the adaptability of the model to new tasks.

[0043] (3) Model testing. Given the limited clinical data, 5 cases were selected from the clinical data set for testing (including 2 normal cases and 3 lesion cases). Based on the known pathological diagnosis information of the patient, the original image was cropped and sampled, and the sub-images were generated and imported into the model. The model predicts the tissue category of each sub-image and determines whether the patient has colorectal cancer by summarizing the prediction results. Finally, the prediction results were compared with the actual lesion conditions of the patient, and the detection indicators based on the patient level were calculated to verify the performance of the model on clinical data.

[0044] 3. Classification system background processing and detection process (1) Image preprocessing. The system automatically performs the preprocessing operation of the slice image by inputting the original image path of the pathological slice. Specifically, it includes the following steps: (1) Black pixel removal: First, most of the black pixel area in the image is removed, and then the first sampling is further performed to clean up the residual black pixels to ensure the integrity of the effective area. (2) Cutting and blocking (second sampling): According to the actual size of the image, the original pathological slice image is automatically judged to be cropped into multiple pathological slice small blocks of size 224×224. Among them, the parts that cannot be rounded off in the last column and the last row will be sampled forward from the rightmost or bottommost side to obtain a complete 224×224 small block. (3) Output storage: Mark the traces of the second sampling in the first sampling image and save it as a copy for the return of the detection result. The preprocessed image file is saved to the specified path to provide input data for subsequent classification and recognition tasks.

[0045] (2) Classification and detection. The system takes the file path of the pre-processed pathological slice blocks as input and calls the pre-trained deep learning classification model to predict and classify each small block. The classification model divides each slice block into the following 9 tissue types: adipose tissue (ADI), lymphocytes (LYM), normal colon mucosa (NORM), background (BACK), mucus tissue (MUC), tumor-associated stromal tissue (STR), debris (DEB), smooth muscle tissue (MUS), and colorectal adenocarcinoma epithelial tissue (TUM).

[0046] The prediction process first involves image preprocessing, which is a key step to ensure that the model can correctly parse the input image, including image resizing, normalization and other operations. Specifically, the image is first resized, usually rescaled to the fixed size required by the network to adapt to the size of the input layer; then, pixel normalization is performed, usually the mean and variance of each channel are normalized to ensure the consistency and stability of the model input. The preprocessed image is then sent to the model for forward propagation, during which the model uses the feature representation learned during training to predict the classification task of the input image. During the forward propagation process, the model performs a series of convolution operations, pooling operations, and fully connected layer calculations based on the features of the input image, and finally outputs the prediction score for each category. These prediction scores are converted into probability distributions of categories through the softmax function, where the softmax function exponentially normalizes the scores of each category and ensures that the sum of the probabilities of all categories is 1. Subsequently, the category with the highest probability is selected as the predicted category, and the corresponding prediction probability is returned. To improve detection accuracy and visualization, when the predicted probability of a category in a small block exceeds a preset threshold (except for the "TUM" category, the threshold for other categories is usually set to 0.8), the model will draw a bounding box on the first sampling copy and display the corresponding category label and its predicted probability in the box for subsequent analysis, diagnosis and archiving.

[0047] Based on the standards of public datasets, an automated image preprocessing method was developed to process microscope images taken by portable devices such as mobile phones in clinical environments. This method can adaptively crop images and perform patch sampling, which reduces the huge workload of doctors in the data processing link while improving the accuracy of detection. This method is also applicable to digital images generated by microscope scanning, and has high versatility and adaptability.

[0048] In this embodiment, in order to enhance the generalization ability of the model, the clinical data collected by the hospital are included in the transfer learning process, and the model that has been transferred from the NCT-CRC-HE-100K dataset is optimized twice. After the training is completed, by comparing the performance of the test set using the pre-trained model VGG16, ResNet-50, GoogleNet to transfer learning of clinical datasets, the MobileNet-V1 model trained only on NCT-CRC-HE-100K, and the MobileNet-V1 model after two transfer learnings, the results show that the transfer learning strategy of this experiment significantly improves the applicability of the model to clinical data, verifying the feasibility and effectiveness of the method. As shown in Table 3.

[0049] Table 3 Performance of different models on clinical datasets

[0050] As shown in Table 3, FLOPs represents the amount of computation, Para represents the number of parameters (Parameter), Acc represents the accuracy (Accuracy), Pre represents the precision (Precision), Rec represents the recall rate (Recall), and F1-score represents the F1 score. MobileNet-V1 (trans) represents the MobileNet-V1 model that has undergone clinical dataset transfer learning, and MobileNet-V1 (pre) represents the model that has not undergone clinical dataset transfer learning but has undergone NCt-CRC-HE-100K transfer learning. VGG16 (trans), ResNet-50 (trans), and GoogleNet (trans) represent the models after the pre-trained model has undergone transfer learning on the clinical dataset.

[0051] According to the experimental results, the MobileNet-V1 (pre) model performed poorly on the clinical dataset, with an accuracy 58.53% lower than that of the MobileNet-V1 (trans) model, indicating that the model has a significant performance improvement after transfer learning on the clinical dataset. Although the accuracy of the ResNet-50 (trans) model is 0.61% higher than that of the MobileNet-V1 (trans), the number of parameters and the amount of computation of the MobileNet-V1 (trans) model are only 7.28 times and 7 times that of the ResNet-50 (trans) model, respectively, demonstrating the better computational efficiency of the MobileNet-V1 (trans) model. In contrast, the accuracy of the VGG16 model is comparable to that of the MobileNet-V1 (trans) model, but its number of parameters and computation are 41.58 times and 26.3 times that of the MobileNet-V1 (trans) model, respectively, indicating that the MobileNet-V1 (trans) model is more efficient in terms of computing resource consumption. The accuracy of the GoogleNet (trans) model is 1.54% lower than that of the MobileNet-V1 (trans) model, and its parameter volume and computational complexity are 6.75 times and 4.83 times that of the latter, respectively, further demonstrating the computational efficiency advantage of MobileNet-V1 (trans) in practical applications. In summary, after multiple trainings on CRC-related datasets, the MobileNet-V1 model is not only comparable to other models in performance, but also has significant advantages in computing resource consumption, making it particularly suitable for deployment in actual clinical applications.

[0052] The performance of the model was tested by selecting 5 cases. The results showed that the applicability and effectiveness of the model in clinical diagnosis were verified, providing reliable technical support for actual pathological diagnosis.

[0053] Table 4 Clinical data testing

[0054] As shown in Table 4: Acc represents accuracy, Pre represents precision, Rec represents recall, F1-score represents F1 score, and Speed ​​represents detection speed. It can be seen from the table that the model can correctly predict clinical pathology images and has a fast detection speed.

[0055] The automatic image cropping and sampling algorithm is integrated with the trained classification detection model to develop an automatic detection system for colorectal cancer tissue pathology sections. Users only need to click the corresponding button and enter the data through the system interface to complete the automatic sampling and classification detection of pathology sections. The system is outstanding in significantly reducing the threshold of user operation skills. Its design concept is closely linked to the two core elements of simplicity and efficiency. By providing an intuitive and easy-to-use operation interface, it greatly optimizes the user experience. This innovative design not only simplifies the complex process of pathology diagnosis, but also significantly improves work efficiency and convenience, bringing substantial improvements to the field of pathology diagnosis.

[0056] For the aforementioned embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily required by the present application.

[0057] The above embodiments describe the basic principles and main features of the present invention and the advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the changes and modifications made by those skilled in the art shall be within the scope of protection of the appended claims of the present invention without departing from the spirit and scope of the present invention.

Claims

1. A colorectal cancer pathology image processing and classification method based on transfer learning, characterized in that: The following steps are involved: Step S1: construct a preliminary model; Step S2: Processing the original pathological slice image to obtain a clinical data set; Step S3: The processed clinical data set is used to perform a second transfer learning and model test with the preliminary model to obtain the final model; Step S4: After inputting the data and automatically processing the data, the final model is used to classify and detect the image, and the data results are displayed in the underlying model.

2. The colorectal cancer pathology image processing and classification method based on transfer learning as claimed in claim 1, characterized in that: The step S1 specifically includes the following sub-steps: Step S11: data division, selecting a public colorectal cancer tissue pathology section data set, and dividing the data into a training set, a validation set, and a test set according to a preset ratio; Step S12: Network construction, based on the PyTorch deep learning framework, using MobileNet-V1 as the backbone network to build the model, and using Kather-5K to pre-train the model to obtain a pre-trained model; Step S13: Network training: the first transfer learning of the pre-model is performed on the data set. The model is iteratively trained on the training set and the validation set. When the loss function value on the validation set reaches the optimal value, the early stopping mechanism guides the model to end the training and save the optimal model parameters. Step S14: Model testing, importing the test set into the trained model for evaluation, calculating the accuracy index, completing a complete model training based on the public data set, and obtaining a preliminary model.

3. The colorectal cancer pathology image processing and classification method based on transfer learning as claimed in claim 2, characterized in that: The data processing in step S2 includes: fully automatically cropping and sampling the original pathological slice image containing the diseased tissue, matching it with the image of the public data set in a unified format, and performing data enhancement operations.

4. The colorectal cancer pathology image processing and classification method based on transfer learning as claimed in claim 1, characterized in that: The second transfer learning in step S3 includes: loading the preliminary model, freezing the parameters of the low-level feature extraction layer in the model, unfreezing the parameters of the high-level feature extraction layer, and adjusting them according to the characteristics of the new data during the training process.

5. The colorectal cancer pathology image processing and classification method based on transfer learning as claimed in claim 4, characterized in that: The step S3 model testing specifically includes: Cases were selected from the clinical data set for testing. Based on the known pathological diagnosis information of the patients, the original images were cropped and sampled, and sub-images were generated and then imported into the model. The model predicts the tissue category of each sub-graph and determines whether the patient has colorectal cancer by summarizing the prediction results; The prediction results were compared with the actual lesion conditions of the patients, and the detection indicators based on the patient level were calculated to verify the performance of the model on clinical data.

6. The colorectal cancer pathology image processing and classification method based on transfer learning as claimed in claim 1, characterized in that: The automated processing of input data in step S4 specifically includes: Step A1: remove most of the black pixel areas in the image, and then perform the first sampling to clean up the remaining black pixels; Step A2: automatically determine the actual size of the image and crop the original pathological slice image into multiple pathological slice small blocks; Step A3: Mark the traces of the second sampling in the first sampling image, save it as a copy, and return the detection result; save the preprocessed image file to the specified path to provide input data for subsequent classification and recognition tasks.

7. The colorectal cancer pathology image processing and classification method based on transfer learning as claimed in claim 6, characterized in that: The pathological slice small block size of the image cropped in step A2 is 224×224, and also includes: the parts of the last column and the last row that cannot be rounded are sampled forward starting from the rightmost or the bottommost side to obtain a complete 224×224 small block.

8. The colorectal cancer pathology image processing and classification method based on transfer learning as claimed in claim 7, characterized in that: The image classification and detection specifically includes: taking the file path of the generated pathological slice small blocks as input, calling a pre-trained deep learning classification model to predict and classify each small block, and the classification model classifies each slice small block according to tissue type.

9. The colorectal cancer pathology image processing and classification method based on transfer learning as claimed in claim 8, characterized in that: The tissue types include: adipose tissue ADI, lymphocytes LYM, normal colon mucosa NORM, background BACK, mucus tissue MUC, tumor-associated stromal tissue STR, debris DEB, smooth muscle tissue MUS and colorectal adenocarcinoma epithelial tissue TUM.

10. The colorectal cancer pathology image processing and classification method based on transfer learning as claimed in claim 8, characterized in that: The prediction classification includes the following sub-steps: Step B1: The image is resized and rescaled to the fixed size required by the network to fit the size of the input layer; Step B2: pixel standardization, standardizing the mean and variance of each channel; Step B3: The preprocessed image is sent to the model for forward propagation. The model uses the feature representation learned during the training process to predict the classification task of the input image. Based on the features of the input image, a series of convolution operations, pooling operations, and fully connected layer calculations are performed, and finally the prediction score for each category is output; Step B4: Take the category with the highest probability as the predicted category and return the corresponding predicted probability.

Images