Immunotherapy curative effect prediction method based on pathological section deep learning scoring
By employing a deep learning scoring method based on pathological slides, and utilizing a digital slide scanner and a ResNet50 model, the accuracy of predicting the efficacy of immunotherapy in small cell lung cancer patients was addressed, enabling efficient personalized treatment planning and survival prediction.
Patent Information
- Application Number
- CN202510973373.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-15
- Publication Date
- 2025-10-31
AI Technical Summary
Current technologies are insufficient in accurately predicting the response of small cell lung cancer patients to immunotherapy, making it difficult to develop personalized treatment plans and affecting treatment outcomes and patient survival rates.
By employing a deep learning-based scoring method for pathological slides, high-resolution WSI images were acquired using a digital slide scanner. Image segmentation, quality assessment and screening, color normalization, and feature extraction were performed. The ResNet50 model was then trained to generate patient-level pathological depth scores, which serve as predictive indicators for the efficacy of immunotherapy.
It improves the accuracy of predicting the efficacy of immunotherapy by reducing human intervention through automation and standardization, providing quantitative indicators for personalized treatment, and enhancing the accuracy of prediction and its correlation with survival prediction.
Smart Images
Figure CN120876394A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of immunotherapy technology, specifically to a method for predicting the efficacy of immunotherapy based on deep learning scoring of pathological sections. Background Technology
[0002] Immunotherapy refers to a treatment method that artificially enhances or suppresses the body's immune function to treat diseases, targeting a weakened or overactive immune state. There are many types of immunotherapy, which are applicable to the treatment of a variety of diseases.
[0003] However, existing devices have the following problems:
[0004] In the field of cancer treatment, immunotherapy has become an important treatment method after surgery, radiotherapy, and chemotherapy, bringing new hope to many cancer patients. Small cell lung cancer is an aggressive subtype of lung cancer, characterized by high malignancy, rapid metastasis, and poor prognosis, accounting for approximately 10%-15% of all lung cancer cases. Although immunotherapy has shown some potential in the treatment of small cell lung cancer, the response of different patients to immunotherapy varies significantly. Some patients may benefit from immunotherapy, while others may not respond or even experience disease progression. Therefore, accurately predicting the efficacy of immunotherapy in small cell lung cancer patients is of great significance for developing personalized treatment plans, improving treatment outcomes, and increasing patient survival rates. Currently, clinical methods for predicting the efficacy of immunotherapy mainly include those based on clinical characteristics, molecular biomarkers, and imaging assessments. However, these methods have certain limitations; the predictive value of clinical characteristics is limited and difficult to be accurate. Summary of the Invention
[0005] (a) Technical problems to be solved
[0006] To address the shortcomings of existing technologies, this invention provides a method for predicting the efficacy of immunotherapy based on deep learning scoring of pathological slides. This method has advantages such as accurately predicting the efficacy of immunotherapy for small cell lung cancer and solves the limitations of existing technologies.
[0007] (II) Technical Solution
[0008] To achieve the above objectives, the present invention provides the following technical solution: a method for predicting the efficacy of immunotherapy based on deep learning scoring of pathological slides, comprising pathological image acquisition, image segmentation, image quality assessment and screening, image color normalization, feature extraction and model training, and patient-level score generation and efficacy prediction index establishment, wherein: the pathological image acquisition method uses a digital slide scanner to digitize H&E stained tissue slides of small cell lung cancer patients to obtain high-resolution WSI images; the image segmentation method divides each WSI image into several fixed-size image blocks to form a standardized set of tile images; the image quality assessment and screening method uses an automated image quality assessment algorithm to remove tiles that do not meet quality requirements and retain valid image samples; the image color normalization method applies a color standardization algorithm to the retained tile images to reduce model bias caused by staining differences; the feature extraction and model training method divides patients into those with good efficacy and those with poor efficacy in the training set, and uses a ResNet50 model for training to obtain a well-trained model; the patient-level score generation and efficacy prediction index establishment method calculates the average of all valid tile scores of the same patient to obtain a patient-level pathological depth score, which serves as an indicator for predicting the efficacy of immunotherapy.
[0009] Preferably, the size of the image block is 256×256 pixels.
[0010] Preferably, the automated image quality assessment algorithm is used to remove tiles that contain a large amount of white background, are blurry or have insufficient content, or exhibit scanning blur or abnormal staining.
[0011] Preferably, the color normalization algorithm uses the Macenko method.
[0012] Preferably, the training of the ResNet50 model includes the following steps:
[0013] S1. Data preprocessing: Normalize the tile images in the training set;
[0014] S2, Model Initialization: Initialize the weight parameters of the ResNet50 model;
[0015] S3. Model Training: Input the tile images and their corresponding labels from the training set into the model for training, and optimize the model's weight parameters.
[0016] S4. Model Validation: Use the validation set to validate and evaluate the trained model, and adjust and optimize the model.
[0017] Preferably, the patient-level pathological depth score is calculated using mean pooling.
[0018] Preferably, the pathological depth score is used to predict whether a patient is an immunotherapy responder.
[0019] Preferably, the method further includes an analysis of the correlation between pathological depth scores and patient survival.
[0020] Preferably, the method is applicable to predicting the efficacy of immunotherapy for small cell lung cancer.
[0021] Preferably, the system includes a pathological image acquisition module, an image segmentation module, an image quality assessment and screening module, an image color normalization module, a feature extraction and model training module, and a patient-level score generation module. The pathological image acquisition module is used to acquire WSI images of small cell lung cancer patients. The image segmentation module is used to segment the WSI images into a set of tile images. The image quality assessment and screening module is used to screen valid tile images. The image color normalization module is used to perform color normalization processing on the tile images. The feature extraction and model training module is used to train a ResNet50 model. The patient-level score generation module is used to generate a patient-level pathological depth score.
[0022] (III) Beneficial Effects
[0023] Compared with existing technologies, this invention provides a method for predicting the efficacy of immunotherapy based on deep learning scoring of pathological slides, which has the following beneficial effects:
[0024] This immunotherapy efficacy prediction method based on deep learning scoring of pathological slides improves input data quality by standardizing data processing, image segmentation, quality screening, and color normalization to reduce staining differences and invalid data interference. Through efficient feature extraction, it automatically learns deep features of pathological images using a ResNet50 model, avoiding the subjectivity and limitations of traditional manual feature extraction. For accurate efficacy prediction, it integrates whole-slide information through patient-level pathological deep scoring, average pooling calculation, and other methods, accurately predicting immunotherapy response and associated patient survival, providing quantitative indicators for personalized treatment. Furthermore, through automation and standardization, automated processing at each stage, and quality assessment algorithms, it reduces manual intervention and is suitable for predicting the efficacy of immunotherapy in small cell lung cancer, addressing the accuracy limitations of existing clinical prediction methods. Attached Figure Description
[0025] Figure 1 This is a flowchart of the main system deployment of an immunotherapy efficacy prediction method based on deep learning scoring of pathological sections proposed in this invention.
[0026] Figure 2 This invention provides a pathological image acquisition module for a method to predict the efficacy of immunotherapy based on deep learning scoring of pathological slides.
[0027] Figure 3 This invention provides an image segmentation module for a method to predict the efficacy of immunotherapy based on deep learning scoring of pathological slides.
[0028] Figure 4 This invention provides an image quality assessment and screening module for an immunotherapy efficacy prediction method based on deep learning scoring of pathological slides.
[0029] Figure 5 This invention provides an image color normalization module for a method to predict the efficacy of immunotherapy based on deep learning scoring of pathological sections.
[0030] Figure 6 This invention provides a feature extraction and model training module for an immunotherapy efficacy prediction method based on deep learning scoring of pathological sections.
[0031] Figure 7 This invention presents a patient-level score generation and efficacy prediction module for an immunotherapy efficacy prediction method based on deep learning scoring of pathological slides. Detailed Implementation
[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0033] Please see Figures 1 to 7 This invention provides a technical solution: a method for predicting the efficacy of immunotherapy based on deep learning scoring of pathological slides.
[0034] I. Acquisition of Pathological Images
[0035] Implementation steps:
[0036] 1. Sample collection: Select H&E stained tissue sections obtained from surgery or biopsy of patients with clinically diagnosed small cell lung cancer. The samples must include cancerous tissue and adjacent tissue areas.
[0037] 2. Digital scanning: The tissue sections were scanned using a 3D HiStech Pannoramic MIDI all-glass slide scanner. The scanning resolution was set to ≥20x objective lens, corresponding to a pixel resolution of ≥0.5μm / pixel, to generate high-resolution all-glass slide images WSI in SVS or NDPI format.
[0038] 3. Metadata Recording: Synchronously record basic patient information, age, gender, stage, treatment plan, immunotherapy drugs and cycles, efficacy evaluation results, RECIST 1.1 criteria for complete remission / partial remission / disease stabilization / disease progression, and survival data.
[0039] II. Image Segmentation
[0040] Implementation steps:
[0041] 1. Preprocessing tools: The Python library OpenSlide is used to read WSI images and decompose the entire WSI into fixed-size image tiles based on the image coordinate system;
[0042] 2. Segmentation parameters: Set the image block size to 256×256 pixels, and use a non-overlapping segmentation method to ensure that there is no overlapping area between adjacent tiles, forming a standardized set of tile images;
[0043] 3. Edge processing: For tiles with incomplete WSI edges, fill them with white or black pixels or discard them to ensure that all tiles are the same size.
[0044] III. Image Quality Assessment and Screening
[0045] Implementation steps:
[0046] Automated evaluation algorithm:
[0047] 1. White background detection: Calculate the percentage of pixels in the tile with an RGB value greater than 230. If it exceeds 70%, it is considered invalid and contains a large number of blank areas.
[0048] 2. Tissue integrity detection: Tissue regions are extracted using the Otsu threshold segmentation method. If the tissue area accounts for less than 30%, i.e. the tissue content is insufficient, it is removed.
[0049] 3. Sharpness detection: Calculate the average gradient magnitude of the image; if it is lower than the preset threshold of 50, it is judged as scan blur.
[0050] 4. Staining anomaly detection: Based on the Macenko staining separation algorithm, the H&E staining concentration distribution is calculated. If the mean of a certain staining channel deviates from the mean of the training set by ±3 times the standard deviation, it is judged as a staining anomaly.
[0051] 5. Screening rules: Tiles that meet all four detection conditions above are retained; otherwise, they are discarded, thus forming a valid image sample set.
[0052] IV. Image Color Normalization
[0053] Implementation steps:
[0054] 1. Color Normalization Method: The Macenko algorithm is used to normalize the colors of the preserved tile images. The specific process is as follows:
[0055] 2. Color space conversion: Convert RGB images to OD optical density space to eliminate the influence of lighting;
[0056] 3. Stain separation: The color vectors of H&E staining are extracted by singular value decomposition (SVD) to eliminate non-specific staining interference;
[0057] 4. Normalization process: Align the current image's coloring vector with the target reference coloring vector based on high-quality sample statistics, and adjust the coloring density to a uniform standard;
[0058] 5. Validation method: After normalization, the images are visually inspected, and the color consistency and statistical tests such as the KS test are used to compare the staining distribution to ensure that the staining differences are significantly reduced.
[0059] V. Feature Extraction and Model Training
[0060] Implementation steps:
[0061] 1. Data partitioning: The patient cohort was divided into training set, validation set and test set in a ratio of 8:1:1. Responders in the good efficacy group were defined as CR / PR by RECIST assessment after treatment, and non-responders in the poor efficacy group were defined as SD / PD.
[0062] 2. ResNet50 model training process:
[0063] S1 data preprocessing: normalize the pixel values of the tile image to [0,1], and randomly apply data augmentation, rotation ±15°, horizontal flip, and brightness / contrast adjustment to improve the model's generalization.
[0064] S2 model initialization: Load ImageNet pre-trained ResNet50 weights, freeze the first 3 convolutional blocks, retain general feature extraction capabilities, and replace the last fully connected layer with a 2-classification output response / non-response;
[0065] S3 model training: Adam optimizer was used, learning rate 1e-4, batch size 32, training period 20 epochs, loss function was cross-entropy loss, and training / validation loss and accuracy were recorded every 5 epochs.
[0066] S4 Model Validation: The validation set is used to evaluate model performance. Metrics include accuracy, precision, recall, and AUC-ROC. If the validation loss does not decrease for three consecutive epochs, early stopping is triggered to preserve the optimal weights.
[0067] 3. Hardware configuration: The training process is based on NVIDIA GPU, RTX3090 acceleration, and uses the PyTorch framework to implement parallel computing.
[0068] VI. Patient-level score generation and efficacy prediction
[0069] Implementation steps:
[0070] 1. Score Calculation: For all valid tiles of the same patient, the predicted probability and responder probability of each tile are output by the trained ResNet50 model. The patient-level pathological depth score is calculated by mean pooling, where N is the number of valid tiles for the patient and pi is the response probability of the i-th tile.
[0071] 2. Establishment of forecasting indicators:
[0072] Efficacy prediction: A scoring threshold of 0.5 was set. Those with a score ≥ the threshold were judged as responders to immunotherapy, and those with a score below the threshold were judged as non-responders. The predictive efficacy was verified by the test set with AUC-ROC ≥ 0.8.
[0073] Survival analysis: The Kaplan-Meier method was used to plot survival curves. The differences in overall survival (OS) between high and low score groups were compared using the Log-rank test. The Cox proportional hazards model was used to analyze the independent association between score and survival.
[0074] VII. Module Implementation Architecture
[0075] 1. Pathological image acquisition module: integrates digital scanner driver interface, supports import of multiple WSI formats, and automatically associates with patient clinical data;
[0076] 2. Image segmentation module: Developed based on OpenSlide and OpenCV, providing a parameter configuration interface for segmentation size, overlap rate, etc.
[0077] 3. Quality assessment module: Encapsulates a custom Python algorithm, supports batch automated filtering, and outputs a list of valid tile indexes;
[0078] 4. Color Normalization Module: Implements the Macenko algorithm using the Python library StainTools, supporting visual comparison of images before and after normalization.
[0079] 5. Model Training Module: Based on the PyTorch / TensorFlow framework, it provides hyperparameter tuning, learning rate, batch size, and training process monitoring functions.
[0080] 6. Scoring Generation Module: Integrates patient-level data and supports outputting in-depth scoring and prediction results in Excel / CSV format.
[0081] VIII. Verification of Implementation Examples
[0082] One hundred patients with small cell lung cancer were selected, with 60 cases in the training set, 20 cases in the validation set, and 20 cases in the test set. After processing according to the above method, the prediction accuracy of the test set reached 82%, and the AUC-ROC was 0.85, which was significantly better than the AUC=0.68 of the traditional molecular marker PD-L1 expression prediction. Survival analysis showed that the median OS of the high depth score group was 14.5 months, which was significantly longer than the low depth score group by 9.2 months (P<0.05), thus verifying the effectiveness and clinical value of the method.
[0083] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0084] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A method for predicting the efficacy of immunotherapy based on deep learning scoring of pathological slides, characterized in that, This includes pathological image acquisition, image segmentation, image quality assessment and screening, image color normalization, feature extraction and model training, and patient-level score generation and efficacy prediction index establishment, among which: The pathological image acquisition process involves digitizing H&E-stained tissue sections from small cell lung cancer patients using a digital slide scanner to obtain high-resolution WSI images. Image segmentation divides each WSI image into several fixed-size image blocks, forming a standardized set of tile images. Image quality assessment and screening utilizes an automated image quality assessment algorithm to remove tiles that do not meet quality requirements, retaining valid image samples. Image color normalization applies a color normalization algorithm to the retained tile images to reduce model bias caused by staining differences. Feature extraction and model training categorize patients in the training set into those with good and poor treatment outcomes, and a ResNet50 model is used for training to obtain a well-trained model. Patient-level score generation and efficacy prediction index establishment involve averaging all valid tile scores for the same patient to obtain a patient-level pathological depth score, which serves as a predictive index for immunotherapy efficacy.
2. The method for predicting the efficacy of immunotherapy based on deep learning scoring of pathological sections according to claim 1, characterized in that: The size of the image block is 256×256 pixels.
3. The method for predicting the efficacy of immunotherapy based on deep learning scoring of pathological sections according to claim 1, characterized in that: The automated image quality assessment algorithm is used to remove tiles that have a large amount of white background, are blurry or have insufficient content, or have abnormal scanning or staining.
4. The method for predicting the efficacy of immunotherapy based on deep learning scoring of pathological sections according to claim 1, characterized in that: The color normalization algorithm uses the Macenko method.
5. The method for predicting the efficacy of immunotherapy based on deep learning scoring of pathological sections according to claim 1, characterized in that: The training of the ResNet50 model includes the following steps: S1. Data preprocessing: Normalize the tile images in the training set; S2, Model Initialization: Initialize the weight parameters of the ResNet50 model; S3. Model Training: Input the tile images and their corresponding labels from the training set into the model for training, and optimize the model's weight parameters. S4. Model Validation: Use the validation set to validate and evaluate the trained model, and adjust and optimize the model.
6. The method for predicting the efficacy of immunotherapy based on deep learning scoring of pathological sections according to claim 1, characterized in that: The patient-level pathological depth score was calculated using mean pooling.
7. The method for predicting the efficacy of immunotherapy based on deep learning scoring of pathological sections according to claim 1, characterized in that: The pathological depth score is used to predict whether a patient is a responder to immunotherapy.
8. The method for predicting the efficacy of immunotherapy based on deep learning scoring of pathological sections according to claim 1, characterized in that: The method also includes an analysis of the association between pathological depth scores and patient survival.
9. The method for predicting the efficacy of immunotherapy based on deep learning scoring of pathological sections according to claim 1, characterized in that: The method described is applicable to predicting the efficacy of immunotherapy for small cell lung cancer.
10. The method for predicting the efficacy of immunotherapy based on deep learning scoring of pathological sections according to claim 1, characterized in that: The system includes a pathological image acquisition module, an image segmentation module, an image quality assessment and screening module, an image color normalization module, a feature extraction and model training module, and a patient-level score generation module. The pathological image acquisition module is used to acquire WSI images of small cell lung cancer patients. The image segmentation module is used to segment the WSI images into a set of tile images. The image quality assessment and screening module is used to screen valid tile images. The image color normalization module is used to perform color normalization processing on the tile images. The feature extraction and model training module is used to train a ResNet50 model. The patient-level score generation module is used to generate patient-level pathological depth scores.