Immune score prediction method for advanced lung cancer

Abstract

The invention discloses an advanced lung cancer immunity score prediction method. The method comprises the following steps: acquiring slice image data of a biological tissue of a lung cancer patient to be predicted; based on a neural network algorithm, improving the quality of the slice image data to obtain optimized image data; inputting the optimized image data into a feature extraction model to obtain a corresponding image feature vector; and inputting the image feature vector into a trained immunotherapy score prediction neural network to obtain an immunotherapy effect corresponding to the to-be-predicted lung cancer patient, the immunotherapy score prediction neural network is obtained by training a training data set comprising a plurality of training tissue slice images and corresponding immunotherapy score marks. Therefore, accurate immunotherapy prediction directly based on the image can be realized, the prediction cost is reduced, and the prediction efficiency and the prediction accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a method and system for predicting the effect of immunotherapy for lung cancer. Background Art

[0002] Lung cancer is one of the malignant tumors with the highest morbidity and mortality in the world, accounting for about one-eighth of cancer cases worldwide. As people pay more attention to health, how to effectively treat lung cancer has become an important focus, especially how to effectively predict the prognosis of lung cancer treatment is also a key factor. In recent years, lung cancer treatment technology has continued to advance. With the emergence of immunotherapy technology, the survival rate of lung cancer patients has also been significantly improved, and existing technologies have begun to appear to evaluate the immune microenvironment of tumors to predict the immunotherapy effect of lung cancer patients. However, when predicting the immunotherapy effect of lung cancer patients, the existing technology does not consider combining pathological image analysis and neural network models to improve the accuracy of prediction, and some existing technologies that use gene signature methods are also costly and inconvenient. It can be seen that the existing technology has defects that need to be solved urgently. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a method and system for predicting the effect of immunotherapy for lung cancer, which can realize accurate immunotherapy prediction directly based on images, reduce prediction costs, and improve prediction efficiency and prediction accuracy.

[0004] In order to solve the above technical problems, the first aspect of the present invention discloses a method for predicting the effect of immunotherapy for lung cancer, the method comprising: Acquire slice image data of biological tissue of a patient to be diagnosed with lung cancer; Based on a neural network algorithm, the quality of the slice image data is improved to obtain optimized image data; Inputting the optimized image data into a feature extraction model to obtain a corresponding image feature vector; The image feature vector is input into a trained immunotherapy score prediction neural network to obtain the corresponding immunotherapy effect of the lung cancer patient to be predicted; the immunotherapy score prediction neural network is trained by a training data set including multiple training tissue section images and corresponding immunotherapy score annotations.

[0005] As an optional implementation, in the first aspect of the present invention, the quality of the slice image data is improved based on a neural network algorithm to obtain optimized image data, including: The slice image data belonging to the FS image is input into a trained image conversion model to obtain output FFPE-style optimized image data; the image conversion model is trained by a training data set including multiple training FS images and corresponding FFPE-style image annotations.

[0006] As an optional implementation, in the first aspect of the present invention, the feature extraction model is a Phikon model.

[0007] As an optional embodiment, in the first aspect of the present invention, the inputting of the image feature vector into a trained immunotherapy score prediction neural network to obtain the immunotherapy effect corresponding to the lung cancer patient to be predicted includes: Inputting the image feature vector into a plurality of trained immunotherapy score prediction neural networks to obtain a therapeutic effect prediction result of each immunotherapy score prediction neural network for the image feature vector; Calculating the average of all the efficacy prediction results to obtain an average efficacy parameter; The immunotherapy effect corresponding to the lung cancer patient to be predicted is determined according to the average efficacy parameter and a preset judgment threshold.

[0008] As an optional embodiment, in the first aspect of the present invention, the immunotherapy score prediction neural network includes an input layer, a Transformer encoder, a fully connected layer and a classification probability output layer connected in sequence.

[0009] As an optional embodiment, in the first aspect of the present invention, the immunotherapy score prediction neural network is trained based on the LA-MIL method and optimized by the AdamW optimizer. The initial learning rate is set to 2e-05 and gradually adjusted within 20 training cycles. The batch size is set to 1 and a weighted binary cross entropy loss function is used. After each training cycle, the F1 score on the validation data set is calculated and the model weight with the highest F1 score is selected for saving. Multiple trained immunotherapy score prediction neural networks are obtained through multi-fold training and cross-validation.

[0010] As an optional implementation, in the first aspect of the present invention, the step of inputting the optimized image data into a feature extraction model to obtain a corresponding image feature vector comprises: Acquiring patient information of the lung cancer patient to be predicted; Inputting the patient information into the trained LLM model to obtain corresponding cancer type information; Determining an image segmentation model corresponding to the cancer type information in a preset image segmentation model library; the image segmentation model is trained based on a training data set including a plurality of training case images corresponding to the cancer type information and core image area annotations; Inputting the optimized image data into the image segmentation model to obtain a key area image corresponding to the optimized image data; Determining a feature extraction model corresponding to the cancer type information in a preset feature extraction model library; the feature extraction model is trained based on a training pathology image corresponding to the cancer type information; The key area image is input into the feature extraction model to obtain a corresponding image feature vector.

[0011] As an optional embodiment, in the first aspect of the present invention, the immunotherapy score annotation is determined by the following steps: Acquiring tissue sample data corresponding to the training tissue slice image; Determining the immune infiltration score corresponding to the tissue sample data based on the immune infiltration score model; Calculate the weighted sum average of the immune infiltration scores corresponding to all the training tissue section images to obtain a score reference value; wherein the weighted calculation weight corresponding to each immune infiltration score includes a first weight and a second weight; the first weight is proportional to the image clarity parameter corresponding to the training tissue section image; the second weight is proportional to the score reliability parameter corresponding to the training tissue section image; the score reliability parameter is the similarity between the patient data corresponding to the tissue sample data corresponding to the training tissue section image and the patient data corresponding to the model comparison sample data of the immune infiltration score model; According to the comparison result between the immune infiltration score and the score reference value, the immunotherapy score label corresponding to the training tissue section image is determined.

[0012] A second aspect of an embodiment of the present invention discloses a lung cancer immunotherapy effect prediction system, the system comprising: An image acquisition module, used to acquire slice image data of biological tissue of a patient to be diagnosed with lung cancer; A quality optimization module, used for improving the quality of the slice image data based on a neural network algorithm to obtain optimized image data; A feature extraction module, used for inputting the optimized image data into a feature extraction model to obtain a corresponding image feature vector; The efficacy prediction module is used to input the image feature vector into a trained immunotherapy score prediction neural network to obtain the corresponding immunotherapy effect of the lung cancer patient to be predicted; the immunotherapy score prediction neural network is trained by a training data set including multiple training tissue slice images and corresponding immunotherapy score annotations.

[0013] As an optional implementation, in the second aspect of the present invention, the quality optimization module improves the quality of the slice image data based on a neural network algorithm to obtain a specific method for optimizing the image data, including: The slice image data belonging to the FS image is input into a trained image conversion model to obtain output FFPE-style optimized image data; the image conversion model is trained by a training data set including multiple training FS images and corresponding FFPE-style image annotations.

[0014] As an optional implementation, in the second aspect of the present invention, the feature extraction model is a Phikon model.

[0015] As an optional embodiment, in the second aspect of the present invention, the efficacy prediction module inputs the image feature vector into a trained immunotherapy score prediction neural network to obtain the specific method of the immunotherapy effect corresponding to the lung cancer patient to be predicted, including: Inputting the image feature vector into a plurality of trained immunotherapy score prediction neural networks to obtain a therapeutic effect prediction result of each immunotherapy score prediction neural network for the image feature vector; Calculating the average of all the efficacy prediction results to obtain an average efficacy parameter; The immunotherapy effect corresponding to the lung cancer patient to be predicted is determined according to the average efficacy parameter and a preset judgment threshold.

[0016] As an optional embodiment, in the second aspect of the present invention, the immunotherapy score prediction neural network includes an input layer, a Transformer encoder, a fully connected layer and a classification probability output layer connected in sequence.

[0017] As an optional embodiment, in the second aspect of the present invention, the immunotherapy score prediction neural network is trained based on the LA-MIL method and optimized by the AdamW optimizer. The initial learning rate is set to 2e-05 and gradually adjusted within 20 training cycles. The batch size is set to 1 and a weighted binary cross entropy loss function is used. After each training cycle, the F1 score on the validation data set is calculated and the model weight with the highest F1 score is selected for saving. Multiple trained immunotherapy score prediction neural networks are obtained through multi-fold training and cross-validation.

[0018] As an optional implementation, in the second aspect of the present invention, the specific manner in which the feature extraction module inputs the optimized image data into the feature extraction model to obtain the corresponding image feature vector includes: Acquiring patient information of the lung cancer patient to be predicted; Inputting the patient information into the trained LLM model to obtain corresponding cancer type information; Determining an image segmentation model corresponding to the cancer type information in a preset image segmentation model library; the image segmentation model is trained based on a training data set including a plurality of training case images corresponding to the cancer type information and core image area annotations; Inputting the optimized image data into the image segmentation model to obtain a key area image corresponding to the optimized image data; Determining a feature extraction model corresponding to the cancer type information in a preset feature extraction model library; the feature extraction model is trained based on a training pathology image corresponding to the cancer type information; The key area image is input into the feature extraction model to obtain a corresponding image feature vector.

[0019] As an optional embodiment, in the second aspect of the present invention, the immunotherapy score annotation is determined by the following steps: Acquiring tissue sample data corresponding to the training tissue slice image; Determining the immune infiltration score corresponding to the tissue sample data based on the immune infiltration score model; Calculate the weighted sum average of the immune infiltration scores corresponding to all the training tissue section images to obtain a score reference value; wherein the weighted calculation weight corresponding to each immune infiltration score includes a first weight and a second weight; the first weight is proportional to the image clarity parameter corresponding to the training tissue section image; the second weight is proportional to the score reliability parameter corresponding to the training tissue section image; the score reliability parameter is the similarity between the patient data corresponding to the tissue sample data corresponding to the training tissue section image and the patient data corresponding to the model comparison sample data of the immune infiltration score model; According to the comparison result between the immune infiltration score and the score reference value, the immunotherapy score label corresponding to the training tissue section image is determined.

[0020] The third aspect of the present invention discloses another lung cancer immunotherapy effect prediction system, the system comprising: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute part or all of the steps in the method for predicting the effect of immunotherapy for lung cancer disclosed in the first aspect of the present invention.

[0021] The fourth aspect of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute some or all of the steps in the method for predicting the effect of lung cancer immunotherapy disclosed in the first aspect of the present invention.

[0022] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: The present invention can improve the quality of slice image data according to the neural network algorithm to obtain optimized image data, and then obtain the corresponding image feature vector based on the feature extraction model, and finally accurately predict the corresponding immunotherapy effect score of lung cancer patients through the trained immunotherapy score prediction neural network, thereby realizing accurate immunotherapy prediction directly based on images, reducing prediction costs, and improving prediction efficiency and prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 It is a flow chart of a method for predicting the effect of immunotherapy for lung cancer disclosed in an embodiment of the present invention.

[0025] Figure 2 It is a structural schematic diagram of a lung cancer immunotherapy effect prediction system disclosed in an embodiment of the present invention.

[0026] Figure 3 It is a schematic diagram of the structure of another lung cancer immunotherapy effect prediction system disclosed in an embodiment of the present invention.

[0027] Figure 4 It is a schematic diagram of an algorithm flow of a lung cancer immunotherapy effect prediction system disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or equipment that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or equipment.

[0030] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0031] Before describing the specific implementation methods of the present invention, the technical background of the present invention is further described: Lung cancer is one of the malignant tumors with the highest morbidity and mortality in the world, accounting for about one-eighth of cancer cases worldwide. Among all types of lung cancer, non-small cell lung cancer (NSCLC) accounts for about 85%, and most NSCLC patients are already in the advanced stage when diagnosed. Despite the continuous advancement of lung cancer treatment technology in recent years, the prognosis of patients with advanced NSCLC is still poor. This is mainly because traditional treatments (such as chemotherapy and radiotherapy) have limited therapeutic effects on advanced lung cancer, and most patients still cannot obtain long-term survival benefits after treatment. The emergence of immune checkpoint inhibitors (ICIs) has changed the treatment paradigm of NSCLC and brought new treatment hope to patients with advanced non-squamous NSCLC. Studies have shown that ICIs have shown significant advantages in improving patient survival compared with traditional chemotherapy. Previous studies have further confirmed that chemotherapy combined with immunotherapy can significantly improve the survival rate of patients with non-squamous NSCLC in first-line treatment, and the toxicity of this combination therapy is tolerable to patients. However, despite the encouraging response rate of chemotherapy combined with immunotherapy, the proportion of patients who can achieve long-term survival benefits is still low, which makes it urgent to use more accurate biomarkers to predict patients' treatment response and clinical outcomes. Currently, biomarkers such as PD-L1 expression and tumor mutation burden (TMB) have been widely used to predict the effect of immunotherapy, but their predictive accuracy is affected by the overlap between responders and non-responders, so their clinical application is still limited.

[0032] Although existing technologies have made some progress in predicting immunotherapy in patients with lung cancer, especially advanced non-small cell lung cancer (NSCLC), there are still several significant shortcomings: 1. Limitations of biomarkers: The main biomarkers currently used to predict the effect of immunotherapy, such as PD-L1 expression and tumor mutation burden (TMB), can guide clinical decision-making to a certain extent, but their predictive ability is affected by the overlap between responders and non-responders, resulting in an inability to accurately distinguish which patients will benefit from immunotherapy. This imprecision limits its widespread application in clinical practice.

[0033] 2. Challenges in the clinical application of gene signatures: Although the gene signature method can provide important information about the tumor immune microenvironment, its evaluation process is complex and costly, making it difficult to be widely used in clinical practice. In addition, the evaluation of gene signatures requires specialized equipment and technical support, which limits its clinical promotion and use.

[0034] 3. Limitations of computational pathology applications: Although computational pathology has shown some potential in analyzing pathological slide images and predicting clinical outcomes, current research is still in its early stages. In particular, research and applications are limited in identifying subgroups of NSCLC patients who may benefit from chemotherapy combined with immunotherapy. This means that the application of computational pathology in clinical treatment decision-making has not yet been fully verified and widely accepted.

[0035] The present invention discloses a lung cancer immunotherapy effect prediction method and system, which can improve the quality of slice image data according to a neural network algorithm to obtain optimized image data, and then obtain the corresponding image feature vector based on a feature extraction model, and finally accurately predict the corresponding immunotherapy effect score of lung cancer patients through a trained immunotherapy score prediction neural network, thereby realizing accurate immunotherapy prediction directly based on images, reducing prediction costs, and improving prediction efficiency and prediction accuracy. The following are detailed descriptions.

[0036] Embodiment 1 See also Figure 1 , Figure 1 : is a flow chart of a method for predicting the effect of lung cancer immunotherapy disclosed in an embodiment of the present invention. Figure 1 The described lung cancer immunotherapy effect prediction method can be applied to a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). Figure 1 As shown, the method for predicting the effect of lung cancer immunotherapy may include the following operations: 101. Obtain slice image data of biological tissue of a patient whose lung cancer is to be predicted.

[0037] 102. Based on the neural network algorithm, the quality of slice image data is improved to obtain optimized image data. 103. Input the optimized image data into a feature extraction model to obtain a corresponding image feature vector. 104. Input the image feature vector into the trained immunotherapy score prediction neural network to obtain the corresponding immunotherapy effect of the lung cancer patient to be predicted.

[0038] Optionally, the immunotherapy score prediction neural network can be a support vector machine (SVM), a random forest (RandomForest), a fully convolutional neural network (FCN) or an XGBoost network.

[0039] Optionally, the immunotherapy score prediction neural network is trained using a training data set including a plurality of training tissue slice images and corresponding immunotherapy score annotations.

[0040] It can be seen that the above-mentioned embodiment of the invention can improve the quality of slice image data according to the neural network algorithm to obtain optimized image data, and then obtain the corresponding image feature vector based on the feature extraction model, and finally accurately predict the corresponding immunotherapy effect score of lung cancer patients through the trained immunotherapy score prediction neural network, thereby realizing accurate immunotherapy prediction directly based on images, reducing prediction costs, and improving prediction efficiency and prediction accuracy.

[0041] As an optional embodiment, in the above steps, based on a neural network algorithm, the quality of the slice image data is improved to obtain optimized image data, including: The slice image data belonging to the FS image is input into the trained image conversion model to obtain the output FFPE-style optimized image data; the image conversion model is trained by a training data set including multiple training FS images and corresponding FFPE-style image annotations.

[0042] Optionally, the image conversion model can be an AI-FFPE model, which is actually an image style conversion method based on a generative adversarial network (GAN), mainly used to convert frozen section (FS) images into formalin-fixed paraffin-embedded (FFPE) image styles. This conversion can improve the image quality of frozen sections, making them more stable and accurate in subsequent analysis.

[0043] It can be seen that through the above-mentioned optional embodiments, the slice image data belonging to the FS image can be converted into FFPE-style optimized image data with better quality based on the trained image conversion model to improve the quality of the image, thereby assisting in realizing accurate immunotherapy prediction directly based on the image, reducing prediction costs, and improving prediction efficiency and prediction accuracy.

[0044] As an optional embodiment, in the above steps, the feature extraction model is a Phikon model.

[0045] It can be seen that through the above optional embodiments, it is limited to fully extract image features through the Phikon model to improve the accuracy of subsequent predictions, assist in realizing accurate immunotherapy predictions directly based on images, reduce prediction costs, and improve prediction efficiency and prediction accuracy.

[0046] As an optional embodiment, in the above step, inputting the image feature vector into a trained immunotherapy score prediction neural network to obtain the immunotherapy effect corresponding to the lung cancer patient to be predicted includes: Inputting the image feature vector into a plurality of trained immunotherapy score prediction neural networks to obtain the efficacy prediction result of each immunotherapy score prediction neural network for the image feature vector; Calculate the average of all efficacy prediction results to obtain the average efficacy parameter; According to the average efficacy parameters and the preset judgment threshold, the corresponding immunotherapy effect of the lung cancer patient to be predicted is determined.

[0047] It can be seen that through the above optional embodiments, multiple efficacy prediction results can be predicted through multiple trained immunotherapy score prediction neural networks, and average calculation and threshold judgment can be performed, thereby improving the accuracy of efficacy prediction, realizing accurate immunotherapy prediction directly based on images, reducing prediction costs, and improving prediction efficiency and prediction accuracy.

[0048] As an optional embodiment, in the above steps, the immunotherapy score prediction neural network includes an input layer, a Transformer encoder, a fully connected layer, and a classification probability output layer connected in sequence.

[0049] It can be seen that through the above optional embodiments, the network structure of the immunotherapy score prediction neural network is defined, which can be used to accurately predict the therapeutic effect, assist in realizing accurate immunotherapy prediction directly based on images, reduce prediction costs, and improve prediction efficiency and prediction accuracy.

[0050] As an optional embodiment, in the above steps, the immunotherapy score prediction neural network is trained based on the LA-MIL method and optimized by the AdamW optimizer. The initial learning rate is set to 2e-05 and gradually adjusted within 20 training cycles. The batch size is set to 1, and the weighted binary cross entropy loss function is used. After each training cycle, the F1 score on the validation data set is calculated and the model weight with the highest F1 score is selected for saving. Multiple trained immunotherapy score prediction neural networks are obtained through multi-fold training and cross-validation.

[0051] It can be seen that through the above optional embodiments, the training details of the immunotherapy score prediction neural network are defined to train a neural network model with more accurate prediction effects, assist in realizing accurate immunotherapy predictions directly based on images, reduce prediction costs, and improve prediction efficiency and prediction accuracy.

[0052] As an optional embodiment, in the above step, inputting the optimized image data into the feature extraction model to obtain the corresponding image feature vector includes: Acquire patient information of a lung cancer patient to be predicted; Input the patient information into the trained LLM model to obtain the corresponding cancer type information; Determining an image segmentation model corresponding to the cancer type information in a preset image segmentation model library; optionally, the image segmentation model is trained based on a training data set including training case images corresponding to a plurality of cancer type information and core image area annotations; Inputting the optimized image data into the image segmentation model to obtain the key area image corresponding to the optimized image data; Determining a feature extraction model corresponding to the cancer type information in a preset feature extraction model library; optionally, the feature extraction model is trained based on a training pathology image corresponding to the cancer type information; The key area image is input into the feature extraction model to obtain the corresponding image feature vector.

[0053] It can be seen that through the above optional embodiments, the cancer type can be determined based on the LLM model in combination with the patient's information, and then the corresponding image segmentation model and feature extraction model can be determined based on the screening of the corresponding type, so as to achieve more accurate image key area determination and feature extraction, and assist in realizing accurate immunotherapy prediction directly based on the image, reduce prediction costs, and improve prediction efficiency and prediction accuracy.

[0054] As an optional embodiment, in the above steps, the immunotherapy score annotation is determined by the following steps: Obtaining tissue sample data corresponding to the training tissue slice image; Based on the immune infiltration scoring model, determining the immune infiltration score corresponding to the tissue sample data; Calculate the weighted sum average of the immune infiltration scores corresponding to all training tissue section images to obtain a score reference value; optionally, the weighted calculation weight corresponding to each immune infiltration score includes a first weight and a second weight; the first weight is proportional to the image clarity parameter corresponding to the training tissue section image; the second weight is proportional to the score reliability parameter corresponding to the training tissue section image; the score reliability parameter is the similarity between the patient data corresponding to the tissue sample data corresponding to the training tissue section image and the patient data corresponding to the model comparison sample data of the immune infiltration score model; According to the comparison result between the immune infiltration score and the score reference value, the immunotherapy score annotation corresponding to the training tissue section image is determined.

[0055] It can be seen that through the above optional embodiments, the immune infiltration score of the tissue sample can be determined based on the immune infiltration scoring model, and then the appropriate scoring reference value can be determined based on the weighted sum average calculation, so as to finally obtain accurate immunotherapy score annotations for the training samples, so that the trained immunotherapy score prediction neural network has higher prediction accuracy, and assists in realizing accurate immunotherapy prediction directly based on images, reducing prediction costs, and improving prediction efficiency and prediction accuracy.

[0056] In a more specific embodiment, a computational pathology-based pathological immune score (PIS) system is implemented to provide a more accurate and reliable biomarker for predicting the clinical outcomes of patients with advanced non-squamous NSCLC after chemotherapy combined with immunotherapy. Its main purposes include: 1. Improve prediction accuracy: By integrating pathological image analysis of tumor tissue with quantification of tumor immune microenvironment (TIME) characteristics, the pathological immune score (PIS) system of the present invention can more accurately identify which advanced non-squamous NSCLC patients may benefit from chemotherapy combined with immunotherapy, thereby improving the accuracy of treatment effect prediction.

[0057] 2. Reduce evaluation costs and complexity: Compared with the gene signature method, the PIS system of the present invention relies on computational pathology analysis, which not only simplifies the biomarker evaluation process but also reduces costs, making it easier to promote and apply in clinical practice.

[0058] 3. Enhance the convenience of clinical application: By taking advantage of computational pathology, the PIS system of the present invention can provide additional immune scoring information based on conventional pathological image analysis, thereby helping clinicians to formulate personalized treatment plans more conveniently.

[0059] The training and workflow of a scoring algorithm model for this case immune scoring system can be found in Figure 4 First, whole-slice images (WSIs) from the Cancer Genome Atlas Lung Adenocarcinoma (TCGA-LUAD) dataset were used as training data. These WSIs include formalin-fixed paraffin-embedded (FFPE) samples and frozen section (FS) samples. Since the quality of frozen section images is usually low and may affect subsequent analysis, a neural network algorithm model such as the AI-FFPE method is used to convert FS sections into FFPE style.

[0060] To enhance the representativeness of the dataset, a total of 1,333 WSIs were obtained from the TCGA-LUAD dataset and randomly divided into three parts for model training and cross-validation. The included dataset contained adult patients with primary lung adenocarcinoma from the United States. The inclusion criteria for patients were: (1) digital formalin-fixed paraffin-embedded (FFPE) samples with hematoxylin and eosin (H&E) stained tissue sections were available; (2) gene expression profile data based on RNA sequencing were available. After initial screening, a total of 515 patients' WSIs were included.

[0061] Specifically, the implementation scheme of the algorithm model in this implementation scheme includes: Step 1: Obtain whole-slice images from the TCGA-LUAD dataset and perform data augmentation and preprocessing.

[0062] Step 2: Convert frozen sections (FS) to formalin-fixed paraffin-embedded (FFPE) style using the AI-FFPE method and generate image tiles.

[0063] Step 3: Use the Phikon model for feature extraction and use the LA-MIL method for model training.

[0064] Step 4: Model validation and survival analysis were performed on the CPTAC-LUAD and ORIENT-11 external cohorts.

[0065] Specifically, since WSIs are usually large in size and not suitable for direct input into standard image classification models, a weakly supervised multiple instance learning (MIL) method was used to deal with this problem. First, a simple threshold method was used to identify tissue regions in WSIs, and a set of image patches were generated from these regions in a sliding window manner. Each image patch was 224 x 224 pixels in size and extracted at two magnifications (20X: 0.5 µm / pixel, 10X: 1 µm / pixel) to contain information from different scales.

[0066] Furthermore, after the image blocks are generated, the Phikon model is used to extract features from each image block. The Phikon model is a basic model for pathological images. After being pre-trained on more than 40 million pan-cancer digital pathology image blocks, it can encode each image block into a 768-dimensional feature vector. In order to capture more detailed information in the image blocks, the 768-dimensional features extracted at two magnifications are combined to form a 1536-dimensional feature vector for each image block. Although the Phikon model is used as a feature extractor in this method, other basic models for pathological images can also be used for feature extraction of image blocks.

[0067] Next, the extracted features are weakly supervised trained using the local self-attention map based transformer MIL method (Local Attention-MIL, LA-MIL). Unlike the classic MIL method, the LA-MIL method can effectively model the interdependencies between image patches, especially the local spatial relationships between adjacent patches, through its inherent self-attention mechanism. This method is particularly suitable for tissue pathology image analysis because it focuses on the local attention between image patches rather than the global self-attention, which makes it more accurate when processing biopsy samples.

[0068] Specifically, the basic structure of LA-MIL is Transformer. It includes an input layer, which takes as input the feature vector of the extracted tile sequence; a Transformer encoder, which is composed of multiple local self-attention layers and feedforward neural network layers. Compared with the usual self-attention layer, the local self-attention layer allows the model to only focus on the adjacent image blocks when processing each input image block; a classification head, which inputs the output of the Transformer encoder into a fully connected layer for further processing. The output of the last fully connected layer is converted into a binary classification probability through a softmax function. Among them, the adjacent tiles of each tile are determined using the k-nearest neighbor graph network method, and the specific number of k can be 16, 64 or 128.

[0069] The trained labels are binary labels after stratification based on the ESTIMATE immune infiltration score of the sample. The ESTIMATE score is a scoring method for inferring tumor cell composition, matrix and immune infiltration based on transcriptome data. The label annotation can be calculated by using the R package software of Immuno-Oncology Biological Research (IOBR) to calculate the ESTIMATE immune infiltration score of each sample, and the samples can be divided into high and low categories based on the median value of the ESTIMATE immune infiltration score of the samples in the entire training set as the threshold. The samples can also be classified based on the weighted sum average value in the above embodiment as the threshold.

[0070] During the training process, the AdamW optimizer was used, and the initial learning rate was set to 2e-05 and gradually adjusted within 20 training cycles. The batch size was set to 1, and the weighted binary cross entropy loss function was used to handle imbalanced data. In order to enhance the stability of the model, the F1 score on the validation set was calculated after each training cycle, and the model weight with the highest F1 score was selected for saving. Finally, three-fold training and cross-validation produced three different models, each of which generated a classification probability output. The classification probability values ​​of the three models were averaged to obtain the final predicted probability value. If the final predicted probability of a sample is greater than or equal to 50%, the sample is classified as PIS high, and if the final predicted probability is less than 50%, the sample is classified as PIS low.

[0071] To further verify the effectiveness of the PIS system in clinical practice, a survival analysis was performed on the ORIENT-11 cohort. The accuracy of the PIS system in predicting treatment was verified by comparing the progression-free survival (PFS) and overall survival (OS) of patients with high and low ESTIMATE scores predicted by the model. The results showed that the survival of patients predicted to have high scores was significantly better than that of patients predicted to have low scores, indicating that the PIS system can effectively predict the response of patients with advanced non-squamous NSCLC to chemotherapy combined with immunotherapy.

[0072] Embodiment 2 See also Figure 2 , Figure 2 Schematic diagram of a lung cancer immunotherapy effect prediction system disclosed in an embodiment of the present invention. Figure 2 The lung cancer immunotherapy effect prediction system described can be applied to a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). Figure 2 As shown, the lung cancer immunotherapy effect prediction system may include: The image acquisition module 201 is used to acquire slice image data of biological tissue of a patient to be diagnosed with lung cancer.

[0073] The quality optimization module 202 is used to improve the quality of the slice image data based on the neural network algorithm to obtain optimized image data. The feature extraction module 203 is used to input the optimized image data into the feature extraction model to obtain the corresponding image feature vector. The efficacy prediction module 204 is used to input the image feature vector into the trained immunotherapy score prediction neural network to obtain the corresponding immunotherapy effect of the lung cancer patient to be predicted.

[0074] Optionally, the immunotherapy score prediction neural network is trained using a training data set including a plurality of training tissue slice images and corresponding immunotherapy score annotations.

[0075] It can be seen that the above-mentioned embodiment of the invention can improve the quality of slice image data according to the neural network algorithm to obtain optimized image data, and then obtain the corresponding image feature vector based on the feature extraction model, and finally accurately predict the corresponding immunotherapy effect score of lung cancer patients through the trained immunotherapy score prediction neural network, thereby realizing accurate immunotherapy prediction directly based on images, reducing prediction costs, and improving prediction efficiency and prediction accuracy.

[0076] As an optional embodiment, the quality optimization module improves the quality of the slice image data based on the neural network algorithm, and obtains a specific method of optimizing the image data, including: The slice image data belonging to the FS image is input into the trained image conversion model to obtain the output FFPE-style optimized image data; the image conversion model is trained by a training data set including multiple training FS images and corresponding FFPE-style image annotations.

[0077] It can be seen that through the above-mentioned optional embodiments, the slice image data belonging to the FS image can be converted into FFPE-style optimized image data with better quality based on the trained image conversion model to improve the quality of the image, thereby assisting in realizing accurate immunotherapy prediction directly based on the image, reducing prediction costs, and improving prediction efficiency and prediction accuracy.

[0078] As an optional embodiment, the feature extraction model is a Phikon model.

[0079] It can be seen that through the above optional embodiments, it is limited to fully extract image features through the Phikon model to improve the accuracy of subsequent predictions, assist in realizing accurate immunotherapy predictions directly based on images, reduce prediction costs, and improve prediction efficiency and prediction accuracy.

[0080] As an optional embodiment, the efficacy prediction module inputs the image feature vector into the trained immunotherapy score prediction neural network to obtain the specific method of the immunotherapy effect corresponding to the lung cancer patient to be predicted, including: Inputting the image feature vector into a plurality of trained immunotherapy score prediction neural networks to obtain the efficacy prediction result of each immunotherapy score prediction neural network for the image feature vector; Calculate the average of all efficacy prediction results to obtain the average efficacy parameter; According to the average efficacy parameters and the preset judgment threshold, the corresponding immunotherapy effect of the lung cancer patient to be predicted is determined.

[0081] It can be seen that through the above optional embodiments, multiple efficacy prediction results can be predicted through multiple trained immunotherapy score prediction neural networks, and average calculation and threshold judgment can be performed, thereby improving the accuracy of efficacy prediction, realizing accurate immunotherapy prediction directly based on images, reducing prediction costs, and improving prediction efficiency and prediction accuracy.

[0082] As an optional embodiment, the immunotherapy score prediction neural network includes an input layer, a Transformer encoder, a fully connected layer, and a classification probability output layer connected in sequence.

[0083] It can be seen that through the above optional embodiments, the network structure of the immunotherapy score prediction neural network is defined, which can be used to accurately predict the therapeutic effect, assist in realizing accurate immunotherapy prediction directly based on images, reduce prediction costs, and improve prediction efficiency and prediction accuracy.

[0084] As an optional embodiment, the immunotherapy score prediction neural network is trained based on the LA-MIL method and optimized by the AdamW optimizer. The initial learning rate is set to 2e-05 and gradually adjusted within 20 training cycles. The batch size is set to 1, and a weighted binary cross entropy loss function is used. After each training cycle, the F1 score on the validation data set is calculated and the model weight with the highest F1 score is selected for saving. Multiple trained immunotherapy score prediction neural networks are obtained through multi-fold training and cross-validation.

[0085] It can be seen that through the above optional embodiments, the training details of the immunotherapy score prediction neural network are defined to train a neural network model with more accurate prediction effects, assist in realizing accurate immunotherapy predictions directly based on images, reduce prediction costs, and improve prediction efficiency and prediction accuracy.

[0086] As an optional embodiment, the specific manner in which the feature extraction module inputs the optimized image data into the feature extraction model to obtain the corresponding image feature vector includes: Acquire patient information of a lung cancer patient to be predicted; Input the patient information into the trained LLM model to obtain the corresponding cancer type information; Determining an image segmentation model corresponding to the cancer type information in a preset image segmentation model library; optionally, the image segmentation model is trained based on a training data set including training case images corresponding to a plurality of cancer type information and core image area annotations; Inputting the optimized image data into the image segmentation model to obtain the key area image corresponding to the optimized image data; Determining a feature extraction model corresponding to the cancer type information in a preset feature extraction model library; optionally, the feature extraction model is trained based on a training pathology image corresponding to the cancer type information; The key area image is input into the feature extraction model to obtain the corresponding image feature vector.

[0087] It can be seen that through the above optional embodiments, the cancer type can be determined based on the LLM model in combination with the patient's information, and then the corresponding image segmentation model and feature extraction model can be determined based on the screening of the corresponding type, so as to achieve more accurate image key area determination and feature extraction, and assist in realizing accurate immunotherapy prediction directly based on the image, reduce prediction costs, and improve prediction efficiency and prediction accuracy.

[0088] As an optional embodiment, the immunotherapy score annotation is determined by the following steps: Obtaining tissue sample data corresponding to the training tissue slice image; Based on the immune infiltration scoring model, determining the immune infiltration score corresponding to the tissue sample data; Calculate the weighted sum average of the immune infiltration scores corresponding to all training tissue section images to obtain a score reference value; optionally, the weighted calculation weight corresponding to each immune infiltration score includes a first weight and a second weight; the first weight is proportional to the image clarity parameter corresponding to the training tissue section image; the second weight is proportional to the score reliability parameter corresponding to the training tissue section image; the score reliability parameter is the similarity between the patient data corresponding to the tissue sample data corresponding to the training tissue section image and the patient data corresponding to the model comparison sample data of the immune infiltration score model; According to the comparison result between the immune infiltration score and the score reference value, the immunotherapy score annotation corresponding to the training tissue section image is determined.

[0089] It can be seen that through the above optional embodiments, the immune infiltration score of the tissue sample can be determined based on the immune infiltration scoring model, and then the appropriate scoring reference value can be determined based on the weighted sum average calculation, so as to finally obtain accurate immunotherapy score annotations for the training samples, so that the trained immunotherapy score prediction neural network has higher prediction accuracy, and assists in realizing accurate immunotherapy prediction directly based on images, reducing prediction costs, and improving prediction efficiency and prediction accuracy.

[0090] Embodiment 3 See also Figure 3 , Figure 3 This is another lung cancer immunotherapy effect prediction system disclosed in an embodiment of the present invention. Figure 3The described lung cancer immunotherapy effect prediction system is applied to a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). Figure 3 As shown, the lung cancer immunotherapy effect prediction system may include: A memory 301 storing executable program codes; a processor 302 coupled to the memory 301; The processor 302 calls the executable program code stored in the memory 301 to execute the steps of the method for predicting the effect of immunotherapy for lung cancer described in the first embodiment.

[0091] Embodiment 4 An embodiment of the present invention discloses a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps of the method for predicting the effect of immunotherapy for lung cancer described in the first embodiment.

[0092] Embodiment 5 An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps of the method for predicting the effect of immunotherapy for lung cancer described in Example 1.

[0093] The above describes specific embodiments of the present specification, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0094] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0095] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0096] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may be in the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of this specification may be in the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0097] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0098] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0099] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0100] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0101] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0102] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0103] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0104] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0105] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0106] Finally, it should be noted that the method and system for predicting the effect of immunotherapy for lung cancer disclosed in the embodiments of the present invention only disclose the preferred embodiments of the present invention, which are only used to illustrate the technical scheme of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical schemes described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical schemes from the spirit and scope of the technical schemes of the embodiments of the present invention.

Claims

1. A method for predicting the effect of immunotherapy for lung cancer, characterized in that: The method comprises: Acquire slice image data of biological tissue of a patient to be diagnosed with lung cancer; Based on a neural network algorithm, the quality of the slice image data is improved to obtain optimized image data; Inputting the optimized image data into a feature extraction model to obtain a corresponding image feature vector; The image feature vector is input into a trained immunotherapy score prediction neural network to obtain the corresponding immunotherapy effect of the lung cancer patient to be predicted; the immunotherapy score prediction neural network is trained by a training data set including multiple training tissue section images and corresponding immunotherapy score annotations.

2. The method for predicting the effect of lung cancer immunotherapy according to claim 1, characterized in that: The method of improving the quality of the slice image data based on a neural network algorithm to obtain optimized image data includes: The slice image data belonging to the FS image is input into a trained image conversion model to obtain output FFPE-style optimized image data; the image conversion model is trained by a training data set including multiple training FS images and corresponding FFPE-style image annotations.

3. The method for predicting the effect of lung cancer immunotherapy according to claim 1, characterized in that: The feature extraction model is the Phikon model.

4. The method for predicting the effect of immunotherapy for lung cancer according to claim 1, characterized in that: The step of inputting the image feature vector into a trained immunotherapy score prediction neural network to obtain the immunotherapy effect corresponding to the lung cancer patient to be predicted includes: Inputting the image feature vector into a plurality of trained immunotherapy score prediction neural networks to obtain a therapeutic effect prediction result of each immunotherapy score prediction neural network for the image feature vector; Calculating the average of all the efficacy prediction results to obtain an average efficacy parameter; The immunotherapy effect corresponding to the lung cancer patient to be predicted is determined according to the average efficacy parameter and a preset judgment threshold.

5. The method for predicting the effect of lung cancer immunotherapy according to claim 1, characterized in that: The immunotherapy score prediction neural network includes an input layer, a Transformer encoder, a fully connected layer, and a classification probability output layer connected in sequence.

6. The method for predicting the effect of lung cancer immunotherapy according to claim 4, characterized in that: The immunotherapy score prediction neural network was trained based on the LA-MIL method and optimized by the AdamW optimizer. The initial learning rate was set to 2e-05 and gradually adjusted within 20 training cycles. The batch size was set to 1. A weighted binary cross entropy loss function was used. After each training cycle, the F1 score on the validation data set was calculated and the model weight with the highest F1 score was selected for saving. Multiple trained immunotherapy score prediction neural networks were obtained through multi-fold training and cross-validation.

7. The method for predicting the effect of lung cancer immunotherapy according to claim 2, characterized in that: The step of inputting the optimized image data into a feature extraction model to obtain a corresponding image feature vector comprises: Acquiring patient information of the lung cancer patient to be predicted; Inputting the patient information into the trained LLM model to obtain corresponding cancer type information; Determining an image segmentation model corresponding to the cancer type information in a preset image segmentation model library; the image segmentation model is trained based on a training data set including a plurality of training case images corresponding to the cancer type information and core image area annotations; Inputting the optimized image data into the image segmentation model to obtain a key area image corresponding to the optimized image data; Determining a feature extraction model corresponding to the cancer type information in a preset feature extraction model library; the feature extraction model is trained based on a training pathology image corresponding to the cancer type information; The key area image is input into the feature extraction model to obtain a corresponding image feature vector.

8. The method for predicting the effect of lung cancer immunotherapy according to claim 1, characterized in that: The immunotherapy score annotation is determined by the following steps: Acquiring tissue sample data corresponding to the training tissue slice image; Determining the immune infiltration score corresponding to the tissue sample data based on the immune infiltration score model; Calculate the weighted sum average of the immune infiltration scores corresponding to all the training tissue section images to obtain a score reference value; wherein the weighted calculation weight corresponding to each immune infiltration score includes a first weight and a second weight; the first weight is proportional to the image clarity parameter corresponding to the training tissue section image; the second weight is proportional to the score reliability parameter corresponding to the training tissue section image; the score reliability parameter is the similarity between the patient data corresponding to the tissue sample data corresponding to the training tissue section image and the patient data corresponding to the model comparison sample data of the immune infiltration score model; According to the comparison result between the immune infiltration score and the score reference value, the immunotherapy score label corresponding to the training tissue section image is determined.

9. A lung cancer immunotherapy effect prediction system, characterized in that: The system comprises: An image acquisition module, used to acquire slice image data of biological tissue of a patient to be diagnosed with lung cancer; A quality optimization module, used for improving the quality of the slice image data based on a neural network algorithm to obtain optimized image data; A feature extraction module, used for inputting the optimized image data into a feature extraction model to obtain a corresponding image feature vector; The efficacy prediction module is used to input the image feature vector into a trained immunotherapy score prediction neural network to obtain the corresponding immunotherapy effect of the lung cancer patient to be predicted; the immunotherapy score prediction neural network is trained by a training data set including multiple training tissue slice images and corresponding immunotherapy score annotations.

10. A lung cancer immunotherapy effect prediction system, characterized in that: The system comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the method for predicting the effect of lung cancer immunotherapy according to any one of claims 1-8.

Images