Method for cross-modal prediction of immunohistochemical molecular phenotype based on HE digital pathological section

Through a cross-modal prediction method based on H&E digital pathology slides, a convolutional neural network is used to automatically learn image features, which solves the problems of low efficiency, high cost and poor repeatability of immunohistochemistry molecular phenotyping examination, achieves accurate diagnosis of lung cancer subtypes and reduces costs, and provides model interpretability and standardization tools.

CN120808862APending Publication Date: 2025-10-17YANGTZE DELTA REGION INST (QUZHOU) UNIV OF ELECTRONIC SCI & TECH OF CHINA
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202510634457.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-05-16
Publication Date
2025-10-17

AI Technical Summary

Technical Problem

Existing immunohistochemical molecular phenotyping methods have problems such as long experimental cycle, high cost, highly subjective results, and poor reproducibility. In particular, it is difficult to achieve rapid and accurate histological subtype identification in the diagnosis of lung cancer.

Method used

A cross-modal prediction method based on H&E digital pathology slides was adopted. The convolutional neural network model CNN was trained through transfer learning. The original features were fused with attention features to automatically learn the key areas in the image and predict the immunohistochemical molecular phenotype.

Benefits of technology

It improves the diagnostic accuracy of poorly differentiated lung cancer subtypes, reduces dependence on expensive tests, reduces diagnostic costs, provides visualization tools to explain model decisions, enhances the standardization and interpretability of results, and is suitable for comparison of results among multiple laboratories.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120808862A_ABST
    Figure CN120808862A_ABST
Patent Text Reader

Abstract

The invention relates to a method based on Hamp; the invention discloses a method for cross-modal prediction of immunohistochemical molecular phenotypes of E digital pathological sections. The problems that in the prior art, an immunohistochemical molecular phenotype detection method is long in consumed time and poor in diagnosis accuracy are solved. The method comprises the following steps: S1, data preprocessing: performing Hamp processing on the data according to annotation information; e, segmenting the digital pathological section and distributing labels; s2, using transfer learning to train a convolutional neural network model CNN as a first-stage model, and outputting a prediction result; s3, post-processing a prediction result output by the first-stage model, obtaining a slice level label according to a label distribution strategy, and inputting a second-stage model by taking the prediction result as a tumor slice; and S4, cross-modal prediction: distributing labels of nine biomarkers for each section to train a second-stage model. The method has the advantages that the diagnosis accuracy of low-differentiation lung cancer subtypes and the classification accuracy of complex and heterogeneity tumor tissues are improved, and the acceptability and the transformability of an algorithm are improved.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present application relates to the field of bioinformatics and computer science, and particularly relates to a method for predicting immunohistochemical molecular phenotype based on H&E digital pathology sections across modalities. BACKGROUND

[0002] Lung cancer is mainly divided into two pathological histological types: non-small cell lung cancer (NSCLC) and small cell lung cancer (SCLC), of which NSCLC accounts for about 85%. The histological subtype identification of lung cancer is crucial for individualized treatment, affecting the eligibility of molecular testing and the selection of treatment strategies. However, due to the polymorphism and atypia of histological features, many cases are difficult to be classified clearly by traditional methods, resulting in inaccurate diagnosis and improper treatment.

[0003] With the wide application of new diagnostic techniques such as transbronchial lung biopsy (TBLB) and percutaneous needle lung biopsy (PNLB), the use of biopsy specimens in lung cancer screening and detection of advanced patients is increasing. However, in actual clinical practice, only a small number of lung cancer patients are recommended for IHC examination, because the amount of biopsy specimen tissue is limited and the cost of examination is high, it is often difficult to be classified clearly by IHC, resulting in the diagnosis result being only "non-small cell lung cancer-not otherwise specified (NSCLC-NOS) type".

[0004] Currently, the methods for predicting immunohistochemical molecular phenotype mainly fall into two categories: one is traditional experimental wet experiment, and the other is dry experiment in the field of computational biology and pathology, which mainly relies on computing and data analysis techniques to predict immunohistochemical molecular phenotype. The present application mainly develops a new method for cross-modal prediction, which can predict immunohistochemical molecular phenotype from H&E digital pathology sections. At present, other methods similar to the present application are:

[0005] 1. Wet experiments based on traditional experimental science: This is the traditional method of immunohistochemical molecular phenotyping, involving actual biological sample processing and chemical reactions. Wet experiments include various techniques such as immunofluorescence, immunoenzyme labeling, direct method, indirect method, peroxidase-antiperoxidase (PAP) method, avidin-biotin-peroxidase complex (ABC) method, and polymer linkage method. These methods use specific antibodies to bind to antigens, and then use enzyme or fluorescent markers to detect and locate specific molecules in cells or tissues. For example, immunofluorescence uses fluorescently labeled antibodies to detect specific antigens in the sample. Immunoenzyme labeling uses enzyme-labeled antibodies to bind to antigens, and then adds enzyme substrates to generate colored insoluble products or particles with certain electron density, allowing the localization of various antigen components on the cell surface and within the cell to be studied by light microscopy or electron microscopy.

[0006] 2. Dry experiments based on computational methods: Morphology-molecular integration is a computational biology and pathology method that combines digital information on immune cell infiltration and molecular data to create a comprehensive "cancer immune map"[4]. This method uses digital information collected by high-resolution tissue imaging technology, combined with molecular data including gene expression and protein expression, to create a multidimensional data set through bioinformatics tools and algorithms to analyze the characteristics of the immune map.

[0007] The disadvantages of the prior art The disadvantages of wet experimental methods, long experimental period, different laboratories have different differences, and patients cannot meet the experimental needs.

[0008] Wet experiments based on traditional experimental science: Wet experiments involve multiple steps such as sample fixation, sectioning, deparaffinization, rehydration, antigen repair, blocking, antibody incubation, color development, and mounting, each of which requires precise control and has high technical requirements. In addition, the time consumption of wet experiments is also a problem, from sample preparation to result acquisition may take several hours or even several days, which is not efficient enough in the demand for rapid diagnosis. The cost problem cannot be ignored, wet experiments require specific reagents and equipment, especially when dealing with a large number of samples or using special antibodies, the cost may increase significantly. The results of wet experiments may also be affected by the operator's technology and experience, which leads to the subjectivity and variability of the results, affecting the accuracy and repeatability of clinical diagnosis.

[0009] Dry lab experiments based on computational methods: The acquisition of morpho-molecular integrated data is not only time-consuming and costly, but for some rare or difficult-to-obtain samples, data collection is even more difficult. Integrating and analyzing data from different sources and types, such as images and gene expression data, requires reliance on complex bioinformatics tools and algorithms. This not only requires professional knowledge and skills, but may also be limited by the level of development of existing technologies and methods. In addition, standardization and repeatability issues are also obstacles to the promotion of morpho-molecular integration between different laboratories or research teams. It is essential to standardize data collection, processing and analysis, otherwise it may affect the repeatability of research results and comparison between different studies.

[0010] In summary, in the field of immunohistochemical molecular phenotype examination, wet experiments and dry experiments as two main methods, each has some shortcomings. Wet experiments involve multiple steps, each step needs to be accurately controlled, time-consuming, and not efficient enough in the demand for rapid diagnosis. The cost will increase significantly when dealing with a large number of samples or using special antibodies. The results are affected by the operator's technology and experience, which leads to subjectivity and variability of the results, affecting the accuracy and repeatability of clinical diagnosis. The morpho-molecular integration method is time-consuming and costly to obtain data, and relies on complex bioinformatics tools and algorithms. The standardization and repeatability of morpho-molecular integration methods are also obstacles to the promotion of morpho-molecular integration between different laboratories or research teams. SUMMARY

[0011] The purpose of the present application is to solve the above problems, and provide a method for predicting immunohistochemical molecular phenotype based on H&E digital pathology sections across modalities.

[0012] To achieve the above purpose, the following technical solutions are adopted in the present application: A method for predicting immunohistochemical molecular phenotype based on H&E digital pathology sections across modalities, comprising the following steps:

[0013] S1, data preprocessing: according to the annotation information, the H&E digital pathology sections are divided and labeled;

[0014] S2, using transfer learning to train a convolutional neural network model CNN as a first stage model and output a prediction result;

[0015] S3, post-processing the prediction result output by the first stage model, obtaining the slice-level label according to the label allocation strategy, and inputting the prediction result as a tumor section into a second stage model;

[0016] S4, cross-modal prediction: training a second stage model for each section to assign labels of nine biomarkers.

[0017] In the method for predicting immunohistochemical molecular phenotypes across modalities based on H&E digital pathological sections described above, step S1 comprises the following steps:

[0018] S11, taking the H&E digital pathological section as input, obtaining the tissue mask and tumor mask through the annotation file;

[0019] S12, dividing the digital pathological section into 256*256 pixel size sections according to the obtained mask;

[0020] S13, obtaining the label of each section according to the annotation information;

[0021] S14, feeding the preprocessed section into the first stage model, and the output of the first stage model is a probability score between 0 and 1;

[0022] S15, assigning the prediction result according to the set label assignment strategy.

[0023] In the method for predicting immunohistochemical molecular phenotypes across modalities based on H&E digital pathological sections described above, in step S2, the network basic architecture of the convolutional neural network model CNN is EfficientNetV2-S, after several training iterations, the network model is optimized, the six-classification model with the highest prediction accuracy and stability is saved, and is used for subsequent feature extraction.

[0024] In step S2, the prediction result output by the first stage model includes a probability vector of six components, each component representing the probability value of the section being predicted by the first stage model as lung adenocarcinoma LUAD, lung squamous cell carcinoma LUSC, small cell lung cancer SCLC, normal NORMAL, lung nodule G, and lung inflammation PN.

[0025] In step S3, after obtaining the section probability value, a rule is set to determine the ownership of the section label.

[0026] In the method for predicting immunohistochemical molecular phenotypes across modalities based on H&E digital pathological sections described above, the rule includes the following contents:

[0027] ① If the prediction probability of NORMAL exceeds 0.9, it is inferred that the section is NORMAL;

[0028] ② Otherwise, if the probability of lung adenocarcinoma LUAD, lung squamous cell carcinoma LUSC, and small cell lung cancer SCLC is greater than 0.1, the label is assigned to the tumor category with the maximum probability;

[0029] ③ Otherwise, if the prediction value of lung nodule G or lung inflammation PN is higher than other threshold values, the corresponding category label is assigned;

[0030] IV. If none of the above conditions are met, the slice will be labeled with the class having the maximum probability value.

[0031] In steps S3-S4, the slices predicted as tumor are input into the second stage model, and a label of whether each of the nine biomarkers is expressed is assigned to each slice, with the label being set to 0 (not expressed) or 1 (expressed) depending on whether it is expressed.

[0032] In step S4, the second stage model is trained, and the second stage model outputs a predicted probability of whether the nine biomarkers are expressed, and the final output values of CK7, TTF-1, and NapsinA are used as a LUAD score using the average probability method and the threshold method.

[0033] In the above method for predicting an immunohistochemical molecular phenotype based on an H&E digital pathology slice across modalities, the final output values of CK7, TTF-1, and NapsinA are used as a LUAD score.

[0034] The output values of CK5 / 6, P63, and P40 are used as a LUSC score.

[0035] TTF-1, CD56, Syn, and CgA are used as a SCLC score.

[0036] The subtype label is assigned according to the highest score.

[0037] In the above method for predicting an immunohistochemical molecular phenotype based on an H&E digital pathology slice across modalities:

[0038] CK7 represents Cytokeratin 7: Cell keratin 7;

[0039] TTF-1 represents Thyroid Transcription Factor-1: Thyroid transcription factor-1;

[0040] NapsinA represents Aspartic Protease A;

[0041] CK5 / 6 represents Cytokeratin 5 / 6: Cell keratin 5 / 6;

[0042] P63 represents Protein P63: P63 protein;

[0043] P40 represents Protein P40: P40 protein;

[0044] Syn represents Synaptophysin: Synaptic protein;

[0045] CgA represents Chromogranin A: Chromogranin A:

[0046] LUAD means lung adenocarcinoma: Lung Adenocarcinoma Cancer;

[0047] LUSC means lung squamous cell carcinoma: Lung Squamous Cell Cancer;

[0048] SCLC means small cell lung cancer: Small Cell Lung Cancer.

[0049] Compared with the prior art, the advantages of the present application are:

[0050] 1. The diagnostic accuracy of low differentiation lung cancer subtypes is improved, especially in the case of non-small cell lung cancer not specified (NSCLC-NOS); the diagnostic cost is reduced by reducing the dependence on expensive immunohistochemical (IHC) detection; the problem of insufficient tissue samples is solved, so that accurate diagnosis can be made even with limited sample size.

[0051] 2. In the technical implementation of feature fusion, the original feature and attention feature fusion method is adopted, which allows the model to automatically learn and focus on the key areas or features in the image, so as to more accurately capture the morphological and textural information related to specific immunohistochemical markers, and help to improve the classification accuracy of complex and heterogeneous tumor tissues.

[0052] 3. In the modeling idea, the clinical transformation is adhered to as the guide, in order to avoid the influence of the black box characteristics of CNN on the explainability of the model, only the CNN is used as a link in the overall modeling process, the heat map generated by the attention mechanism can intuitively show which areas are most important for model decision, providing a visual tool to explain the working principle of the model. In addition, the threshold classification method designed in accordance with the clinical diagnosis and treatment standard and the conventional maximum threshold classification method significantly improve the clinical detection sensitivity of serious diseases, and are more in line with clinical needs, improving the acceptability and transformability of the algorithm. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1 is a method flowchart of the present application;

[0054] Figure 2 is a schematic diagram of obtaining tissue mask and tumor mask by annotation file in the present application;

[0055] Figure 3 is a six-classification model schematic diagram in the present application;

[0056] Figure 4 is a schematic diagram of original feature and attention feature fusion in the present application. DETAILED DESCRIPTION

[0057] The present application will be further described in detail below in combination with the drawings and specific embodiments.

[0058] As shown in Figures 1-4 A method for predicting immunohistochemical molecular phenotype based on H&E digital pathology sections cross-modal, comprising the following steps:

[0059] S1, data preprocessing: according to the annotation information, the H&E digital pathology sections are divided and labeled;

[0060] S2, using transfer learning to train a convolutional neural network model CNN as the first stage model and output the prediction results;

[0061] S3, post-processing the prediction results output by the first stage model, obtaining the slice level label according to the label allocation strategy, and taking the prediction results as the input of the second stage model for the tumor section;

[0062] S4, cross-modal prediction: training the second stage model for each section to assign nine biomarker labels, and outputting the prediction probability and score.

[0063] Cross-modal prediction has excellent scalability, which can predict ALK, PD-1, PD-L1 protein, and EGFR and KRAS gene mutations. By integrating these prediction capabilities into a unified framework, it can provide clinicians with more comprehensive tumor molecular feature information, thereby guiding the development of personalized treatment strategies.

[0064] Since H&E staining is part of routine pathological diagnosis, almost all tissue biopsies will undergo this staining step. Therefore, predicting immunohistochemical results from H&E images can avoid repeated sampling or additional preparation of specialized immunohistochemical sections, saving time and cost. Secondly, this method improves standardization, as there may be differences in the performance of immunohistochemistry by different laboratories, while the prediction model based on H&E images is stable and not affected by specific laboratory conditions, which helps to achieve more extensive standardized applications. In addition, the automated prediction model reduces subjective interpretation errors and provides more objective and consistent results. More importantly, combining advanced computing methods such as deep learning can extract rich morphological features from H&E images and correlate them with known immunohistochemical markers, building a complex biomarker network that is easy to expand to predict the expression of other tumor-related molecular markers, such as ALK, PD-L1, KRAS, etc., has great clinical application value and social and economic value. For diseases that require continuous monitoring of treatment response and disease progression, such as cancer, using H&E image prediction models can achieve dynamic monitoring without increasing the burden on patients.

[0065] Step S1 includes the following steps:

[0066] S11, input H&E digital pathology slices, obtain tissue mask and tumor mask through annotation file;

[0067] S12, divide the H&E digital pathology slices into 256*256 pixel size slices according to the obtained mask;

[0068] S13, obtain the label of each slice according to the annotation information;

[0069] S14, send the preprocessed slice into the first stage model, and the output of the first stage model is a probability score between 0 and 1;

[0070] S15, assign the prediction result according to the set label assignment strategy.

[0071] In step S1, the immunohistochemical molecular phenotype is predicted from the slice, the first stage model is used to quickly obtain a result similar to immunohistochemistry, reduce the dependence on expensive immunohistochemical detection, and reduce the cost of diagnosis.

[0072] In step S2, the network basic architecture of the convolutional neural network model CNN is EfficientNetV2-S, after several training iterations, the network model is optimized, the six-classification model with the highest prediction accuracy and stability is saved, and is used for subsequent feature extraction.

[0073] In step S2, the prediction result output by the first stage model includes a probability vector of six components, each component respectively represents the probability value of the slice being predicted by the first stage model as lung adenocarcinoma LUAD, lung squamous carcinoma LUSC, small cell lung cancer SCLC, normal NORMAL, lung nodule G, and lung inflammation PN.

[0074] According to the size of the probability value and the threshold setting, the final classification conclusion is obtained.

[0075] In step S3, after obtaining the slice probability value, a rule is set to determine the ownership of the slice label.

[0076] The rule includes the following contents:

[0077] ① If the prediction probability of NORMAL is greater than 0.9, it is inferred that the slice is NORMAL;

[0078] ② Otherwise, if the probability of lung adenocarcinoma LUAD, lung squamous carcinoma LUSC, and small cell lung cancer SCLC is greater than 0.1, the label is assigned to the tumor category with the maximum probability;

[0079] ③ Otherwise, if the prediction value of lung nodule G or lung inflammation PN is higher than other threshold values, the corresponding category label is assigned;

[0080] IV. If none of the above conditions are met, the slice will be labeled as the class with the highest probability value.

[0081] In steps S3-S4, the slices predicted as tumors are input into the second stage model, and a label of whether each slice expresses nine biomarkers is assigned, with the label set to 0 (not expressed) or 1 (expressed) according to whether it is expressed.

[0082] By combining multiple biomarkers highly related to specific lung cancer subtypes, the biological characteristics of the tumor are more comprehensively reflected, and the instability of the results caused by expression heterogeneity or technical reasons of a single marker is reduced.

[0083] Step S4 includes the following steps:

[0084] S41, using EfficientNetV2-S to extract original features,

[0085] S42, passing the original features of the second stage model through maximum pooling and average pooling to capture the maximum and average values of the feature map;

[0086] S43, using a 1x1 convolutional layer to convert the spliced feature map into a single-channel attention map, and multiplying it with the original feature to obtain an attention feature;

[0087] S44, introducing two hyperparameters to balance the contribution between the two feature maps, and outputting the prediction probability through a fully connected layer.

[0088] In step S4, the second stage model is trained, and the prediction probability of whether the nine biomarkers are expressed is output by the second stage model. The average probability method and threshold method are used to take the final output values of CK7, TTF-1 and NapsinA as the LUAD score.

[0089] Preferably, the final output values of CK7, TTF-1 and NapsinA are taken as the LUAD score;

[0090] The output values of CK5 / 6, P63 and P40 are taken as the LUSC score;

[0091] TTF-1, CD56, Syn, and CgA are taken as the SCLC score;

[0092] According to the highest score, the subtype label is assigned.

[0093] The comprehensive score can make up for the shortcomings of a single marker by considering the performance of multiple markers, providing more reliable results. Second, the comprehensive score provides a standardized evaluation framework, making it easier to compare and communicate results between different laboratories, reducing differences caused by subjective interpretation. More in line with clinical diagnosis examples. Greatly improve the interpretability and clinical acceptability of the model, promote the clinical transformation of the algorithm.

[0094] In detail, CK7 represents Cytokeratin 7: Cytokeratin 7;

[0095] TTF-1 represents Thyroid Transcription Factor-1: Thyroid Transcription Factor-1;

[0096] Napsin A represents Aspartic Protease A;

[0097] CK5 / 6 represents Cytokeratin 5 / 6: Cytokeratin 5 / 6;

[0098] P63 represents Protein P63: Protein P63;

[0099] P40 represents Protein P40: Protein P40;

[0100] Syn represents Synaptophysin: Synaptophysin;

[0101] CgA represents Chromogranin A: Chromogranin A:

[0102] LUAD represents Lung Adenocarcinoma Cancer: Lung Adenocarcinoma Cancer;

[0103] LUSC represents Lung Squamous Cell Cancer: Lung Squamous Cell Cancer;

[0104] SCLC represents Small Cell Lung Cancer: Small Cell Lung Cancer.

[0105] In summary, the principle of the present embodiment is to extract rich morphological features from H&E digital pathology slice images combined with advanced computing methods of deep learning, and correlate them with known immunohistochemical markers, to construct a complex biomarker network. It is easy to expand to be applied to the expression prediction of other molecular markers related to tumors. For diseases that need to continuously monitor treatment response and disease progression, using H&E image prediction model can realize dynamic change monitoring without increasing the burden of patients.

[0106] The specific embodiments described herein are merely illustrative of the spirit of the application. Various modifications or changes in the specific embodiments described herein can occur to those skilled in the art to which the application pertains without departing from the spirit of the application, and it is understood that such modifications or changes are to be considered as within the scope of the application as defined by the appended claims.

Claims

1. A method for cross-modality prediction of immunohistochemical molecular phenotypes based on H&E digital pathology slides, characterized in that: The following steps are involved: S1. Data preprocessing: Segment H&E digital pathology slides and assign labels based on annotation information; S2. Use transfer learning to train the convolutional neural network model CNN as the first stage model and output the prediction results; S3. Post-process the prediction results output by the first-stage model, obtain slice-level labels according to the label assignment strategy, and use the prediction results as the tumor slices for the second-stage model input; S4. Cross-modal prediction: Assign nine biomarker labels to each H&E digital pathology slide to train the second-stage model and output the prediction probability and score.

2. The method for cross-modality prediction of immunohistochemical molecular phenotypes based on H&E digital pathology sections according to claim 1, characterized in that: Step S1 includes the following steps: S11, taking H&E digital pathology slides as input, obtaining tissue masks and tumor masks through annotation files; S12, dividing the H&E digital pathology slide into slices of 256*256 pixels according to the obtained mask; S13, obtaining a label for each slice according to the annotation information; S14, sending the pre-processed slices to the first stage model, the output of the first stage model is a probability score between 0 and 1; S15. Allocate the prediction results according to the set label allocation strategy.

3. The method for cross-modality prediction of immunohistochemical molecular phenotypes based on H&E digital pathology sections according to claim 2, characterized in that: In step S2, the basic network architecture of the convolutional neural network model CNN is EfficientNetV2-S. After several training iterations, the network model is optimized and the six-category model with the highest prediction accuracy and stability is saved for subsequent feature extraction.

4. The method for cross-modality prediction of immunohistochemical molecular phenotypes based on H&E digital pathology sections according to claim 3, characterized in that: In step S2, the prediction result output by the first-stage model includes a probability vector of six components, each component representing the probability value of the slice predicted by the first-stage model as lung adenocarcinoma LUAD, lung squamous cell carcinoma LUSC, small cell lung cancer SCLC, normal NORMAL, lung nodule G, and pneumonia PN.

5. The method for cross-modality prediction of immunohistochemical molecular phenotypes based on H&E digital pathology sections according to claim 4, characterized in that: In step S3, after obtaining the H&E digital pathology slice probability value, a rule is set to determine the attribution of the slice label.

6. The method for cross-modality prediction of immunohistochemical molecular phenotypes based on H&E digital pathology sections according to claim 4, characterized in that: The rules include the following: ① If the predicted probability of NORMAL exceeds 0.9, the slice is inferred to be NORMAL; ② Otherwise, if the probability of the lung adenocarcinoma LUAD, lung squamous cell carcinoma LUSC, or small cell lung cancer SCLC category is greater than 0.1, the label is assigned to the tumor category with the highest probability; ③ Otherwise, if the predicted value of pulmonary nodule G or pneumonia PN is higher than other thresholds, the corresponding category label is assigned; ④If any of the above conditions are not met, the slice will be labeled as the category with the maximum probability value.

7. The method for cross-modality prediction of immunohistochemical molecular phenotypes based on H&E digital pathology sections according to claim 6, characterized in that: In steps S3-S4, the slices predicted to be tumors are used as input to the second-stage model, and each slice is assigned a label indicating whether the nine biomarkers are expressed, with the label set to 0 (not expressed) or 1 (expressed) depending on whether they are expressed.

8. The method for cross-modality prediction of immunohistochemical molecular phenotypes based on H&E digital pathology sections according to claim 7, characterized in that: In step S4, the second-stage model is trained, and the second-stage model is used to output the predicted probabilities of whether the nine biomarkers are expressed; Step S4 includes the following steps: S41, use EfficientNetV2-S to extract original features, S42, the original features of the second stage model are subjected to maximum pooling and average pooling to capture the maximum value and average value of the feature map; S43, use a 1x1 convolution layer to convert the concatenated feature map into a single-channel attention map, and multiply it with the original feature to obtain the attention feature; S44. Two hyperparameters are introduced to balance the contribution between the two feature maps, and the predicted probability is output through the fully connected layer.

9. The method for cross-modality prediction of immunohistochemical molecular phenotypes based on H&E digital pathology slides according to claim 8, characterized in that: The average probability method and threshold method were used to determine the final output values ​​of CK7, TTF-1, and NapsinA as the LUAD score; the output values ​​of CK5 / 6, P63, and P40 as the LUSC score; and TTF-1, CD56, Syn, and CgA as the SCLC score. Subtype labels were assigned based on the highest score.

10. The method for cross-modality prediction of immunohistochemical molecular phenotypes based on H&E digital pathology slides according to claim 9, characterized in that: The CK7 stands for cytokeratin 7; TTF-1 stands for Thyroid Transcription Factor-1; NapsinA indicates aspartic proteinase A; CK5 / 6 indicates cytokeratin 5 / 6; P63 represents P63 protein: ProteinP63; P40 represents P40 protein: ProteinP40; Syn stands for synaptophysin; CgA stands for Chromogranin A: LUAD stands for lung adenocarcinoma: LungAdenocarc inomaCancer; LUSC stands for lung squamous cell carcinoma: Lung Squamous Cell Cancer; SCLC stands for small cell lung cancer.