Immunohistochemical staining result intelligent analysis and report generation system based on deep learning

Abstract

The invention discloses an immunohistochemical staining result intelligent analysis and report generation system based on deep learning, and relates to the crossing field of medical image processing and artificial intelligence technology, and the system comprises a digital slice scanning module which converts an immunohistochemical staining slide into a high-resolution full-view digital image; the image preprocessing and region identification module is used for carrying out normalization processing on the image, and identifying and segmenting a tumor cell enriched region of interest based on a deep learning model; the cell segmentation and classification module is used for carrying out pixel-level segmentation on the cells in the region of interest by adopting the trained instance segmentation model; the quantitative analysis module is used for automatically calculating an immunohistochemical quantitative scoring index based on a classification result; and the intelligent report generation module is used for generating an immunohistochemical staining result analysis report containing the standardized diagnosis description based on the scoring index, the cell classification result and the corresponding image. According to the invention, an immunohistochemical staining result analysis report can be automatically generated in a standardized and high-precision manner.

Description

Technical Field

[0001] This application relates to the intersection of medical image processing and artificial intelligence technologies, and in particular to an intelligent analysis and report generation system for immunohistochemical staining results based on deep learning. Background Technology

[0002] Immunohistochemistry (IHC) is an indispensable technique in pathological diagnosis and research, used to locate specific antigens (such as proteins) on tissue sections, thereby enabling disease diagnosis, classification, prognosis assessment, and guidance of targeted therapy. Currently, the interpretation of IHC results mainly relies on subjective evaluation by pathologists under a microscope. This traditional method has several significant drawbacks: 1. Subjectivity and variability: Different observers, and even the same observer at different times, may have different judgments on staining intensity and the proportion of positive cells, resulting in low reproducibility of results.

[0003] 2. Inefficiency: Manually counting and evaluating large numbers of cells is time-consuming and laborious, especially when tumor heterogeneity is high or when multiple fields of view need to be analyzed.

[0004] 3. Inaccurate quantification: The human eye has difficulty in accurately and continuously quantifying staining intensity, and can usually only make rough classifications (such as 0, 1+, 2+, 3+), which may lead to the loss of potential biological information.

[0005] 4. Non-standardized reports: The format and content of diagnostic reports are highly dependent on physicians' personal habits, lacking a unified standard, which is not conducive to cross-sectional data comparison and scientific research analysis. Summary of the Invention

[0006] The purpose of this application is to provide an automated, standardized, and highly accurate intelligent analysis and report generation system for immunohistochemical staining results.

[0007] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a deep learning-based intelligent analysis and report generation system for immunohistochemical staining results, including: The digital slide scanning module is used to convert immunohistochemical stained slides into high-resolution full-view digital images; The image preprocessing and region recognition module is connected to the digital slice scanning module and is used to perform color normalization processing on the full-view digital image and identify and segment the region of interest enriched with tumor cells based on a deep learning model. The cell segmentation and classification module, connected to the image preprocessing and region recognition module, is used to perform pixel-level segmentation of each cell in the region of interest using a trained instance segmentation model, and automatically classify each segmented cell according to the staining intensity to obtain the classification result. The quantitative analysis module, connected to the cell segmentation and classification module, is used to automatically calculate at least one internationally recognized immunohistochemical quantitative scoring index based on the classification results. The intelligent report generation module, connected to the quantitative analysis module, is used to generate an immunohistochemical staining result analysis report containing standardized diagnostic descriptions based on the immunohistochemical quantitative scoring indicators, the classification results of each segmented cell, and the image corresponding to the cell.

[0008] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application provides a deep learning-based intelligent analysis and report generation system for immunohistochemical staining results. The system includes: a digital slide scanning module, an image preprocessing and region recognition module, a deep learning-based cell segmentation and classification module, a quantitative analysis module, and an intelligent report generation module. This application can automatically identify tumor cell regions, accurately segment cells, and provide multiple internationally recognized quantitative scores, ultimately automatically generating a structured pathological diagnostic report. This system significantly improves the objectivity, accuracy, and efficiency of IHC result analysis and has significant clinical application value. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a schematic diagram of the functional modules of an intelligent analysis and report generation system for immunohistochemical staining results based on deep learning, provided in an embodiment of this application.

[0011] Figure 2 This is a schematic diagram of the functional modules of an intelligent analysis and report generation system for immunohistochemical staining results provided in another embodiment of this application. Detailed Implementation

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0013] Currently, the interpretation of immunohistochemical results mainly relies on subjective evaluation by pathologists under a microscope. In recent years, although some digital pathology image analysis software has been introduced, most rely on traditional image processing algorithms (such as color thresholding), which suffer from poor accuracy and robustness in situations with complex backgrounds, uneven staining, or overlapping cells. Deep learning technology, especially convolutional neural networks (CNNs), has shown significant advantages in image segmentation and classification tasks; however, a systematic and targeted approach to applying it to the entire IHC process and generating standardized reports remains incomplete.

[0014] Therefore, the purpose of this application is to overcome the shortcomings of the prior art and provide an automated, standardized, and high-precision intelligent analysis and report generation system for immunohistochemical staining results, so as to reduce human error and improve analysis efficiency and the reproducibility of results.

[0015] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, this application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0016] like Figure 1 As shown, this embodiment provides a deep learning-based intelligent analysis and report generation system for immunohistochemical staining results, including: The digital slide scanning module is used to convert immunohistochemical stained slides into high-resolution full-view digital images; The image preprocessing and region recognition module is connected to the digital slice scanning module and is used to perform color normalization processing on the full-view digital image and identify and segment the region of interest enriched with tumor cells based on a deep learning model. The cell segmentation and classification module, connected to the image preprocessing and region recognition module, is used to perform pixel-level segmentation of each cell in the region of interest using a trained instance segmentation model, and automatically classify each segmented cell according to the staining intensity to obtain the classification result. The quantitative analysis module, connected to the cell segmentation and classification module, is used to automatically calculate at least one internationally recognized immunohistochemical quantitative scoring index based on the classification results. The intelligent report generation module, connected to the quantitative analysis module, is used to generate an immunohistochemical staining result analysis report containing standardized diagnostic descriptions based on the immunohistochemical quantitative scoring indicators, the classification results of each segmented cell, and the image corresponding to the cell.

[0017] In this embodiment, the digital slide scanning module converts IHC-stained slides into full-field digital images (WSI) using a fully automated digital slide scanner. The scanner possesses high-resolution, high-speed scanning capabilities, clearly capturing the subtle features of cells and tissues on the slide. The scanning process is fast and stable, effectively reducing image distortion and noise, providing high-quality basic data for subsequent analysis.

[0018] The image preprocessing and region recognition module distinguishes between tumor regions, stroma regions, necrotic regions, and normal tissue regions by combining traditional image processing algorithms with convolutional neural networks.

[0019] Specifically, the image preprocessing and region recognition module performs color normalization on the full-view digital image and identifies and segments the regions of interest rich in tumor cells based on a deep learning model, as shown below: (1) Color normalization: In digital pathological image analysis, in order to eliminate the color inconsistency caused by the difference in staining conditions and scanning equipment of different slices, a standard staining template (such as the Macenko standardization algorithm) is used to calibrate the color distribution of all images to a unified standard color space, thereby significantly reducing the systematic error introduced by staining variation and improving the reliability and comparability of subsequent analysis.

[0020] (2) Tissue region identification: The entire pathological image is semantically segmented by image processing techniques (such as the automatic thresholding method based on Otsu) or by using a lightweight convolutional neural network (CNN) to accurately distinguish biologically significant tissue regions and effectively remove blank backgrounds, artifacts and other non-tissue parts, ensuring that subsequent processing is only carried out on image regions with analytical value.

[0021] (3) Region of Interest (ROI) identification: Using pre-trained deep classification networks (such as ResNet) or semantic segmentation models (such as U-Net), the system automatically detects and locates key regions in pathological images, especially tumor cell-rich areas. This step can accurately identify and delineate the boundaries of tumor regions and further distinguish different tissue component types, such as tumor cell areas, stroma areas, and necrotic areas, providing structured regional input for subsequent quantitative analysis and pathological evaluation.

[0022] The cell segmentation and classification module uses a deep learning model based on U-Net, Mask R-CNN or similar structure for instance segmentation. This network is trained using a large amount of IHC image data annotated by pathology experts.

[0023] Specifically, the cell segmentation and classification module adopts an instance segmentation model based on Mask R-CNN, which can simultaneously perform cell localization (bounding boxes), segmentation (pixel-level masking), and classification. The instance segmentation model classifies each segmented cell according to its staining intensity (based on the average optical density or color features of the DAB staining region) into: level 0 (negative), level 1+ (weakly positive), level 2+ (moderately positive), and level 3+ (strongly positive).

[0024] The quantitative analysis module calculates quantitative scoring indicators including, but not limited to: percentage of positive cells, staining intensity score, H-Score, Allred Score, and immunohistochemical score.

[0025] Specifically, upon receiving cell classification results, the system automatically calculates the following core indicators: (1) Percentage of positive cells (%) = (number of positive cells / total number of cells) × 100%.

[0026] (2) H-Score=∑(P i ×i), where i is the intensity level (1-3), P i This represents the percentage of cells corresponding to the intensity level. Values ​​range from 0 to 300.

[0027] (3) Allred Score: The total score is 0-8, which combines the percentage of positive cells (0-5) and the average staining intensity score (0-3).

[0028] (4) Immunohistochemical score (IHCScore): The algorithm can be customized according to the clinical guidelines for specific antibodies (such as PD-L1).

[0029] The intelligent report generation module is used to populate the analysis results into a preset report template. The report content automatically includes: a thumbnail of the analysis area, a pseudo-color overlay of positive / negative cells, statistical pie / bar charts of various cell types, calculated quantitative scores, and a standardized analysis description automatically generated based on the score range (e.g., "HER2 IHC score is 2+, further FISH validation is recommended").

[0030] Specifically, the report generated by the intelligent report generation module is a standardized and structured document, which includes basic patient information, detected antibodies, a schematic diagram of the analysis area, statistical charts of cells at each positive level, a final quantitative score, and textual conclusions based on preset rules.

[0031] In addition, such as Figure 2 As shown, the system also includes a result review and interactive editing module, which connects the cell segmentation and classification module and the quantitative analysis module. This module provides a visual interface for users to correct cell classification results, allowing pathologists to visually review and manually correct the results of the system's automatic segmentation and classification, and then feed the corrected results back to the quantitative analysis module for recalculation.

[0032] The result review and interactive editing module has an active learning function, which can use data manually corrected by pathologists as new training samples for incremental learning and performance optimization of the deep learning model.

[0033] Specifically, the results review and interactive editing module provides a graphical interface, allowing pathologists to easily view the AI's cell segmentation and classification results. This enables doctors to reclassify misclassified cells or adjust the analysis area. The system treats the doctor's corrections as the "gold standard" and can use this corrective data to fine-tune the original model, achieving continuous self-optimization (active learning).

[0034] This application also provides an embodiment, taking HER2 immunohistochemical staining analysis in breast cancer as an example. The specific execution process of the system is as follows: 1) Scanning: Convert the HER2 IHC stained slides into WSI files using a scanner.

[0035] 2) Preprocessing and Region Identification: The system performs color normalization on the WSI and then uses a trained tissue segmentation model to identify invasive cancer regions as ROIs, avoiding ductal carcinoma in situ and normal breast tissue.

[0036] 3) Cell Segmentation and Classification: Within the Region of Interest (ROI), the Mask R-CNN model accurately segments each cell with an intact cell membrane. Based on the intensity of the dark brown (DAB) staining of the cell membrane, the model classifies the cells: (1) Unstained or weakly indistinct membrane staining: Grade 0 (negative).

[0037] (2) Some cell membranes have weak, discontinuous staining: grade 1+ (weak positive).

[0038] (3) The cell membrane has moderate and continuous staining: grade 2+ (moderate positive).

[0039] (4) The cell membrane has strong and intact linear staining: grade 3+ (strong positive).

[0040] 4) Quantitative analysis: The system statistically showed that among the 2000 cells analyzed, grade 3+ cells accounted for 65%, grade 2+ cells for 10%, grade 1+ cells for 5%, and negative cells for 20%. Calculations yielded: (1) Percentage of positive cells = (65% + 10% + 5%) = 80% (2) H-Score=(65×3)+(10×2)+(5×1)=195+20+5=220 (3) According to the HER2 interpretation criteria (>10% of cells showing strong and intact membrane staining is 3+), this case was interpreted as HER2 positive (3+).

[0041] 5) Report generation: The system automatically generates a report, which includes the patient ID, the detected antibody (HER2), the analysis area map, the cell classification statistics map, and clearly states the conclusion "HER2 protein expression: immunohistochemical score is 3+ (positive)".

[0042] 6) Physician Review: Pathologists quickly review the evidence images and analysis results provided by the system on the review interface. After confirming that everything is correct, they issue the report. If a physician finds any errors in the classification of individual areas, they can manually correct them. The corrected data will be recorded by the system for model updates.

[0043] In summary, this application has the following technical effects: 1) Improved objectivity and accuracy: By using deep learning models to replace the human eye for cell-level segmentation and intensity classification, the interference of subjective factors is greatly reduced, and the results are more accurate and reproducible.

[0044] 2) Improve work efficiency: The fully automated analysis process frees pathologists from the heavy work of microscopic examination and counting, and the analysis speed is increased by several times or even dozens of times.

[0045] 3) Achieve precise quantification: Provides a variety of internationally recognized quantitative scores, offering richer and more reliable data support for precision medicine and clinical research.

[0046] 4) Promote report standardization: Automatically generated unified format reports facilitate communication and comparison between different hospitals and laboratories, and also facilitate data entry and subsequent statistical analysis.

[0047] 5) Human-machine collaboration and continuous evolution: The built-in review and interactive editing functions ensure that doctors have control over the final results, while the active learning mechanism enables the system to continuously adapt to new data and become smarter with use.

[0048] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0049] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. In summary, the content of this specification should not be construed as a limitation of this application.

Claims

1. A deep learning-based intelligent analysis and report generation system for immunohistochemical staining results, characterized in that, include: The digital slide scanning module is used to convert immunohistochemical stained slides into high-resolution full-view digital images; The image preprocessing and region recognition module is connected to the digital slice scanning module and is used to perform color normalization processing on the full-view digital image and identify and segment the region of interest enriched with tumor cells based on a deep learning model. The cell segmentation and classification module, connected to the image preprocessing and region recognition module, is used to perform pixel-level segmentation of each cell in the region of interest using a trained instance segmentation model, and automatically classify each segmented cell according to the staining intensity to obtain the classification result. The quantitative analysis module, connected to the cell segmentation and classification module, is used to automatically calculate at least one internationally recognized immunohistochemical quantitative scoring index based on the classification results. The intelligent report generation module, connected to the quantitative analysis module, is used to generate an immunohistochemical staining result analysis report containing standardized diagnostic descriptions based on the immunohistochemical quantitative scoring indicators, the classification results of each segmented cell, and the image corresponding to the cell.

2. The intelligent analysis and report generation system for immunohistochemical staining results based on deep learning according to claim 1, characterized in that, The image preprocessing and region recognition module is used to distinguish tumor regions, stroma regions, necrotic regions and normal tissue regions using traditional image processing algorithms and convolutional neural networks.

3. The intelligent analysis and report generation system for immunohistochemical staining results based on deep learning according to claim 1, characterized in that, The cell segmentation and classification module uses an instance segmentation model with either a U-Net structure or a Mask R-CNN structure; the instance segmentation model is trained using immunohistochemical image data annotated by pathology experts.

4. The intelligent analysis and report generation system for immunohistochemical staining results based on deep learning according to claim 1, characterized in that, The quantitative analysis module calculates immunohistochemical quantitative scoring indicators including the percentage of positive cells, staining intensity score, H-Score, Allred Score, and immunohistochemical score.

5. The intelligent analysis and report generation system for immunohistochemical staining results based on deep learning according to claim 1, characterized in that, Also includes: The results review and interactive editing module connects the cell segmentation and classification module and the quantitative analysis module. It provides a visual interface for users to correct cell classification results and feeds back the corrected cell classification results to the quantitative analysis module to recalculate the immunohistochemical quantitative scoring index.

6. The intelligent analysis and report generation system for immunohistochemical staining results based on deep learning according to claim 5, characterized in that, The result review and interactive editing module is also used to incrementally optimize the instance segmentation model by using the data corrected by the user as new training samples.

7. The intelligent analysis and report generation system for immunohistochemical staining results based on deep learning according to claim 1, characterized in that, The report generated by the intelligent report generation module includes an analysis area diagram, statistical charts of cells at each positive level, a final quantitative score, and text conclusions automatically generated according to preset rules.

8. The intelligent analysis and report generation system for immunohistochemical staining results based on deep learning according to claim 1, characterized in that, The image preprocessing and region recognition module uses a color calibration method based on a standard staining template when performing color normalization.

9. The intelligent analysis and report generation system for immunohistochemical staining results based on deep learning according to claim 1, characterized in that, The quantitative analysis module also allows users to customize quantitative scoring algorithms based on clinical guidelines.

10. The intelligent analysis and report generation system for immunohistochemical staining results based on deep learning according to claim 1, characterized in that, The classification results are negative, weakly positive, moderately positive, or strongly positive.

Images