A method and system for obtaining a combined positive score of PD-L1 expression based on deep learning

Through deep learning model and image preprocessing technology, the identification of tumor cells and immune cells expressed by PD-L1 is solved, and the problem of insufficient identification accuracy in the prior art is achieved, achieving more efficient calculation and detection consistency of combined positive scores.

CN118657755BActive Publication Date: 2024-11-22TIANJIN TUMOR HOSPITAL
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202411088776.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-09
Publication Date
2024-11-22
Estimated Expiration
2044-08-09

AI Technical Summary

Technical Problem

The prior art recognizes tumor cells and immune cells expressed by PD-L1, and the recognition is not high, resulting in insufficient accuracy of the calculation of joint positive scores. There are differences in detection methods between different companies and cloning numbers, which affects the consistency of interpretation and Repeatability.

Method used

Using a deep learning-based method, tumor cells and immune cells are identified through convolutional neural network models, different image preprocessing methods (such as filtering and image contrast enhancement) improve the recognition accuracy, and segment large images into small images for recognition , improve the recognition efficiency and accuracy of the model.

Benefits of technology

It improves the recognition accuracy of tumor cells and immune cells, enhances the accuracy of the calculation of combined positive scores, reduces the difference in interpretation, and improves the efficiency and repeatability of detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118657755B_ABST
    Figure CN118657755B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of medical image processing and recognition technology, and in particular to a method and system for obtaining a joint positive score of PD-L1 expression based on deep learning; when calculating the joint positive score of PD-L1 expression, the present invention adopts a tumor cell recognition model to identify tumor cells, and adopts an immune cell recognition model to identify immune cells, and then adopts different image preprocessing methods according to the recognition characteristics of the corresponding cells when adopting the corresponding models to identify the corresponding cells; different models and different image preprocessing methods improve the accuracy of tumor cell and immune cell recognition; at the same time, since the pixels of digitized slice images are large and contain more information, the digitized slice images are divided into small images with a fixed pixel size, and then the small images are respectively input into the model, which can effectively improve the accuracy of cell recognition compared with directly inputting the digitized slice images into the model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing and recognition, and in particular to a method and system for obtaining a combined positive score of PD-L1 expression based on deep learning. Background Art

[0002] Studies have shown that the expression level of PD-L1 in tumor tissue is closely related to the efficacy of ICI treatment of HNSCC and is currently the most practical and effective predictive marker. The combined positive score (CPS) of PD-L1 expression is a key indicator for screening ICI immunotherapy indications in patients with advanced HNSCC and has been included in the routine clinical pathology testing items recommended by the NCCN guidelines.

[0003] Standardized testing of PD-L1 has been gradually carried out in many pathology departments at home and abroad. The staining results are evaluated by semi-quantitative interpretation of the combined positive score CPS by pathologists to guide the screening of HNSCC patients who benefit from immunotherapy. Since PD-L1 can be expressed in tumor cells and a variety of immune cells, and the positive expression pattern is complex, the interpretation of PD-L1 staining results is very challenging. Multiple practical studies have shown that the consistency and repeatability of clinical pathologists' interpretation of PD-L1 are insufficient, and the interpretation is time-consuming and laborious. At present, the PD-L1 antibodies used in ICI drugs of various companies are not consistent, and the PD-L1 staining patterns of different clones are also slightly different, resulting in differences in PD-L1 detection methods and interpretation standards.

[0004] There are schemes in the prior art that use deep learning models to identify positive tumor cells and positive immune cells and then calculate the combined positive score. However, when identifying related cells, the prior art inputs the digital slice image into a model, and the model first obtains the cell morphology and then classifies the cells to output the number of tumor cells and immune cells; it mainly identifies membrane-stained positive tumor cell nuclei and positive tumor-related immune cells by morphology and color. The recognition of the two is relatively low, and during the training process, since it is necessary to take into account the classification and recognition of different cells, the recognition accuracy of each type of cell will also be affected. Summary of the invention

[0005] In view of this, the present invention proposes a method and system for obtaining a combined positive score of PD-L1 expression based on deep learning, which is generally applicable to the accuracy of obtaining a combined positive score.

[0006] In order to achieve the above object, a method for obtaining a combined positive score of PD-L1 expression based on deep learning is provided, and the method comprises the following steps:

[0007] S1: Acquire digitized slice images for joint positive score calculation;

[0008] S2: performing a first image preprocessing operation on the digitized slice image to obtain a digitized slice image thumbnail;

[0009] S3: performing a second image preprocessing operation on the digital slice image thumbnail;

[0010] S4: Input the digital slice image preprocessed in S3 into the tumor cell recognition model to obtain the number of PD-L1 membrane-stained positive tumor cells in the digital slice image (T TC+ ) and the total number of tumor cells (T TC ); S4 is specifically:

[0011] S4.1: Acquire an image set for building a tumor cell recognition model;

[0012] S4.2: using the image set to train a convolutional neural network model, thereby obtaining a tumor cell recognition model;

[0013] S4.3: respectively inputting the digital slice image thumbnails preprocessed by S3 into the tumor cell recognition model obtained by S4.2 to identify PD-L1 membrane-positive tumor cells and PD-L1 membrane-negative tumor cells;

[0014] S4.4: Count the PD-L1 membrane-positive tumor cells and the PD-L1 membrane-negative tumor cells to obtain the number T of PD-L1 membrane-positive tumor cells in the digital slice image. TC+ The total number of tumor cells T TC ;

[0015] S5: performing a third image preprocessing operation on the digital slice image thumbnail;

[0016] S6: respectively inputting the small pictures of the digitized slice images preprocessed by S5 into the immune cell recognition model to obtain the number of PD-L1 positive immune cells in the digitized slice images;

[0017] S7: Obtaining the joint positive score of the digitized slice image.

[0018] Preferably, the step S1 specifically includes: performing a full-slice scan on the sample using a digital slice scanning device, thereby obtaining a digital slice image.

[0019] Preferably, in S2, the first image preprocessing operation is specifically: performing a fixed-step image segmentation operation on the digitized slice image to segment the digitized slice image into a plurality of 256×256×3 pixel digitized slice image thumbnails.

[0020] Preferably, the second image preprocessing operation is: performing a filtering operation on the digitized slice image thumbnail; and the third image preprocessing operation is specifically: performing an image contrast enhancement operation on the digitized slice image thumbnail.

[0021] Preferably, the tumor cell recognition model is a convolutional neural network model.

[0022] Preferably, the total number of tumor cells (T TC ) is the number of tumor cells with positive PD-L1 membrane staining (T TC+ ) and the number of PD-L1 membrane staining negative tumor cells (T TC- ) and.

[0023] Preferably, the immune cell recognition model is a convolutional neural network model.

[0024] Preferably, the S6 is specifically:

[0025] S6.1: Acquire an image set for constructing an immune cell recognition model;

[0026] S6.2: using the image set to train a convolutional neural network model, thereby obtaining an immune cell recognition model;

[0027] S6.3: Inputting the digital slice image thumbnails preprocessed in S5 into the immune cell recognition model obtained in S6.2 to identify PD-L1 positive immune cells;

[0028] S6.4: Count the PD-L1 positive immune cells to obtain the number of PD-L1 positive immune cells in the digital slice image (T IC+ ).

[0029] According to another aspect of the present invention, a system for acquiring a combined positive score of PD-L1 expression based on deep learning is provided. The system adopts the above-mentioned method for acquiring a combined positive score of PD-L1 expression based on deep learning, and the system comprises:

[0030] A digital slice image acquisition module, used for acquiring a digital slice image for joint positive score calculation;

[0031] A segmentation module, used for performing a first image preprocessing operation on the digitized slice image to obtain a digitized slice image thumbnail;

[0032] A second image preprocessing module, used for performing a second image preprocessing operation on the digital slice image thumbnail;

[0033] The tumor cell recognition model is used to input the digital slice image thumbnails that have been preprocessed by the second image into the tumor cell recognition model to obtain the number T of tumor cells with positive PD-L1 membrane staining in the digital slice image. TC+ The total number of tumor cells T TC ;

[0034] A third image preprocessing module, used for performing a third image preprocessing operation on the digital slice image thumbnail;

[0035] An immune cell recognition model, used to input the digital slice image sub-images that have been preprocessed by the third image into the immune cell recognition model respectively, to obtain the number of PD-L1 positive immune cells in the digital slice image;

[0036] The combined positive score calculation module is used to calculate the combined positive score of the digital slice image. The advantages and beneficial effects of the present invention are:

[0037] When calculating the combined positive score of PD-L1 expression, the present invention uses a tumor cell recognition model to identify tumor cells, uses an immune cell recognition model to identify immune cells, and then uses the corresponding model to identify corresponding cells. Different image preprocessing methods are used according to the recognition characteristics of the corresponding cells. When identifying tumor cells, a filtering operation is used to remove image noise. When identifying immune cells, an image enhancement operation is used to improve image contrast. At the same time, the above different models and different image preprocessing methods improve the accuracy of tumor cell and immune cell recognition.

[0038] At the same time, since the pixels of digital slice images are large and contain more information, the digital slice images are divided into small images with a fixed pixel size, and then the small images are input into the model separately. Compared with directly inputting the digital slice images into the model, this can effectively improve the accuracy of cell recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the description of the present invention or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 A schematic diagram of the combination of digitization of tissue pathology sections and deep learning in the prior art;

[0041] Figure 2 A flowchart of a method for obtaining a combined positive score of PD-L1 expression based on deep learning provided in an embodiment of the present invention;

[0042] Figure 3 A flowchart of inputting a digital slice image thumbnail preprocessed by S3 into a tumor cell recognition model to obtain the number of PD-L1 membrane-stained positive tumor cells and the total number of tumor cells in the digital slice image provided by an embodiment of the present invention;

[0043] Figure 4 The present invention provides a flowchart of inputting a digitized slice image thumbnail preprocessed by S5 into an immune cell recognition model to obtain the number of PD-L1 positive immune cells in the digitized slice image. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] With the application of digital pathology (DP) in clinical pathological diagnosis, digital slides and computer-assisted analysis tools are being incorporated into the scope of pathology. The digitization of tissue pathology slides not only makes it easier to obtain and use pathological images, but more importantly, it provides a high-resolution image that contains a large amount of pixel-level digital information, providing a computational basis for quantitative analysis of the morphological characteristics of tumor tissue. Digital technology is driving changes in the data collection and observation mode of pathological images, prompting the transformation of traditional tissue pathology to computational pathology of quantitative analysis.

[0046] Machine Learning (ML) is an interdisciplinary subject that mainly uses computer tools and mathematical models to solve various learning tasks. It has developed rapidly in recent years and has gradually been applied to the fields of medical big data and medical image analysis. Deep Learning (DL) is an important branch of machine learning. Compared with other machine learning algorithms, deep learning has stronger feature extraction capabilities for big data samples. The concept of deep learning originated from the study of artificial neural networks, which combines low-level features to form more abstract high-level representations of attribute categories or features. In the field of computer vision, the architecture based on Convolutional Neural Networks (CNN) is one of the most commonly used network structures in deep learning image recognition. Convolutional neural networks contain convolutional calculations and are feedforward neural networks with deep structures. The main structures are: convolutional layers, pooling layers, and fully connected layers. By stacking these layer structures, a deep convolutional neural network (DCNN) is formed, which has strong representation learning capabilities. Deep learning algorithms for target detection, image classification, and image semantic segmentation are widely used in various medical image recognition tasks.

[0047] The deep learning algorithm based on convolution operation is very suitable for feature extraction and analysis of digital pathology sections, and can organically integrate digital pathology images with computer technology. At present, deep learning has achieved good applications in pathological classification diagnosis, tumor cell detection, tumor mutation and prognosis prediction, greatly assisting pathologists in diagnosis and treatment and scientific research. The application research of deep learning in pathology has promoted the combination of computer tools and digital pathology, and brought new opportunities for the development of traditional tissue pathology.

[0048] Artificial Intelligence (AI) plays an increasingly important role in pathological image analysis. With the assistance of AI, the accuracy of pathological diagnosis can be significantly improved through the "human-machine combination" diagnostic model. AI can obtain precise vector parameters from histological image data and provide more biologically relevant information than the semi-quantitative scoring currently used in clinical pathology practice. In the context of digitalization of pathological sections, AI technology promotes the gradual transformation of pathological diagnosis from qualitative analysis to quantitative analysis, and this transformation will make pathological analysis more accurate and objective.

[0049] Advances in tumor immunotherapy have driven an increase in demand for predictive testing for tumor diagnosis to screen patients and guide treatment decisions. Standardized biomarker evaluation protocols are critical to accelerating the clinical development of tumor immunotherapy. Developing specialized computer analysis models in parallel with clinical immunotherapy may help coordinate solutions to the problem of differences in interpretation criteria for companion diagnostics for immunotherapy. AI-based image analysis tools can perform complex and quantitative analysis of pathological markers with high reproducibility. Therefore, AI image analysis results are easier to share and standardize in different diagnostic laboratories, thereby assisting pathologists in the development of standards for the detection and interpretation of pathological indicators.

[0050] At the same time, research at the tissue pathology level is increasingly important for the discovery of biomarkers and clinical treatment stratification. More and more molecular indicators are used to interpret immunohistochemical staining results to guide clinical treatment decision-making. In the context of precision medicine, pathologists are facing a substantial increase in diagnostic workload and complexity, and diagnostic plans must also focus on efficiency and accuracy. The development of artificial intelligence can promote the automation of such quantitative interpretation work in tissue pathology, which is both a challenge and an opportunity for current pathology research. It can significantly improve efficiency and save costs, and improve the current shortage of pathologists in China.

[0051] Histopathological images are highly integrated. The panoramic features of the immune microenvironment of tumor tissue can be captured by computer imaging technology. Based on the specific data of digital image parameters, the computer can qualitatively and quantitatively calculate the expression values ​​of tumor immunotherapy markers and the main characteristics of the tissue immune microenvironment. Machine learning methods can integrate and analyze histopathological image data and use them for model construction. Using computer technology based on machine learning to accurately and intelligently analyze digital pathological information of predictive indicators of immunotherapy efficacy and prognosis is a powerful method to form a tumor "cancer immune map".

[0052] The deep learning algorithm based on convolution operation is very suitable for feature extraction and analysis of digital pathology slices, and can organically integrate digital pathology images with computer technology. Figure 1 A schematic diagram showing the combination of digitization of tissue pathology sections and deep learning. At present, deep learning has achieved good applications in pathological classification diagnosis, tumor cell detection, tumor mutation and prognosis prediction, greatly assisting pathologists in diagnosis and treatment and scientific research. The application research of deep learning in pathology has promoted the combination of computer tools and digital pathology, and brought new opportunities for the development of traditional tissue pathology.

[0053] Embodiment 1, as attached Figure 2 As shown, a method for obtaining a combined positive score of PD-L1 expression based on deep learning comprises the following steps:

[0054] S1: Acquire digitized slice images for joint positive score calculation;

[0055] Specifically, the sample was scanned in its entirety using a digital slice scanning device (model: 3DHISTECH Pannoramic 250) to obtain a digital slice image;

[0056] S2: performing a first image preprocessing operation on the digitized slice image to obtain a digitized slice image thumbnail;

[0057] The first image preprocessing operation specifically includes: performing a fixed-step image segmentation operation on the digitized slice image to segment the digitized slice image into a plurality of 256×256×3 pixel digitized slice image thumbnails;

[0058] In this embodiment, since it is necessary to count the membrane-stained positive tumor cells, nuclear-positive tumor-associated immune cells and the total number of tumor cells in the image and then calculate the PD-L1 combined positive score when calculating the combined positive score in the digitized slice image, the size of a digitized slice RGB image is about 20000×20000×3 pixels. Such a large number of pixels contain a lot of information, and the model recognition accuracy is not high. Therefore, the digitized slice image is divided into small images with a fixed pixel size, and then the small images are input into the model respectively. Compared with directly inputting the digitized slice image into the model, the accuracy of cell recognition can be significantly improved.

[0059] S3: performing a second image preprocessing operation on the digital slice image thumbnail;

[0060] Among them, in this step, after the image preprocessing in this step, the tumor cell identification step will be entered, and the tumor cells are greatly affected by image noise. In order to improve the accuracy of tumor cell identification, the second image preprocessing operation is: filtering the digital slice image thumbnail;

[0061] S4: Input the digital slice image small pictures preprocessed by S3 into the tumor cell recognition model respectively to obtain the number of PD-L1 membrane-stained positive tumor cells (T TC+ ) and the total number of tumor cells (T TC );

[0062] Wherein, the tumor cell recognition model is a convolutional neural network model;

[0063] The small images divided into a complete digital slice image were input into the tumor cell recognition model, and then the recognition results of the model were statistically analyzed to obtain the number of PD-L1 membrane-stained positive tumor cells (T TC+ ) and the total number of tumor cells (TTC );

[0064] It is worth noting that, in this embodiment, the focus is on training the tumor cell recognition model to improve the accuracy of related cell recognition;

[0065] Specifically, Figure 3 As shown, the S4 is specifically:

[0066] S4.1: Acquire an image set for building a tumor cell recognition model;

[0067] Among them, the surgical tissue samples and archival PD-L1 staining sections of HNSCC in the past five years were collected from Tianjin Medical University Cancer Hospital, the supporting unit of the project. It is expected that 500 HNSCC cases will be collected as the first batch of images for building a tumor cell recognition model; then the sample set will be segmented. Each PD-L1 stained digital pathology RGB image is about 20000×20000×3 pixels in size and can be segmented into about 6000 training input images of 256×256×3 pixels. Therefore, 500 PD-L1 digital slice images can be segmented into about 3 million image data sets for model construction;

[0068] Finally, the image dataset was manually annotated. The digital slice images stained with the PD-L1 standard antibody 22c3 were manually annotated using image annotation software. No less than 10 1×1 mm 2 Representative areas were manually labeled by professional pathologists. Personnel training and consistency analysis of training sample annotations were conducted before labeling. After labeling, HNSCC pathology experts conducted verification and used the labeled tissue section images as the image set for constructing the tumor cell recognition model.

[0069] It is worth emphasizing that in this step, the manual annotation specifically includes: annotating the PD-L1 membrane-stained positive tumor cells and the PD-L1 membrane-stained negative tumor cells in the digital slice image;

[0070] Wherein, the image set includes a PD-L1 membrane-stained positive tumor cell image set and a PD-L1 membrane-stained negative tumor cell image set;

[0071] S4.2: using the image set to train a convolutional neural network model to obtain a tumor cell recognition model, wherein the tumor cell recognition model includes a positive tumor cell recognition model and a negative tumor cell recognition model;

[0072] In S4.2, the PD-L1 membrane-stained positive tumor cell image set is used to train the positive tumor cell recognition model, and the PD-L1 membrane-stained negative tumor cell image set is used to train the negative tumor cell recognition model;

[0073] Specifically, whether the tumor cell recognition model is trained is determined by accuracy, sensitivity, specificity, F1 score, and IoU.

[0074] S4.3: respectively inputting the digital slice image thumbnails preprocessed by S3 into the positive tumor cell recognition model and the negative tumor cell recognition model obtained by S4.2 to identify PD-L1 membrane-stained positive tumor cells and PD-L1 membrane-stained negative tumor cells;

[0075] Among them, PD-L1 membrane staining-positive tumor cells and PD-L1 membrane staining-negative tumor cells are marked with different colors;

[0076] In this embodiment, when identifying positive tumor cells and negative tumor cells, the same digitized slice thumbnail is input into the positive tumor cell recognition model and the negative tumor cell recognition model. The tumor cell recognition model only recognizes one type of tumor cells when recognizing the image. That is, the positive tumor cell recognition model recognizes the positive tumor cells and then identifies the positive tumor cells; then the same digitized slice thumbnail is input into the negative tumor cell recognition model to identify the negative tumor cells. In this way, the accuracy of recognition can be effectively improved.

[0077] S4.4: Count the PD-L1 membrane-stained positive tumor cells and the PD-L1 membrane-stained negative tumor cells to obtain the number of PD-L1 membrane-stained positive tumor cells in the digital slice image (T TC+ ) and the total number of tumor cells (T TC );

[0078] The total number of tumor cells (T TC ) is the number of tumor cells with positive PD-L1 membrane staining (T TC+ ) and the number of PD-L1 membrane staining negative tumor cells (T TC- );

[0079] In fact, there are solutions in the prior art that use deep learning models to identify positive tumor cells and positive immune cells and then calculate the combined positive score. However, when identifying related cells, the prior art inputs the digital slice image into a model, and the model first obtains the cell morphology and then classifies the cells to output the number of tumor cells and immune cells; it mainly recognizes membrane-stained positive tumor cells and positive tumor-related immune cells by morphology and color. The recognition of the two is relatively low, and during the training process, since it is necessary to take into account the classification and recognition of different cells, the recognition accuracy of each type of cell will also be affected.

[0080] Therefore, this embodiment uses a tumor cell recognition model to identify positive and negative tumor cells, and then the model outputs the number of tumor cells and PD-L1 membrane-stained positive tumor cells, thereby improving the accuracy of tumor cell recognition.

[0081] S5: performing a third image preprocessing operation on the digital slice image thumbnail;

[0082] The purpose of the third image preprocessing operation is to improve the recognition accuracy of immune cells in the digitized slice image through the image preprocessing operation, and the recognition of immune cells in the digitized slice image is mainly determined by the coloring of the stained slice. Therefore, the third image preprocessing operation is specifically: performing an image contrast enhancement operation on the digitized slice image thumbnail;

[0083] Specifically, the gamma transform method is used to perform image contrast enhancement operation;

[0084] Wherein, the formula of the gamma transform is:

[0085]

[0086] Among them, S is the pixel value after gamma transformation, c is a constant, and r is the pixel value before gamma transformation. is the gamma value, which is used to adjust the contrast of the transformed image;

[0087] The inventors found in their research that, for digital slice images, the higher the contrast, the higher the accuracy of the digital slice images in identifying cells using the artificial intelligence model. Through analysis, it was found that this may be related to the thickness of the digital slices and the number of various types of cells.

[0088] Moreover, the inventors have found through research that there is a correlation in the number trend between PD-L1 positive immune cells and PD-L1 membrane-stained positive tumor cells. For more PD-L1 positive immune cells, a higher contrast can be appropriately set, and for fewer PD-L1 positive immune cells, a lower contrast can be appropriately set to improve the accuracy of recognition; therefore, this step dynamically determines the size of the gamma value in the gamma transform based on the PD-L1 membrane-stained positive tumor cells obtained by S, thereby improving the accuracy of the model in recognizing different digitized slice images.

[0089] Specifically, S5 is specifically: adjusting the size of the gamma value in the gamma transform according to the number of the PD-L1 membrane-stained positive tumor cells obtained in S4; wherein, when the number of the PD-L1 membrane-stained positive tumor cells is greater than a preset threshold, the gamma value is 0.7, and when the number of the PD-L1 membrane-stained positive tumor cells is less than or equal to the preset threshold, the gamma value is 0.5;

[0090] Through this step, different image contrast operations can be used for different digital slice images, laying a good data foundation for the subsequent recognition of artificial intelligence models.

[0091] S6: respectively inputting the small pictures of the digitized slice images preprocessed by S5 into the immune cell recognition model to obtain the number of PD-L1 positive immune cells in the digitized slice images;

[0092] Wherein, the immune cell recognition model is also a convolutional neural network model;

[0093] Specifically, Figure 4 As shown, the S6 is specifically:

[0094] S6.1: Acquire an image set for constructing an immune cell recognition model;

[0095] In this step, the collection process of the image set constructed by the immune cell recognition model is the same as that of the image set constructed by the tumor cell recognition model. The difference lies in the manual annotation of the image set.

[0096] It is worth emphasizing that in this step, the manual annotation specifically includes: annotating the PD-L1 positive immune cells in the digital slice image;

[0097] S6.2: using the image set to train a convolutional neural network model, thereby obtaining an immune cell recognition model;

[0098] Similarly, whether the tumor cell recognition model is trained is determined by accuracy, sensitivity, specificity, F1 score, and IoU.

[0099] S6.3: inputting the digital slice image preprocessed in S5 into the immune cell recognition model obtained in S6.2 to identify PD-L1 positive immune cells;

[0100] S6.4: Count the PD-L1 positive immune cells to obtain the number of PD-L1 positive immune cells in the digital slice image (T IC+ );

[0101] S7: obtaining a joint positive score of the digital slice image;

[0102] The calculation process of the combined positive score (CPS) of the digital slice image is as follows:

[0103] ;

[0104] Embodiment 2: This embodiment includes a system for acquiring a combined positive score of PD-L1 expression based on deep learning. The system adopts the method for acquiring a combined positive score of PD-L1 expression based on deep learning of Embodiment 1. The system includes:

[0105] A digital slice image acquisition module, used for acquiring a digital slice image for joint positive score calculation;

[0106] A segmentation module, used for performing a first image preprocessing operation on the digitized slice image to obtain a digitized slice image thumbnail;

[0107] A second image preprocessing module, used for performing a second image preprocessing operation on the digital slice image thumbnail;

[0108] The tumor cell recognition model is used to input the digital slice image thumbnails that have been preprocessed by the second image into the tumor cell recognition model to obtain the number T of tumor cells with positive PD-L1 membrane staining in the digital slice image. TC+ The total number of tumor cells T TC ;

[0109] A third image preprocessing module, used for performing a third image preprocessing operation on the digital slice image thumbnail;

[0110] An immune cell recognition model, used to input the digital slice image sub-images that have been preprocessed by the third image into the immune cell recognition model respectively, to obtain the number of PD-L1 positive immune cells in the digital slice image;

[0111] The joint positive score calculation module is used to calculate the joint positive score of the digital slice image.

[0112] Embodiment 3, this embodiment includes a computer-readable storage medium, on which a data processing program is stored, and the data processing program is executed by a processor to implement the method for obtaining a combined positive score of PD-L1 expression based on deep learning in embodiment 1.

[0113] It should be understood by those skilled in the art that the embodiments of this article can be provided as methods, devices (equipment), or computer program products. Therefore, this article can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Including but not limited to RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassette, tape, disk storage or other magnetic storage device, or any other medium that can be used to store desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media generally contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0114] This document is described with reference to flowcharts and / or block diagrams of methods, apparatuses (devices), and computer program products according to the embodiments of this document. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0115] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0116] The embodiments and / or implementation methods described above are only used to illustrate the preferred embodiments and / or implementation methods of the technology of the present invention, and are not intended to impose any formal restrictions on the implementation methods of the technology of the present invention. Any person skilled in the art may make some changes or modifications to other equivalent embodiments without departing from the scope of the technical means disclosed in the content of the present invention, but they should still be regarded as the technology or embodiments that are essentially the same as the present invention. Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described here, and that various obvious changes, readjustments and substitutions can be made to those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention is described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may also include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for obtaining a combined positive score of PD-L1 expression based on deep learning, characterized in that: The method comprises the following steps: S1: Acquire digitized slice images for joint positive score calculation; S2: performing a first image preprocessing operation on the digitized slice image to obtain a digital slice image thumbnail; in S2, the first image preprocessing operation specifically comprises: performing a fixed-step image segmentation operation on the digitized slice image to segment the digitized slice image into a plurality of 256×256×3 pixel digital slice image thumbnails; S3: performing a second image preprocessing operation on the digital slice image thumbnail; S4: Input the digital slice image small pictures preprocessed by S3 into the tumor cell recognition model respectively to obtain the number T of PD-L1 membrane-stained positive tumor cells in the digital slice image. TC+ The total number of tumor cells T TC ; The S4 is specifically: S4.1: Acquire an image set for building a tumor cell recognition model; S4.2: using the image set to train a convolutional neural network model, thereby obtaining a tumor cell recognition model; S4.3: respectively inputting the digital slice image thumbnails preprocessed by S3 into the tumor cell recognition model obtained by S4.2 to identify PD-L1 membrane-positive tumor cells and PD-L1 membrane-negative tumor cells; S4.4: Count the PD-L1 membrane-positive tumor cells and the PD-L1 membrane-negative tumor cells to obtain the number T of PD-L1 membrane-positive tumor cells in the digital slice image. TC+ The total number of tumor cells T TC ; S5: performing a third image preprocessing operation on the digital slice image thumbnail; S6: respectively inputting the small pictures of the digitized slice images preprocessed by S5 into the immune cell recognition model to obtain the number of PD-L1 positive immune cells in the digitized slice images; S7: Obtaining a joint positive score of the digital slice image; The second image preprocessing operation is: performing a filtering operation on the digital slice image thumbnail; the third image preprocessing operation is specifically: performing an image contrast enhancement operation on the digital slice image thumbnail; performing the image contrast enhancement operation using a gamma transform method; Wherein, the formula of the gamma transform is: Among them, S is the pixel value after gamma transformation, c is a constant, and r is the pixel value before gamma transformation. is a gamma value used to adjust the contrast of the transformed image; the gamma value in the gamma transform is adjusted according to the number of the PD-L1 membrane-positive tumor cells obtained in S4; wherein, when the number of the PD-L1 membrane-positive tumor cells is greater than a preset threshold, the gamma value is 0.7, and when the number of the PD-L1 membrane-positive tumor cells is less than or equal to the preset threshold, the gamma value is 0.

5.

2. The method for obtaining a combined positive score of PD-L1 expression based on deep learning according to claim 1, characterized in that: The S1 specifically includes: performing a full-slice scan on the sample using a digital slice scanning device, thereby obtaining a digital slice image.

3. The method for obtaining a combined positive score of PD-L1 expression based on deep learning according to claim 1, characterized in that: The tumor cell recognition model is a convolutional neural network model.

4. The method for obtaining a combined positive score of PD-L1 expression based on deep learning according to claim 1, characterized in that: The total number of tumor cells T TC T is the number of tumor cells with positive PD-L1 membrane staining TC+ The number of tumor cells with negative PD-L1 membrane staining T TC- sum.

5. The method for obtaining a combined positive score of PD-L1 expression based on deep learning according to claim 1, characterized in that: The immune cell recognition model is a convolutional neural network model.

6. The method for obtaining a combined positive score of PD-L1 expression based on deep learning according to claim 5, characterized in that: The S6 is specifically: S6.1: Acquire an image set for constructing an immune cell recognition model; S6.2: using the image set to train a convolutional neural network model, thereby obtaining an immune cell recognition model; S6.3: Inputting the digital slice image thumbnails preprocessed in S5 into the immune cell recognition model obtained in S6.2 to identify PD-L1 positive immune cells; S6.4: Count the PD-L1 positive immune cells to obtain the number T of PD-L1 positive immune cells in the digital slice image IC+ .

7. A system for obtaining a combined positive score of PD-L1 expression based on deep learning, the system adopting the method for obtaining a combined positive score of PD-L1 expression based on deep learning according to any one of claims 1 to 6, characterized in that: The system comprises: A digital slice image acquisition module, used for acquiring a digital slice image for joint positive score calculation; A segmentation module, used for performing a first image preprocessing operation on the digitized slice image to obtain a digitized slice image thumbnail; A second image preprocessing module, used for performing a second image preprocessing operation on the digital slice image thumbnail; The tumor cell recognition model is used to input the digital slice image thumbnails that have been preprocessed by the second image into the tumor cell recognition model to obtain the number T of tumor cells with positive PD-L1 membrane staining in the digital slice image. TC+ The total number of tumor cells T TC ; A third image preprocessing module, used for performing a third image preprocessing operation on the digital slice image thumbnail; An immune cell recognition model, used to input the digital slice image sub-images that have been preprocessed by the third image into the immune cell recognition model respectively, to obtain the number of PD-L1 positive immune cells in the digital slice image; The joint positive score calculation module is used to calculate the joint positive score of the digital slice image.

Citation Information

Patent Citations

  • Diagnostic prognostic and therapeutic method for solid non-lymphoid tumors and their metastases

    CN1130944A

  • PD-L1 pathological section automatic interpretation method and system based on deep learning

    CN114235539A

  • Urine cell identification method and system based on multiple models

    CN114495097A

  • PD-L1 immunohistochemical scoring method, system and device and storage medium

    CN114581908A

  • Tumor cell statistical method, device, equipment and storage medium

    CN115908363A