Colorectal cancer ihc staining map immune grading method, system, and storage medium

Abstract

The application discloses a colorectal cancer IHC staining image tumor infiltration front immune grading method, system and storage medium, and the method comprises the following steps: acquiring a colorectal cancer IHC staining image; classifying the staining image by using a trained convolutional neural network classifier to obtain nine types of tissues, namely tumor epithelium, tumor interstitium, tumor necrosis, mucus, muscle, normal gland, lymph aggregation, fat and background; merging the nine types of tissue categories into five categories to automatically determine a tumor infiltration front and a tumor center area; performing cell segmentation and counting on the above-mentioned areas to obtain CD3 and CD8 positive cell densities, and then performing immune grading; the application quantifies the CD3 and CD8 positive cell densities in the tumor infiltration front and the tumor center area by automatically determining the tumor infiltration front and the tumor center area, evaluates the CD3 and CD8 positive cell densities in the tumor infiltration front as a prognostic factor, and thus assists pathologists in diagnosing and treating colorectal cancer patients.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of digital image processing, and particularly relates to a colorectal cancer IHC staining image tumor infiltration front immune grading method, system and storage medium. BACKGROUND

[0002] Colorectal cancer is the third most common malignant tumor, accompanied by high morbidity and mortality. When malignant tumor carcinogenesis occurs, it will cause changes in the tumor microenvironment. Tumor-infiltrating lymphocytes are an important component of the immune microenvironment of solid tumors, which can predict the prognosis and treatment effect of solid tumors. CD3 and CD8 can be used as markers for evaluating the prognosis of colorectal cancer. If the density of CD3 and CD8 positive cells in the tumor infiltration front region and the tumor center of the IHC staining image can be compared with the relationship between the prognosis evaluation of patients, only the key area analysis can achieve the same effect as analyzing the whole slice. Selecting CD3 positive cells or CD8 positive cells with strong prognostic value in the tumor infiltration front region and the tumor center can simplify the previous method of evaluating CD3 and CD8 positive cells in all tissue regions of the staining image, greatly reducing the workload of pathologists and the expenditure of patients, and improving work efficiency. SUMMARY

[0003] The main purpose of the present application is to overcome the shortcomings and deficiencies of the prior art, and provide a colorectal cancer IHC staining image tumor infiltration front immune grading method, system and storage medium. The present application can automatically calculate the immune grading of the tumor infiltration front region based on T lymphocytes according to the IHC staining image of the colorectal cancer patient.

[0004] In order to achieve the above purpose, the present application adopts the following technical scheme:

[0005] In a first aspect, the present application provides a colorectal cancer IHC staining image tumor infiltration front immune grading method, comprising the following steps:

[0006] S1, automatically classifying the tissues of the colorectal cancer IHC staining image to obtain nine types of tissues in the colorectal cancer carcinogenic region, and merging the nine types of tissues into five classifications; the nine types of tissues include tumor epithelium, tumor stroma, tumor necrosis, mucus, muscle, normal gland, lymph aggregation, fat and background; the five classifications include tumor region, normal region, gland region, other and background region;

[0007] S2, automatically segmenting tumor cells or dispersed cell clusters in the tumor infiltration front region, wherein the tumor infiltration front region refers to the region where the tumor region and the normal region overlap;

[0008] S3, automatically identifying a tumor infiltration front region and a tumor center region to obtain specific corresponding positions of the tumor infiltration front region and the tumor center region on a digital pathology section;

[0009] S4, segmenting CD3 and CD8 positive cells in the region to determine densities of the CD3 and CD8 positive cells in the tumor infiltration front region and the tumor center region, thereby laying a foundation for the prognosis of the evaluation index;

[0010] S5, immunohistochemical grading of the CD3 and CD8 positive cell densities in the tumor infiltration front region, wherein the immunohistochemical grading of the colorectal cancer patient is determined according to the densities of the CD3 and CD8 positive cells in the tumor infiltration front region, and the immunohistochemical grading is immunohigh and immunolow.

[0011] As a preferred technical solution, the step S1 is specifically:

[0012] S11, obtaining an IHC staining image labeled with tumor epithelium, tumor stroma, tumor necrosis, mucus, muscle, normal gland, lymph aggregation, fat and background, and an unlabeled IHC staining image;

[0013] S12, training a classifier by using the labeled IHC staining image and the unlabeled IHC staining image, wherein the classifier obtains segmentation images of nine kinds of tissue types by classifying sliding windows;

[0014] S13, merging the segmentation images of the nine kinds of tissue types to obtain five classifications of tumor regions, normal regions, gland regions, other regions and background regions, wherein the tumor regions include tumor necrosis, tumor stroma and tumor epithelium regions, and mucus; the normal regions include fat and muscle; the gland regions include normal glands; the other regions include lymph aggregations; and the background regions include background.

[0015] As a preferred technical solution, the step S2 is specifically:

[0016] S21, converting the five-class IHC staining image from an RGB image to a grayscale image;

[0017] S22, filtering the grayscale image by using two-dimensional Gaussian smoothing to reduce image noise and details;

[0018] S23, determining a global threshold value by using an Otsu method, setting a minimum value of 0 and a maximum value of 255, wherein the Otsu method automatically finds an optimal threshold value for calculation; performing binaryzation processing on the grayscale image, dividing the binaryzation-processed image into a black background and a white region of interest, calculating areas of each connected domain according to pixel area sizes and arranging the areas in descending order, and retaining the most significant region as the region of interest;

[0019] S24, in the case of keeping the overall position and shape of the black and white image after the binarization processing unchanged, the small cracks in the black and white image after the binarization processing are filled by using the morphological closing operation of dilation and erosion, and the isolated small points, burrs and smooth boundaries are effectively removed;

[0020] S25, the overlapping area of the tumor area and the normal tissue area is taken as the tumor infiltration front.

[0021] As a preferred technical solution, step S3 is specifically:

[0022] The tumor area and the normal tissue area are subjected to the morphological closing operation of dilation and erosion, and the local maximum value is obtained by dilation. A convolution kernel B1 is defined, and a reference point is defined separately. The convolution kernel B1 is convolved with the image A1, the maximum value of the pixels in the convolution kernel B1 covering area is calculated, and the maximum value is assigned to the pixel specified by the reference point. Then, the erosion operation is performed. A convolution kernel B2 is defined, and a reference point is defined separately. The convolution kernel B2 is convolved with the image A2, the minimum value of the pixels in the convolution kernel B2 covering area is calculated, and the minimum value is assigned to the pixel specified by the reference point. Finally, the overlapping part of the above two parts is automatically determined.

[0023] As a preferred technical solution, before the CD3 and CD8 positive cell segmentation in step S4, the following steps are further included:

[0024] The image after the morphological closing operation is subjected to color deconvolution, and the DAB channel image I DAB is obtained. DAB2 The mask is used to obtain the gray scale image I DAB3 without the background area.

[0025] As a preferred technical solution, the CD3 and CD8 positive cell segmentation is specifically:

[0026] S41, the step-by-step local threshold segmentation method is used to segment the cell nucleus in the image I DAB3 . The initial window width W is set to 77 pixels, and the preset contrast threshold T d is set to 15. The image I DAB3 is subjected to binarization processing to obtain the binarization mask M1. The mask operation is performed on the image I DAB2 using M1 to obtain I DAB4 . The morphological features of all connected domains in M1 are extracted, including pixel area and compactness. The morphological features of I DAB4Extracting gray scale features: mean and contrast; setting the first segmentation condition as: connected domain with pixel area less than 200, or contrast less than 0.04, or gray scale mean greater than 200, then saving connected domain with pixel area less than 2000 and compactness greater than 0.93 as N1, and saving connected domain not meeting the condition as M2 for next step segmentation;

[0027] S42, second segmentation; if M2 is empty, skip this step and set all pixel points of M2 as 0, if M2 is not empty, use M2 to perform mask operation on IDAB2 to obtain I DAB5 , adjusting the size W of local window to 47 pixels, keeping the contrast threshold unchanged, performing local threshold segmentation to obtain M3; using the watershed algorithm with foreground marking to segment M3, setting the minimum value parameter H as 3 to obtain M4, repeating the morphological feature segmentation step, saving connected domain with compactness greater than 0.95 and area less than 1000 pixels as N2, and saving connected domain not meeting the condition as M5;

[0028] S43, third segmentation; if M5 is empty, skip this step and set all pixel points of N3 as 0, if M5 is not empty, repeating the operation of the previous step, except that the size W of local window of Bernsen segmentation is set as 17 pixels, the minimum value H of the watershed algorithm with foreground marking is set as 1, and the morphological operation is skipped to obtain N3, and finally obtaining the binary result of immune cells as N DAB =N1|N2|N3.

[0029] As a preferred technical solution, in step S5, the tumor infiltrating front CD3, CD8 positive cell density immune grading is specifically:

[0030] S51, calculating the number of CD3, CD8 positive cells in the tumor infiltrating front region, and normalizing the result, D'=(D i -D min ) / (D max -D min ), Di is the normalized score of the IHC staining map of a colorectal cancer patient, D max and D min are the maximum and minimum values of the normalized scores of the IHC staining maps of the colorectal cancer patients in the cohort, respectively;

[0031] S52, in the evaluation of the predictive performance of the cell density variable, the continuity of the variable is maintained to avoid the influence of the cutoff setting on the results, the Cox proportional risk model is established by the survival status of the patients, the indicators of age, gender and survival period OS to calculate the risk ratio of each variable model to predict OS, Pearson correlation analysis, multivariate and univariate logistic regression analysis are used, and the number of characteristics P<0.05 is selected for data evaluation, and the comparison between the two groups adopts X 2 test, and the best cutoff point is determined by adopting the methods of binary classification and three classification;

[0032] S53, immune grading; according to the binary classification best cutoff point determined in step S52, the number of CD3 and CD8 positive cells in the tumor infiltration front area is CD CT -High and CD CT -Low two groups, immune grading is carried out for all patients in the cohort, if the number of CD3 and CD8 positive cells in the tumor infiltration front area in the IHC staining image of the patient is lower than the cutoff point, it is CD CT -Low group, if the number of CD3 and CD8 positive cells in the tumor infiltration front area in the IHC staining image of the patient is higher than the cutoff point, it is CD CT -High group; according to the two best cutoff points of three classification determined in step S52, the number of CD3 and CD8 positive cells in the tumor infiltration front area is divided into CD CT -High, CD CT -Intermediate, CD CT -Low three groups, if the number of CD3 and CD8 positive cells in the tumor infiltration front area in the IHC staining image of the patient is lower than the first cutoff point, it is CD CT -Low group, if the number of CD3 positive cells in the tumor infiltration front area in the IHC staining image of the patient is lower than the first cutoff point and less than the second cutoff point, it is CD CT -Intermediate group, if the number of CD3 and CD8 positive cells in the tumor infiltration front area in the IHC staining image of the patient is higher than the second cutoff point, it is CD CT -High.

[0033] In the second aspect, the application also provides an immune grading system for the tumor infiltration front of the IHC staining image of colorectal cancer, which comprises a tissue classification module, a region segmentation module, a region identification module, a cell segmentation module and an immune grading module.

[0034] The tissue classification module is used for automatically classifying the tissues of the colorectal cancer IHC staining image to obtain nine tissue types of the cancerous region of the colorectal cancer, and the nine tissue types are combined into five classifications; the nine tissue types include tumor epithelium, tumor stroma, tumor necrosis, mucus, muscle, normal gland, lymph aggregation, fat and background; and the five classifications include tumor region, normal region, gland region, other and background region.

[0035] The region segmentation module is used for automatically segmenting tumor cells or dispersed cell groups in the tumor infiltration front region, and the tumor infiltration front region refers to a region where the tumor region and the normal region overlap.

[0036] The region identification module is used for automatically identifying the tumor infiltration front region and the tumor center region to obtain specific corresponding positions of the tumor infiltration front region and the tumor center region on the digital pathology slice.

[0037] The cell segmentation module is used for segmenting CD3 and CD8 positive cells in the region to determine the density of the CD3 and CD8 positive cells in the tumor infiltration front region and the tumor center region, and lays a foundation for the prognosis of the evaluation index.

[0038] The immune grading module is used for immune grading of the density of CD3 and CD8 positive cells in the tumor infiltration front region, and determines the immune grading of the colorectal cancer patient according to the density of the CD3 and CD8 positive cells in the tumor infiltration front region, and the immune grading includes immune high grade and immune low grade.

[0039] In a third aspect, the present application further provides an electronic device, which comprises:

[0040] at least one processor; and

[0041] a memory connected with the at least one processor in communication; wherein

[0042] the memory stores computer program instructions executable by the at least one processor, and the computer program instructions are executed by the at least one processor to enable the at least one processor to execute the immune grading method of the tumor infiltration front of the colorectal cancer IHC staining image.

[0043] In a fourth aspect, the present application further provides a computer readable storage medium storing a program, and the program is executed by a processor to implement the immune grading method of the tumor infiltration front of the colorectal cancer IHC staining image.

[0044] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0045] 1) The present application can accurately segment the tumor infiltration front area and the tumor area, providing a basis for subsequent manual analysis and automatic analysis.

[0046] 2) The present application determines the immune grading of CD3 and CD8 positive cell density in the tumor infiltration front area by comparing the CD3 and CD8 related variables in the tumor infiltration front area and the tumor center area, which can be used as a new prognostic factor independent of TNM staging.

[0047] 3) The present application presents the CD3 and CD8 positive cell density in the tumor infiltration front area, which, combined with clinical data, can be used for prognostic analysis of disease-free survival of patients. Compared with the corresponding model without the prognostic factor proposed in the present application, the C-index of the model is significantly improved, and the results of the model combined with CD3 and CD8 positive cells are similar. The CD3 and CD8 positive cell density in the tumor infiltration front area can predict the survival of patients, and the high density of CD CT High is closely related to longer survival, and low density of CD CT Low is closely related to shorter survival. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0049] Figure 1 The flow chart of the immune grading method of the tumor infiltration front area of the IHC staining image of the colorectal cancer in the embodiment of the present application is shown in the figure.

[0050] Figure 2 The block diagram of the immune grading system of the tumor infiltration front area of the IHC staining image of the colorectal cancer in the embodiment of the present application is shown in the figure.

[0051] Figure 3 The structure diagram of the electronic device in the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0052] In order to make the person skilled in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0053] Reference to an "embodiment" in this application means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. As will be apparent to those of ordinary skill in the art, embodiments described in this application can be combined with one another.

[0054] As shown in the following table, the present embodiment is an immune grading method for tumor infiltration front of colorectal cancer IHC staining image, which comprises the following steps: Figure 1

[0055] S1, IHC staining image is automatically classified into nine types of tissues in the cancerous region of colorectal cancer, and on this basis, the nine types of tissues are subdivided and combined into five categories.

[0056] Further, the nine classification and five classification steps of the cancerous region are as follows:

[0057] S1.1, obtain IHC staining images labeled with tumor epithelium, tumor stroma, tumor necrosis, mucus, muscle, normal gland, lymph aggregation, fat and background, and unlabeled IHC staining images.

[0058] S1.2, use the above labeled and unlabeled images to train the classifier, and the classifier obtains the segmentation image of the nine types of tissues by classifying the sliding window.

[0059] S1.3, to determine the tumor infiltration front region, the nine types of tissue segmentation images are combined to obtain five classifications of tumor region, normal region, gland region, other region and background region; the tumor region includes tumor necrosis, tumor stroma and tumor epithelium region, mucus; the normal region includes fat and muscle; the gland region includes normal gland; the other region includes lymph aggregation; and the background region includes background.

[0060] S2, automatically segment the tumor cells or dispersed cell clusters in the tumor infiltration front region, which refers to the region where the tumor region and the normal region overlap.

[0061] Further, the specific steps of automatic identification of the tumor infiltration front region and the tumor center are as follows:

[0062] S2.1, convert the five-classified staining image, from RGB image to grayscale image.

[0063] S2.2, filter the grayscale image using a two-dimensional Gaussian smoothing with a standard deviation of 10 to reduce image noise and details.

[0064] ​S2.3, using Otsu method to determine the global threshold, the image obtained in the previous step is binarized, and the image is divided into black background and white region of interest. The area of each connected domain is calculated according to the pixel area size and arranged in descending order, and the most significant region is reserved as the region of interest.

[0065] More specifically, a minimum value of 0 and a maximum value of 255 are set, and the Otsu method automatically finds the optimal threshold for calculation; the gray scale image is binarized, and the binarized image is divided into black background and white region of interest, the area of each connected domain is calculated according to the pixel area size and arranged in descending order, and the most significant region is reserved as the region of interest.

[0066] S2.4, under the condition of keeping the overall position and shape of the black and white image after binarization unchanged, the morphological closing operation of expansion and corrosion is adopted to fill the small cracks in the black and white image after binarization, effectively removing isolated small points, burrs and smoothing the boundary.

[0067] S2.5, the overlapping region of the tumor region and the normal tissue region is taken as the tumor infiltration front.

[0068] S3, automatically identifying the tumor infiltration front region and the tumor center region, obtaining the specific corresponding position of the tumor infiltration front region and the tumor center region on the digital pathology section, specifically:

[0069] S31, the tumor region and the normal tissue region are subjected to morphological closing operation of expansion and erosion, and the local maximum value is obtained by expansion. First, define a convolution kernel B, and have a separately defined reference point. Convolve kernel B with image A, calculate the maximum value of the pixels in the kernel B coverage area, and assign this maximum value to the pixel specified by the reference point. The calculation formula is:

[0070] S32, then perform the erosion operation, define a convolution kernel B, and have a separately defined reference point. Convolve kernel B with image A, calculate the minimum value of the pixels in the kernel B coverage area, and assign this minimum value to the pixel specified by the reference point. The calculation formula is:

[0071] S33, automatically determine the overlapping part of the above two parts, for example, the tumor infiltration front width of the embodiment is 500 μm.

[0072] S4, CD3 and CD8 positive cell segmentation in the region; determine the density of CD3 and CD8 positive cells in the tumor infiltration front and the tumor center, and lay a foundation for the prognosis of evaluation index.

[0073] The specific steps of the pretreatment before the cell nucleus segmentation are as follows:

[0074] S4.1, DAB channel image I after color deconvolution DAB Separately, filter the image using a two-dimensional Gaussian smoothing kernel with a standard deviation of s (s = 3) to obtain a Gaussian blurred grayscale image I DAB2 . Using the mask, obtain a grayscale image I removing the background region DAB3 .

[0075] The specific steps for segmenting CD3 and CD8 positive cells in the tumor infiltration front and tumor center region are as follows:

[0076] S4.2, using a stepwise local threshold segmentation method on I DAB3 Segment the nuclei in the image. Set the initial window width W to 77 pixels, and the preset contrast threshold T d to 15, and perform the bernsen algorithm on I DAB3 to obtain a binary mask M1. Use M1 to mask I DAB2 to obtain I DAB4 ; extract morphological features for all connected domains within M1: pixel area and compactness; extract grayscale features for I DAB4 : mean and contrast; set the condition to be: connected domains with pixel area less than 200, or contrast less than 0.04, or grayscale mean greater than 200; then, save connected domains with pixel area less than 2000 and compactness greater than 0.93 as N1. Save connected domains that do not meet the conditions as M2 for the next step of segmentation.

[0077] S4.3, second segmentation; if M2 is empty (i.e. all pixel points are background), skip this step and set all pixel points in M2 to 0. If M2 is not empty, mask I DAB5 using M2 to obtain I DAB . Adjust the size of the local window W to 47 pixels, and keep the contrast threshold unchanged, perform local threshold segmentation to obtain M3. Use the watershed algorithm with foreground marking on M3 to perform segmentation, and set the minimum value parameter H to 3 to obtain M4. Repeat the morphological feature segmentation step to retain connected domains with compactness greater than 0.95 and area less than 1000 pixels, and save them as N2. Save connected domains that do not meet the conditions as M5.

[0078] S4.4, third segmentation. If M5 is empty, skip this step and set all pixel points in N3 to 0. If M5 is not empty, repeat the operation of the previous step on M5, except that the local window size W for bernsen segmentation is set to 17 pixels, the minimum value H in the watershed algorithm with foreground marking is set to 1, and the morphological operation is skipped to obtain N3. The final binary result of immune cells is N DAB= N1|N2|N3.

[0079] S5. Tumor infiltrating front area CD3 positive cell density immune grading; determining the immune grading of the colorectal cancer patient according to the density of the CD3 positive cells in the tumor infiltrating front area, the immune grading being immune high grade and immune low grade.

[0080] Further, the method of tumor infiltrating front area CD3 positive cell density immune grading and the method of CD8 positive cell density immune grading are the same, and the following is described by taking the tumor infiltrating front area CD3 positive cell density immune grading as an example. The specific method is as follows:

[0081] S5.1. Calculate the number of CD3 positive cells in the tumor infiltrating front area, and normalize the results, D'=(D i -D min ) / (D max -D min ), Di is the normalized score of the IHC staining figure of a colorectal cancer patient in the cohort, D max and D min are the maximum and minimum values of the normalized scores of the IHC staining figures of the colorectal cancer patients in the cohort, respectively.

[0082] S5.2. When evaluating the predictive performance of the cell density variable, the continuity of the variable is maintained to avoid the influence of the cutoff setting on the results. Cox proportional risk model is established by the survival status, age, gender, survival time OS and other indicators of the patient to calculate the risk ratio of each variable model to predict OS, and Pearson correlation analysis, multivariate and univariate logistic regression analysis are used to select the number of characteristics P<0.05 for data evaluation. The comparison between the two groups uses X 2 test, and the best cutoff point is determined by using the binary and ternary classification methods.

[0083] S5.3. According to the binary classification best cutoff point determined in S5.2, the number of CD3 positive cells in the tumor infiltrating front area is divided into CD3 CT -High and CD3 CT -Low, and all patients in the cohort are immune graded. If the number of CD3 positive cells in the tumor infiltrating front area of the IHC staining figure of the patient is lower than the cutoff point, it is classified as CD3 CT -Low, and if the number of CD3 positive cells in the tumor infiltrating front area of the IHC staining figure of the patient is higher than the cutoff point, it is classified as CD3 CT -High. According to the two best cutoff points of ternary classification determined in S5.2, the number of CD3 positive cells in the tumor infiltrating front area is divided into CD3 CT -High, CD3 CT- Intermediate, CD3 CT - Low, if the number of CD3 positive cells in the tumor infiltrate front region of the patient's IHC staining image is lower than the first cutoff point, then the group of CD3 is CT - Low, if the number of CD3 positive cells in the tumor infiltrate front region of the patient's IHC staining image is lower than the first cutoff point and less than the second cutoff point, then the group of CD3 is CT - Intermediate, if the number of CD3 positive cells in the tumor infiltrate front region of the patient's IHC staining image is higher than the second cutoff point, then the group of CD3 is CT - High.

[0084] It can be understood that the method of immune grading of CD8 positive cell density is the same as the method of immune grading of CD3 positive cell density described above, and will not be described here.

[0085] The above operations are repeated on the validation set of the Sixth Affiliated Hospital of Sun Yat-sen University, and the results are consistent with those of the training set.

[0086] It should be noted that for the foregoing method embodiments, in order to facilitate description, they are all described as a combination of a series of actions, but those skilled in the art should know that the present application is not limited by the order of the described actions, because according to the present application, certain steps can be performed in other order or simultaneously.

[0087] Based on the same idea as the immune grading method of the tumor infiltrate front of the colorectal cancer IHC staining image in the above embodiment, the present application also provides an immune grading system for the tumor infiltrate front of the colorectal cancer IHC staining image, which can be used to execute the immune grading method for the tumor infiltrate front of the colorectal cancer IHC staining image described above. For the convenience of description, in the structural schematic diagram of the embodiment of the immune grading system for the tumor infiltrate front of the colorectal cancer IHC staining image, only the part related to the embodiment of the present application is shown, and those skilled in the art can understand that the illustrated structure does not constitute a limitation on the device, and can include more or fewer components than the illustrated, or combine certain components, or different component arrangements.

[0088] Please refer to Figure 2 In another embodiment of the present application, an immune grading system 100 for the tumor infiltrate front of the colorectal cancer IHC staining image is provided, which includes a tissue classification module 101, a region segmentation module 102, a region identification module 103, a cell segmentation module 104, and an immune grading module 105.

[0089] The tissue classification module 101 is used for automatically performing tissue classification on the colorectal cancer IHC staining map to obtain nine tissue types of the colorectal cancer canceration region, and the nine tissue types are combined into five classifications; the nine tissue types include tumor epithelium, tumor interstitium, tumor necrosis, mucus, muscle, normal gland, lymph aggregation, fat and background; and the five classifications include tumor region, normal region, gland region, other and background region.

[0090] The region segmentation module 102 is used for automatically segmenting tumor cells or dispersed cell groups in a tumor infiltration front region, and the tumor infiltration front region refers to a region where the tumor region and the normal region overlap.

[0091] The region identification module 103 is used for automatically identifying the tumor infiltration front region and the tumor center region to obtain specific corresponding positions of the tumor infiltration front region and the tumor center region on the digital pathology slice.

[0092] The cell segmentation module 104 is used for region-in-CD3 and CD8 positive cell segmentation to determine the density of the CD3 and CD8 positive cells in the tumor infiltration front region and the tumor center region, and lay a foundation for the prognosis of the evaluation index.

[0093] The immune grading module 105 is used for immune grading of the CD3 and CD8 positive cell density in the tumor infiltration front region, and determines the immune grading of the colorectal cancer patient according to the density of the CD3 and CD8 positive cells in the tumor infiltration front region, and the immune grading is immune high grade and immune low grade.

[0094] It should be noted that the colorectal cancer IHC staining map tumor infiltration front immune grading system of the present application corresponds to the colorectal cancer IHC staining map tumor infiltration front immune grading method of the present application, and the technical features and advantages described in the above embodiment of the colorectal cancer IHC staining map tumor infiltration front immune grading method are applicable to the embodiment of the colorectal cancer IHC staining map tumor infiltration front immune grading system, and the specific content can be referred to the description in the method embodiment of the present application, which will not be described here again, and hereby declared.

[0095] In addition, in the embodiment of the colorectal cancer IHC staining map tumor infiltration front immune grading system of the above embodiment, the logical division of each program module is only an example, and in actual application, the above functions can be completed by different program modules according to the needs, for example, the configuration requirements of the corresponding hardware or the convenience of software implementation, that is, the internal structure of the colorectal cancer IHC staining map tumor infiltration front immune grading system is divided into different program modules to complete all or part of the functions described above.

[0096] Please refer to Figure 3In one embodiment, an electronic device for performing an immunohistochemical (IHC) staining image-based immunohistochemical grading of tumor infiltrating frontiers of colorectal cancer is provided. The electronic device 200 can include a first processor 201, a first memory 202, and a bus. The electronic device 200 can further include a computer program stored in the first memory 202 and executable on the first processor 201, such as an immunohistochemical grading program 203 for performing an IHC staining image-based immunohistochemical grading of tumor infiltrating frontiers of colorectal cancer.

[0097] The first memory 202 can include at least one type of readable storage medium including a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a magnetic memory, a disk, a compact disk, etc. The first memory 202 can be an internal memory unit of the electronic device 200 in some embodiments, such as a mobile hard disk of the electronic device 200. The first memory 202 can also be an external storage device of the electronic device 200 in other embodiments, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the first memory 202 can include both an internal memory unit and an external storage device of the electronic device 200. The first memory 202 can be used to store application software installed in the electronic device 200 and various data, such as a code of the immunohistochemical grading program 203 for performing an IHC staining image-based immunohistochemical grading of tumor infiltrating frontiers of colorectal cancer, and can be used to temporarily store data that has been output or will be output.

[0098] The first processor 201 can include an integrated circuit in some embodiments, such as a single packaged integrated circuit or a plurality of packaged integrated circuits having the same or different functions. The first processor 201 can include one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, combinations thereof, and the like. The first processor 201 is a control unit of the electronic device 200, and is connected to various components of the electronic device 200 via various interfaces and lines. The first processor 201 executes or runs programs or modules stored in the first memory 202 and calls data stored in the first memory 202 to perform various functions and processes data of the electronic device 200.

[0099] Figure 3 Only the electronic device having components is shown, and those skilled in the art can understand that the electronic device can further include other components not shown in the drawing, Figure 3The illustrated structure does not constitute a limitation on the electronic device 200, and can include fewer or more components than illustrated, or combine certain components, or different component arrangements.

[0100] The immune grading program 203 stored by the first memory 202 in the electronic device 200 is a combination of a plurality of instructions, which, when running in the first processor 201, can realize:

[0101] S1, automatically classifying the tissues of the colorectal cancer IHC staining image to obtain nine types of tissues in the cancerous region of the colorectal cancer, and merging the nine types of tissues into five classifications; the nine types of tissues include tumor epithelium, tumor stroma, tumor necrosis, mucus, muscle, normal gland, lymph aggregation, fat, and background; the five classifications include tumor region, normal region, gland region, other, and background region;

[0102] S2, automatically segmenting tumor cells or dispersed cell clusters in the tumor infiltration front region, the tumor infiltration front region being a region where the tumor region and the normal region overlap;

[0103] S3, automatically identifying the tumor infiltration front region and the tumor center region to obtain specific corresponding positions of the tumor infiltration front region and the tumor center region on the digital pathology slice;

[0104] S4, segmenting CD3 and CD8 positive cells in the region to determine the density of CD3 and CD8 positive cells in the tumor infiltration front region and the tumor center region, thereby laying a foundation for the prognosis of the evaluation index;

[0105] S5, immune grading of the CD3 and CD8 positive cell density in the tumor infiltration front region, determining the immune grading of the colorectal cancer patient according to the density of CD3 and CD8 positive cells in the tumor infiltration front region, the immune grading being immune high and immune low.

[0106] Further, the modules / units of the electronic device 200, if realized in the form of software function units and sold or used as independent products, can be stored in a non-volatile computer readable storage medium. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM).

[0107] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer readable storage medium, and when the program is executed, the processes of the above-mentioned embodiment methods can be included. Any reference to memory, storage, database or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0108] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0109] The above embodiments are the preferred embodiments of the present application, but the embodiments of the present application are not limited to the above embodiments, and any changes, modifications, substitutions, combinations and simplifications of the embodiments of the present application without departing from the spirit and principles of the present application are equivalent replacement methods, and are included in the protection scope of the present application.

Claims

1. Immunohistochemical staining method for the immunohistological grading of the tumor infiltrate front of colorectal cancer, characterized in that, Comprising the following steps: S1, automatically classifying the IHC staining image of colorectal cancer to obtain nine types of tissue in the cancerous region of colorectal cancer, and merging the nine types of tissue into five classifications; the nine types of tissue include tumor epithelium, tumor stroma, tumor necrosis, mucus, muscle, normal gland, lymph aggregation, fat and background; the five classifications include tumor region, normal region, gland region, other region and background region; S2, automatically segmenting tumor cells or dispersed cell clusters in the tumor infiltration front region, wherein the tumor infiltration front region refers to a region where the tumor region and the normal region overlap; S3, automatically identifying the tumor infiltration front region and the tumor center region to obtain the specific corresponding positions of the tumor infiltration front region and the tumor center region on the digital pathology slice; S4, segmenting CD3 and CD8 positive cells in the region to determine the density of CD3 and CD8 positive cells in the tumor infiltration front region and the tumor center region, thereby laying a foundation for the prognosis of evaluation indexes; S5, immunohistochemical grading of the density of CD3 and CD8 positive cells in the tumor infiltration front region, wherein the immunohistochemical grading of the density of CD3 and CD8 positive cells in the tumor infiltration front region is as follows: S51, calculate the number of CD3, CD8 positive cells in the tumor infiltration front area, and normalize the results, D' = (D i -D min ) / (D max -D min ), Di is the normalized score of the IHC staining image of a colorectal cancer patient, D max and D min are the maximum and minimum values of the normalized scores of the IHC staining images of the colorectal cancer patients in the cohort, respectively; S52, in the evaluation of the predictive performance of the cell density variable, the continuity of the variable is maintained to avoid the influence of the result by the cutoff setting, the Cox proportional risk model is established by the survival state of the patient, the age, the gender, the index of the survival period OS to calculate the risk ratio of each variable model to predict OS, Pearson correlation analysis, multi-factor and single-factor logistic regression analysis are used, the number characteristics P < 0.05 are selected for data evaluation, the comparison between the two groups adopts X 2 test, the best cutoff point is determined by adopting the binary and ternary classification methods; S53, immune stratification; based on the binary optimal cutoff point determined in step S52, the number of CD3, CD8 positive cells in the tumor infiltrate front region is classified as CD CT -Low CT -Low CT -Low CT -High; based on the two optimal cutoff points determined in step S52 for the three-classification, the number of CD3, CD8 positive cells in the tumor infiltrate front region is classified as CD CT -High, CD CT -Intermediate, CD CT -Low CT -Low CT -Intermediate CT -High.

2. The method of immunohistochemical staining of colorectal cancer according to claim 1, wherein, The step S1 is specifically as follows: S11, obtaining IHC staining images labeled with tumor epithelium, tumor stroma, tumor necrosis, mucus, muscle, normal gland, lymph aggregation, fat and background, and unlabeled IHC staining images; S12, training a classifier using the labeled IHC staining images and the unlabeled IHC staining images, wherein the classifier obtains segmentation images of the nine types of tissue by classifying sliding windows; S13, merging the segmentation images of the nine types of tissue to obtain five classifications of tumor region, normal region, gland region, other region and background region, wherein the tumor region includes tumor necrosis, tumor stroma and tumor epithelium region, mucus; the normal region includes fat and muscle; the gland region includes normal gland; the other region includes lymph aggregation; and the background region includes background.

3. The method of immunohistochemical staining of colorectal cancer tumor infiltrating frontiers according to claim 1, characterized in that, The step S2 is specifically as follows: S21, converting the IHC staining image of the five classifications from an RGB image to a grayscale image; S22, filtering the grayscale image using two-dimensional Gaussian smoothing to reduce image noise and details; S23, determining a global threshold value using the Otsu method, setting a minimum value of 0 and a maximum value of 255, wherein the Otsu method automatically finds the optimal threshold value for calculation; performing binaryzation processing on the grayscale image, dividing the binaryzation-processed image into black background and white region of interest, calculating the area of each connected domain according to the size of the pixel area and arranging them in descending order, and retaining the most significant region as the region of interest; S24, in the case of keeping the overall position and shape of the black and white image after binarization processing unchanged, small cracks in the black and white image after binarization processing are filled by using morphological closing operation of dilation and erosion, and isolated small points, burrs and smooth boundaries are effectively removed; S25, the overlapping area of the tumor area and the normal tissue area is taken as the tumor infiltration front.

4. The method of immunohistochemical staining of colorectal cancer tumor infiltrating frontiers according to claim 3, characterized in that, Step S3 is specifically: The tumor area and the normal tissue area are subjected to morphological closing operation of dilation and erosion, and local maximum value is obtained by dilation. Firstly, a convolution kernel B1 is defined, and a reference point is separately defined. The convolution kernel B1 is convolved with the image A1, the maximum value of the pixels in the area covered by the convolution kernel B1 is calculated, and the maximum value is assigned to the pixel specified by the reference point. Then, an erosion operation is performed, a convolution kernel B2 is defined, and a reference point is separately defined. The convolution kernel B2 is convolved with the image A2, the minimum value of the pixels in the area covered by the convolution kernel B2 is calculated, and the minimum value is assigned to the pixel specified by the reference point. Finally, the overlapping part of the above two parts is automatically determined.

5. The method of immunohistochemical staining of colorectal cancer tumor infiltrate front according to claim 1, characterized in that, Before the CD3 and CD8 positive cell segmentation in the region in step S4, the following steps are further included: The image after morphological closing operation, the DAB channel image I after color deconvolution DAB Separately isolate, filter the image using a two-dimensional Gaussian smoothing kernel with a standard deviation of σ to obtain a Gaussian blurred grayscale image I DAB2 Use a mask to obtain a grayscale image I that removes the background region DAB3 .

6. The method of immunohistochemical staining of colorectal cancer tumor infiltrate front according to claim 1, characterized in that, The CD3 and CD8 positive cell segmentation in the region is specifically: S41, using the step local threshold segmentation method to I DAB3 Segment the nuclei in the image, set the initial window width W to 77 pixels, and set the preset contrast threshold T d 15, to I DAB3 Perform binaryzation processing to obtain a binary mask M1, use M1 to I DAB2 Mask operation to obtain I DAB4 For all connected domains within M1, extract morphological features: pixel area and compactness; for I DAB4 Extract the gray scale feature: Mean value and contrast ratio; The first segmentation condition is set as: a connected domain with a pixel area less than 200, a contrast ratio less than 0.04, or a mean gray value greater than 200. Then, a connected domain with a pixel area less than 2000 and a compactness greater than 0.93 is saved as N1, and a connected domain not meeting the condition is saved as M2, and the next step of segmentation is performed. S42, second segmentation; if M2 is empty, skip this step and set all pixel values of M2 to 0; if M2 is not empty, perform a mask operation on IDAB2 using M2 to obtain I DAB5 , adjust the size W of the local window to 47 pixels, keep the contrast threshold unchanged, perform local threshold segmentation to obtain M3; perform segmentation on M3 using a watershed algorithm with foreground marking, set the minimum value parameter H to 3 to obtain M4, repeat the morphological feature segmentation step, keep the connected domain with compactness greater than 0.95 and area less than 1000 pixels as N2, and keep the connected domain that does not meet the condition as M5; S43, third segmentation, if M5 is empty, skip this step, and set all pixel points of N3 to 0, if M5 is not empty, repeat the operation of the last step on M5, the difference is that the local window size W of Bernsen segmentation is set to 17 pixels, the minimum value H in the watershed algorithm with foreground marking is set to 1, the morphological operation is skipped, N3 is obtained, and the final binary result of the immune cells is N = N1|N2|N3. DAB ​ 7. The immunohistochemical staining pattern of the tumor infiltrating front of colorectal cancer for the immunohistochemical grading system according to any one of claims 1 to 6, characterized in that, The system comprises a tissue classification module, a region segmentation module, a region recognition module, a cell segmentation module and an immune grading module. The tissue classification module is used for automatically classifying the nine tissue types of the colorectal cancer IHC stained image to obtain nine tissue types of the colorectal cancer cancerous region, and the nine tissue types are combined into five classifications. The nine tissue types include tumor epithelium, tumor stroma, tumor necrosis, mucus, muscle, normal gland, lymph aggregation, fat and background. The five classifications include a tumor region, a normal region, a gland region, other and a background region. The region segmentation module automatically segments tumor cells or dispersed cell groups in the tumor infiltration front region, and the tumor infiltration front region refers to the region overlapping the tumor region and the normal region. The region recognition module is used for automatically recognizing the tumor infiltration front region and the tumor center region to obtain the specific corresponding positions of the tumor infiltration front region and the tumor center region on the digital pathology slice. The cell segmentation module is used for CD3 and CD8 positive cell segmentation in the region to determine the CD3 and CD8 positive cell density of the tumor infiltration front and the tumor center region, and lays a foundation for the prognosis of the evaluation index. The immune grading module is used for immune grading of the CD3 and CD8 positive cell density of the tumor infiltration front, and determines the immune grading of the colorectal cancer patient according to the CD3 and CD8 positive cell density of the tumor infiltration front. The immune grading includes immune high grade and immune low grade.

8. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory in communication with the at least one processor; wherein the memory stores computer program instructions executable by the at least one processor, the computer program instructions, when executed by the at least one processor, enable the at least one processor to perform the method of immunohistochemical grading of tumor infiltrating front of colorectal cancer IHC staining map according to any one of claims 1-6.

9. A computer-readable storage medium storing a program, the program comprising instructions which, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. the program, when executed by the processor, implements the method of immunohistochemical grading of tumor infiltrating front of colorectal cancer IHC staining map according to any one of claims 1-6.

Images