Pathological grading method and system based on HER2 immunohistochemical picture

Abstract

The invention discloses a pathology grading method and system based on HER2 immunohistochemical pictures, belongs to the technical field of immunohistochemical (IHC), bioinformatics and medical diagnosis, is used for carrying out pathology grading on a test sample, and comprises the following steps: collecting and processing a plurality of original HER2 immunohistochemical pictures of the test sample, and obtaining a sample picture set for pathology grading; reading a sample picture set and converting an RGB color space in the sample picture set into an HSV color space; determining the staining intensity and range of the cell membrane based on the immunohistochemical picture in the sample picture set converted to the HSV color space; extracting and determining the membrane rate based on the characterization of the dyeing intensity and range of the cell membrane, and carrying out test treatment on the membrane rate to obtain the membrane rate of the test set; the film rate is the ratio of the pixel where the film is located to the background pixel; evaluating the test sample based on the corrected membrane rate data, and performing pathological grading on HER2 expression in the test sample based on a grading system. The invention further discloses a system, electronic equipment and a computer readable storage medium.

Description

Technical Field

[0001] The present invention relates to the fields of immunohistochemistry (IHC), bioinformatics and medical diagnosis technology, and in particular to a pathological grading method and system based on HER2 immunohistochemistry images. Background Art

[0002] Human epidermal growth factor receptor-2 (HER-2) is an important predictive indicator for the prognosis and targeted therapy of breast cancer. Accurate detection of HER-2 status in breast cancer patients is of great significance for clinical diagnosis and treatment.

[0003] The clinical significance and applications of HER-2 immunohistochemical staining include:

[0004] Treatment guidance: HER2-positive patients are generally considered candidates for targeted therapy, which generally has better efficacy and tolerability than traditional chemotherapy regimens.

[0005] Prognostic assessment: HER2 positivity may be associated with disease recurrence rate and patient prognosis, so knowing HER2 status can help predict the patient's disease progression and survival expectations.

[0006] Treatment monitoring: The response to HER2-targeted therapy can be monitored by regular HER2 staining tests to adjust and optimize the treatment plan.

[0007] In conclusion, HER2 immunohistochemistry plays an important role in clinical practice. By accurately evaluating the expression level of HER2, it helps doctors develop individualized treatment plans for breast cancer patients, thereby improving patients' survival rate and quality of life.

[0008] At present, the most commonly used methods for detecting HER-2 in breast cancer at home and abroad are immunohistochemistry (IHC) and fluorescence in situ hybridization (FISH). Among them, IHC detection is easily affected by many factors such as the fixation time of the tested tissue, the selection of fixative, the selection of antibody kits and the interpretation of staining results. Specifically:

[0009] 1. Manual observation and evaluation of stained sections under a microscope

[0010] Technical protocol: HER2 expression was classified using a grading system of 0 to 3+ based on the intensity and extent of cell membrane staining.

[0011] Grading system:

[0012] Grade 0: No membrane staining or only very faint staining was observed.

[0013] Grade 1+: A small number of cells show faint membrane staining.

[0014] Grade 2+: Moderate membrane staining, but most cells are negative. Further confirmation is usually required.

[0015] Grade 3+: Strong and complete membranous staining, with most cells expressing HER2.

[0016] Further confirmation:

[0017] For samples with a grade of 2+, confirmatory testing such as fluorescence in situ hybridization (FISH) is usually required.

[0018] FISH can detect the amplification of the HER2 gene and help establish the exact HER2 status.

[0019] Program flaws: A questionnaire for manual film grading (Femandez A et al.JAMA Oncol Published online February 32022.4) found that the consistency rate of interpretation of HER2-positive and HER2-negative cases exceeded 90%, while the consistency rate was less than 70% in the interpretation of the subset of HER2 IHC 0 and 1+. The data of the consistency study of three pathologists reading 114 breast cancer samples released at the 2022 USCAP conference showed that the consistency rate of different physicians in HER2-positive interpretation reached 100%, but for HER2 low expression and HER2-negative groups, the consistency rate was only about 70%. This study was also published at USCAP this year. Two pathologists re-interpreted breast cancer previously diagnosed as HER2 0 or 1+ and re-grouped the interpretation results. The results showed that the interpretation consistency between pathologists was good, with a Kappa value of 0.62; however, the Kappa value of the consistency between the second round of interpretation and the historical interpretation results was only 0.16. If pathologists from different laboratories make interpretations, the consistency of interpretation may be lower. A study comparing HER2 expression levels assessed by local and central laboratories reported that up to 85% of samples assessed as HER2 0 by local laboratories were actually scored as IHC 1+ or 2+ after reassessment by the central laboratory, with only 15% concordance.

[0020] 2. Intelligent (AI) grading of immunohistochemistry images through machine learning

[0021] Technical solution: In recent years, artificial intelligence (AI) has been used to identify tumor cells through machine learning of immunohistochemistry images. It has also been used to analyze the HER2 expression level in tumor cells by learning the staining patterns of tumor cells, measuring staining intensity, cell membrane integrity and positive ratio, to assist in the interpretation of HER2 results.

[0022] Solution defects: The grade assessment of the results cannot be standardized, and the accuracy of the grade depends on the experience of the assessor. In addition, samples of grade 2+ need to be confirmed using FISH. IHC production of cells, matrix, cavities, and necrosis will cause changes in the range of IHC pathological film staining, affecting the judgment of registration during film reading. Third, machine learning requires a large number of samples for training. Due to the diversity of cell morphology in pathological films, more training samples are needed to improve accuracy. Summary of the invention

[0023] In order to solve the problems existing in the prior art, the present invention provides the following technical solutions, a pathological grading method and system based on HER2 immunohistochemistry images, which proposes to use color to grade and evaluate samples, and only a few groups of samples are needed to grade the samples. After the samples are stained with HER2 antibodies, the HER2 expression is classified using a grading system of 0 to 3+ according to the staining intensity and range of the cell membrane. The grading standard focuses on the staining intensity and range of the cell membrane, and the pixels near the membrane are filtered through the color hue (H), saturation (S) and brightness (V) and the filter, and the pixel parameters are quantified, and the quantified value matches the staining level of HER2 pathology.

[0024] In one aspect, the present invention provides a pathological grading method based on HER2 immunohistochemistry images, which is used to perform pathological grading on a test sample, comprising:

[0025] S1, collecting and processing a plurality of original HER2 immunohistochemistry images of the test sample, thereby obtaining a sample image set for the pathological grading;

[0026] S2, reading the sample picture set and converting the RGB color space in the sample picture set into the HSV color space;

[0027] S3, determining the staining intensity and range of the cell membrane based on the immunohistochemistry images in the sample image set converted into the HSV color space;

[0028] S4, extracting and determining the membrane rate based on the characterization of the staining intensity and range of the cell membrane, and performing test processing on the membrane rate to obtain the membrane rate of the test set; wherein the membrane rate is the ratio of the pixels where the membrane is located to the background pixels;

[0029] S5, evaluating the test sample based on the membrane rate of the test set and pathologically grading the HER expression in the test sample based on a grading system.

[0030] Preferably, the S1 includes:

[0031] S11, performing immunohistochemical staining on the cells or tissues in the multiple original HER2 immunohistochemical images;

[0032] S12, placing the immunohistochemical images after immunohistochemical staining under a microscope to capture images and form the sample image set.

[0033] Preferably, the S11 includes:

[0034] (1) Dewaxing: After baking the quality control slides consisting of the multiple original HER2 immunohistochemistry images, the quality control slides are taken out and transferred to xylene, and after staying in xylene I for 5 minutes, transferred to xylene II for 5 minutes, and finally transferred to xylene III for 5 minutes;

[0035] (2) Hydration: Take out the quality control sheet from the xylene III, drain and transfer to ethanol with different concentration gradients for hydration, stay in anhydrous ethanol I for 5 minutes, transfer to anhydrous ethanol II for 5 minutes, transfer to 95% ethanol for 5 minutes, transfer to 85% ethanol for 5 minutes, transfer to 75% ethanol for 5 minutes, and finally transfer to 50% ethanol for 5 minutes. After the end, take out from 50% ethanol and rinse with tap water for 5-10 minutes;

[0036] (3) Antigen repair: Place the quality control slide in a staining jar and add an appropriate amount of ethylenediaminetetraacetic acid (EDTA) antigen repair solution for heat repair. Boil for 20 minutes, cool naturally, and rinse with phosphate-tween buffer (PBST) three times, each time for 3 minutes.

[0037] (4) Circle drawing: Drain excess PBST from the quality control slide and draw a circle around the tissue with an immunohistochemistry pen;

[0038] (5) Blocking: add 3% hydrogen peroxide to the quality control plate to cover the tissue, incubate at room temperature for 20 minutes, and rinse with PBST three times, each time for 3 minutes;

[0039] (6) Blocking: Add blocking solution dropwise, incubate the quality control plate in a blocking solution with a concentration of 2%-5% for 20 minutes, and rinse with PBST three times, each time for 3 minutes;

[0040] (7) Primary antibody incubation: Drain excess PBST from the quality control slide, add HER2 antibody, and place in a wet box at 4°C overnight. After incubation, return to room temperature and rinse with PBST three times, 3 minutes each time.

[0041] (8) Secondary antibody incubation: Drain excess PBST from the quality control slide, add secondary antibody and incubate at room temperature for 35-40 minutes, then rinse with PBST three times, 3 minutes each time;

[0042] (9) DAB color development: Drain excess phosphate buffered saline (PBST) from the quality control slide, add diaminobenzidine (DAB) color development solution, stop the process at the appropriate time under microscope observation, and wash with tap water;

[0043] (10) Re-staining: Drain excess liquid from the quality control slide, add hematoxylin staining solution, incubate for 3-5 minutes, and rinse with tap water to turn blue;

[0044] (11) Dehydration and transparent treatment: The water used to rinse the quality control sheet is dried, and the ethanol dehydration is performed in the following order: first, dehydration is performed with 75% ethanol for 5 minutes, then with 95% ethanol for 5 minutes, then with anhydrous ethanol I for 5 minutes, then with anhydrous ethanol II for 5 minutes, and then the quality control sheet taken out from anhydrous ethanol II is placed in xylene I for 5 minutes, and then placed in xylene II for 5 minutes for transparent treatment; the dehydration and transparent treatment are performed in a fume hood;

[0045] (12) Sealing: Transfer the quality control slide to the sealing machine for sealing;

[0046] The S12 includes: an upright microscope with a magnification of 200x to take a screenshot and obtain an immunohistochemical image after the image is captured.

[0047] Preferably, the S2 includes:

[0048] S21, reading the sample picture set based on opencv;

[0049] S22, based on the cvCvtColor function of opencv, convert the RGB color space to the HSV color space.

[0050] Preferably, S3 includes:

[0051] S31, randomly extracting the values ​​corresponding to the hue (H), saturation (S) and brightness (V) of the stained colors on the M group of cell membranes in the sample image set; the M group of cell membranes are classified into three levels: dark color, medium color and light color;

[0052] S32, counting the values ​​corresponding to the hue (H), saturation (S) and brightness (V) on the N groups of cell membranes corresponding to each level, so as to determine the maximum value range of the hue (H), saturation (S) and brightness (V);

[0053] S33, extracting a membrane contour based on the maximum value range, and locking the value range of hue (H), saturation (S) and brightness (V) of each level based on the membrane contour as a representation of the staining intensity and range of the cell membrane;

[0054] The S33 includes:

[0055] (1) The values ​​of hue (H), saturation (S) and brightness (V) corresponding to multiple sampling positions corresponding to the three levels are collected by sampling statistics, and the contour image is extracted using the light area HSV;

[0056] (2) processing the mask in the contour image based on the morphological operation function in the OpenCV library to eliminate small objects and fill small holes, and smoothly connect the object boundaries to obtain the film contour;

[0057] (3) filtering pixels near the film outline based on the hue (H), saturation (S) and brightness (V) of the color of the test sample and a filter;

[0058] (4) The color parameters of the pixels near the membrane contour are quantified to form a quantified value, and the quantified value matches the staining level of HER2 pathology, so that it can be used as a representation of the staining intensity and range of the cell membrane.

[0059] Preferably, the S4 includes:

[0060] S41, extracting and determining the membrane rate based on the characterization of the staining intensity and range of the cell membrane;

[0061] S42, performing normalization processing on the film rate to obtain first film rate data;

[0062] S43, sorting and level merging correction are performed on the first film rate data, so as to obtain a parameter value CM_score which is positively correlated with the level as the film rate of the test set; the calculation formula of the film rate of the test set is shown in the following formula (1):

[0063] CM_score=1 / (abs(lgP)) (1);

[0064] If P=0, CM_score=0; where P represents the first film rate data.

[0065] Preferably, S5 comprises: using a grading system from 0 to 3+ to classify HER2 expression, wherein the grading criteria of the grading system focus on the staining intensity and range of the cell membrane; the evaluation criteria of the grading system are the value of the membrane rate CM_score, including:

[0066] 3+: CM_score is greater than or equal to 0.6;

[0067] 2+: The value of CM_score is greater than or equal to 0.3 and less than 0.6;

[0068] 1+: The value of CM_score is greater than or equal to 0.1 and less than 0.3;

[0069] 0: Negative, the CM_score value is less than 0.1.

[0070] The second aspect of the present invention is to provide a pathological grading system based on HER2 immunohistochemistry images, which is used to implement the method of the first aspect, comprising:

[0071] A sample image set module, used for collecting and processing a plurality of original HER2 immunohistochemistry images of a test sample, so as to obtain a sample image set for the pathological grading;

[0072] A color space conversion module, used for reading the sample picture set and converting the RGB color space in the sample picture set into the HSV color space;

[0073] A staining parameter determination module, used to determine the staining intensity and range of the cell membrane based on the immunohistochemistry pictures in the sample picture set converted to the HSV color space;

[0074] A membrane rate determination module for a test set, for extracting and determining the membrane rate based on the characterization of the staining intensity and range of the cell membrane, and performing test processing on the membrane rate to obtain the membrane rate of the test set; wherein the membrane rate is the ratio of the pixels where the membrane is located to the background pixels;

[0075] A pathological grading module is used to evaluate the test sample based on the corrected membrane rate data and to perform pathological grading on the HER2 expression in the test sample based on a grading system.

[0076] A third aspect of the present invention provides an electronic device, comprising a processor and a memory, wherein the memory stores a plurality of instructions, and the processor is configured to read the instructions and execute the method described in the first aspect.

[0077] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a plurality of instructions, and the plurality of instructions can be read by a processor and execute the method described in the first aspect.

[0078] The method, system and electronic device provided by the present invention have the following beneficial effects:

[0079] 1. Reduce the error of manual judgment, and standardize according to the film rate to judge the grade according to the provided evaluation criteria.

[0080] 2. Reduce the errors caused by cell morphology in the process of IHC pathology slide production. Enter the color HSV space to reduce morphological interference. HSV screening and filter processing can greatly reduce noise interference.

[0081] 3. Reduce the influence of partial dyeing depth. Enter the color HSV space to reduce morphological interference. HSV screening and filter processing can also reduce noise interference.

[0082] 4. Reduce the operation of 2+ for fish fluorescence staining, and the membrane rate can distinguish 1+ and 2+ to improve the distinction.

[0083] 5. It can read films in batches, reduce manual work, and perform programmed operations to improve operational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] Figure 1 This is a flow chart of a pathological grading method based on HER2 immunohistochemistry images described in an embodiment of the present invention.

[0085] Figure 2 The original picture provided by the present invention.

[0086] Figure 3 The histogram corresponding to each domain on the hsv provided by the present invention is shown in the order of hue (H), saturation (S) and brightness (V).

[0087] Figure 4 A schematic diagram for characterizing the staining intensity and range of the cell membrane provided by the present invention.

[0088] Figure 5 A schematic diagram of the structure of a pathological grading system based on HER2 immunohistochemistry images provided by the present invention.

[0089] Figure 6 A schematic structural diagram of an embodiment of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0090] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0091] The method provided by the present invention can be implemented in the following terminal environment, and the terminal may include one or more of the following components: a processor, a memory, and a display screen. The memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the method described in the following embodiment.

[0092] The processor may include one or more processing cores. The processor uses various interfaces and lines to connect various parts of the entire terminal, and executes various functions of the terminal and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory, and calling data stored in the memory.

[0093] The memory may include random access memory (RAM) or read-only memory (ROM). The memory may be used to store instructions, programs, codes, code sets or instructions.

[0094] The display screen is used to display the user interface of each application.

[0095] In addition, those skilled in the art can understand that the structure of the above terminal does not constitute a limitation on the terminal, and the terminal may include more or fewer components, or combine certain components, or arrange the components differently. For example, the terminal also includes components such as a radio frequency circuit, an input unit, a sensor, an audio circuit, and a power supply, which will not be described in detail here.

[0096] In addition to biomarkers, specific molecular features, such as gene mutations, can also be incorporated into the prognostic model for patients with early-stage liver cancer after surgery. Gene mutations, especially driver gene mutations, are an important cause of cancer development. Gene mutations are also important targets for cancer treatment and serve as important markers for prognosis.

[0097] Embodiment 1

[0098] like Figure 1 As shown, this embodiment provides a pathological grading method based on HER2 immunohistochemistry images, which is used to perform pathological grading on a test sample, wherein the test sample includes any available form of medical sample, which is within the protection scope of the present invention, and the method includes:

[0099] S1, collecting and processing a plurality of original HER2 immunohistochemistry images of the test sample, thereby obtaining a sample image set for the pathological grading;

[0100] Figure 2 The original picture provided by the present invention, wherein: (a) corresponds to a schematic diagram of a negative picture domain; (b) corresponds to a schematic diagram of a +1 picture domain; (c) corresponds to a schematic diagram of a +2 picture domain; (d) corresponds to a schematic diagram of a +3 picture domain.

[0101] As a preferred embodiment, the S1 includes:

[0102] S11, performing immunohistochemical staining on the cells or tissues in the plurality of original HER2 immunohistochemical images, comprising:

[0103] (1) Dewaxing: After baking the quality control slides consisting of the multiple original HER2 immunohistochemistry images, the quality control slides are taken out and transferred to xylene, and then placed in xylene I for 5 minutes, then transferred to xylene II for 5 minutes, and finally transferred to xylene III for 5 minutes;

[0104] (2) Hydration: Take out the quality control sheet from the xylene III, drain and transfer to ethanol with different concentration gradients for hydration, stay in anhydrous ethanol I for 5 minutes, transfer to anhydrous ethanol II for 5 minutes, transfer to 95% ethanol for 5 minutes, transfer to 85% ethanol for 5 minutes, transfer to 75% ethanol for 5 minutes, and finally transfer to 50% ethanol for 5 minutes. After the end, take out from 50% ethanol and rinse with tap water for 5-10 minutes;

[0105] (3) Antigen repair: Place the quality control slide in a staining jar and add an appropriate amount of EDTA antigen repair solution for heat repair. Boil for 20 minutes, and then rinse with phosphate-buffered saline (PBST) for 3 times, 3 minutes each time, after cooling naturally.

[0106] (4) Circle drawing: Drain excess PBST from the quality control slide and draw a circle around the tissue with an immunohistochemistry pen;

[0107] (5) Blocking: add 3% hydrogen peroxide to the quality control plate to cover the tissue, incubate at room temperature for 20 minutes, and rinse with PBST three times, each time for 3 minutes;

[0108] (6) Blocking: Add a blocking solution (in this embodiment, bovine serum albumin is used. Of course, those skilled in the art should know that other blocking solutions well known in the art can also be selected, all of which are within the scope of protection of the present invention), incubate the quality control plate in a blocking solution with a concentration of 2%-5% for 20 minutes, and rinse with PBST three times, each time for 3 minutes;

[0109] (7) Primary antibody incubation: Drain excess PBST from the quality control slide, add HER2 (4B5) antibody, and place in a wet box at 4°C overnight. After incubation, return to room temperature and rinse with PBST three times, 3 minutes each time.

[0110] (8) Secondary antibody incubation: Drain excess PBST from the quality control slide, add secondary antibody and incubate at room temperature for 35-40 minutes, then rinse with PBST three times, 3 minutes each time;

[0111] (9) DAB color development: Drain excess PBST from the quality control slide, add DAB color development solution, observe under a microscope and stop at the appropriate time, then wash with tap water;

[0112] (10) Re-staining: Drain excess liquid from the quality control slide, add hematoxylin staining solution, incubate for 3-5 minutes, and rinse with tap water to turn blue. The water flow should be slow.

[0113] (11) Dehydration and transparent treatment: The water used to rinse the quality control sheet is dried, and the ethanol dehydration is performed in the following order: first, dehydration is performed with 75% ethanol for 5 minutes, then with 95% ethanol for 5 minutes, then with anhydrous ethanol I for 5 minutes, then with anhydrous ethanol II for 5 minutes, and then the quality control sheet taken out from anhydrous ethanol II is placed in xylene I for 5 minutes, and then placed in xylene II for 5 minutes for transparent treatment; the dehydration and transparent treatment are performed in a fume hood;

[0114] (12) Sealing: Transfer the quality control slide to the sealing machine for sealing;

[0115] S12, placing the immunohistochemical images after immunohistochemical staining under a microscope to capture the images and form the sample image set;

[0116] Specifically, S12 includes: taking a screenshot with an upright microscope at a magnification of 200x to obtain an immunohistochemistry image after the image is captured.

[0117] As a preferred embodiment, the equipment and reagents for performing the immunohistochemical staining include:

[0118] Equipment: antibody slicer, slide spreader, slide baking machine, fully automatic slide stainer, fully automatic coverslipper, optical microscope;

[0119] Reagents: DAB color development solution, phosphate-buffered saline-Tween (PBST), blocking solution, 3% hydrogen peroxide, secondary antibody, hematoxylin staining solution, ethanol, xylene, mounting medium and immunohistochemical pen.

[0120] S2, reading the sample picture set and converting the RGB color space in the sample picture set into the HSV color space;

[0121] As a preferred embodiment, S2 includes:

[0122] S21, read the sample image set based on OpenCV; OpenCV (open source computer vision library) is a cross-platform computer vision library released based on BSD license (open source), which can run on Linux, Windows, Android and Mac OS operating systems. It is lightweight and efficient - consisting of a series of C functions and a small number of C++ classes, and providing interfaces for languages ​​such as Python, Ruby, and MATLAB, and implementing many general algorithms in image processing and computer vision.

[0123] S22, based on the cvCvtColor function of opencv, convert the RGB color space to the HSV color space;

[0124] In this embodiment, the function of the cvCvtColor function is color space conversion.

[0125] Function prototype:

[0126] void cvCvtColor(const CvArr*src, CvArr*dst, int code);

[0127] const CvArr*src: input image;

[0128] CvArr*dst: output image (the output image must be consistent with the input image in size, color bit depth, and channels);

[0129] int code: the converted color space, which can be taken from the macro: CV_BCR2HSV means converting the image from RGB space to HSV space. When the code selects CV_BGR2HSV, for 8-bit images, the RGB values ​​(RGB three-color superposition) need to be normalized to between 0 and 1, so that the H range in the HSV image is 0-360, and the S and V ranges are 0-1.

[0130] Figure 3 The histogram corresponding to each domain on the hsv provided by the present invention is shown in the order of hue (H), saturation (S) and brightness (V), wherein:

[0131] (a1) is the hue (H) histogram of the negative image domain, (a2) is the saturation (S) histogram of the negative image domain; (a3) ​​is the brightness (V) histogram of the negative image domain;

[0132] (b1) is the hue (H) histogram of the +1 image domain, (b2) is the saturation (S) histogram of the +1 image domain; (b3) is the brightness (V) histogram of the +1 image domain;

[0133] (c1) is the hue (H) histogram of the +2 image domain, (c2) is the saturation (S) histogram of the +2 image domain; (c3) is the brightness (V) histogram of the +2 image domain;

[0134] (d1) is the hue (H) histogram of the +3 image domain, (d2) is the saturation (S) histogram of the +3 image domain; (d3) is the brightness (V) histogram of the +3 image domain.

[0135] S3, determining the staining intensity and range of the cell membrane based on the immunohistochemistry images in the sample image set converted into the HSV color space;

[0136] As a preferred implementation, S3 includes:

[0137] S31, randomly extracting the values ​​corresponding to the hue (H), saturation (S) and brightness (V) of the dyed colors on the M groups of cell membranes in the sample image set; the multiple groups of cell membranes are classified according to three levels: dark, medium and light; in this embodiment, there are 3 groups of cell membranes of each level, so M is 9. Of course, the number of groups can be adjusted according to the actual calculation accuracy requirements and consistency requirements, which is within the protection scope of the present invention;

[0138] S32, counting the values ​​corresponding to the hue (H), saturation (S) and brightness (V) on the N groups of cell membranes corresponding to each level, so as to determine the maximum value range of the hue (H), saturation (S) and brightness (V);

[0139] In this embodiment, M is 9 and N is 3.

[0140] S33, extracting the membrane contour based on the maximum value range, and locking the value range of hue (H), saturation (S) and brightness (V) of each level based on the membrane contour as a representation of the staining intensity and range of the cell membrane.

[0141] As a preferred implementation, the S33 includes:

[0142] (1) The values ​​of hue (H), saturation (S) and brightness (V) corresponding to multiple sampling positions corresponding to the three levels are collected by sampling statistics, and the contour image is extracted using the light area HSV;

[0143] (2) processing the mask in the contour image based on the morphological operation function in the OpenCV library to eliminate small objects and fill small holes, and smoothly connect the object boundaries to obtain the film contour;

[0144] (3) filtering pixels near the film outline based on the hue (H), saturation (S) and brightness (V) of the color of the test sample and a filter;

[0145] Program example:

[0146] (I) Program code for processing the mask in the contour image based on the morphological operation function in the OpenCV library

[0147]

[0148]

[0149] (II) Collector program code:

[0150]

[0151]

[0152]

[0153] The results of three samplings at each level:

[0154] Negative three collection data:

[0155] The range of Hue, Saturation, and Value collected for the first time.

[0156] Hue: 114-173

[0157] Saturation: 3-55

[0158] Value:142-190

[0159] The range of Hue, Saturation, and Value collected for the second time.

[0160] Hue: 6-173

[0161] Saturation: 3-62

[0162] Value:138-199

[0163] The range of Hue, Saturation, and Value collected for the third time.

[0164] Hue: 111-173

[0165] Saturation: 4-76

[0166] Value:135-190

[0167] +Three times of data collection:

[0168] The range of Hue, Saturation, and Value collected for the first time.

[0169] Hue: 0-179

[0170] Saturation: 6-37

[0171] Value:137-195

[0172] The range of Hue, Saturation, and Value collected for the second time.

[0173] Hue: 0-179

[0174] Saturation: 6-71

[0175] Value:111-186

[0176] The range of Hue, Saturation, and Value collected for the third time.

[0177] Hue: 0-179

[0178] Saturation: 38-163

[0179] Value:67-193

[0180] ++Three times of data collection:

[0181] The range of Hue, Saturation, and Value collected for the first time.

[0182] Hue: 0-179

[0183] Saturation: 13-97

[0184] Value:75-176

[0185] The range of Hue, Saturation, and Value collected for the second time.

[0186] Hue: 0-179

[0187] Saturation: 0-165

[0188] Value:51-197

[0189] The range of Hue, Saturation, and Value collected for the third time.

[0190] Hue: 0-179

[0191] Saturation: 9-255

[0192] Value:17-98

[0193] +++Three times of data collection:

[0194] The range of Hue, Saturation and Value collected for the first time. Hue: 0-179

[0195] Saturation: 9-197

[0196] Value:57-193

[0197] The range of Hue, Saturation, and Value collected for the second time.

[0198] Hue: 0-179

[0199] Saturation: 44-214

[0200] Value:44-182

[0201] The range of Hue, Saturation, and Value collected for the third time.

[0202] Hue: 0-179

[0203] Saturation: 14-147

[0204] Value:33-178

[0205] (4) The color parameters of the pixels near the membrane contour are quantified to form a quantified value, and the quantified value matches the staining level of HER2 pathology, so that it can be used as a representation of the staining intensity and range of the cell membrane.

[0206] In this embodiment, Python programming is used to complete S3 and the opencv library is used to analyze the image. Based on the differences in the three directions of HSV at four levels, the value range of HSV near the cell membrane is determined.

[0207] Calculation principle

[0208] 1. By sampling and counting the HSV values ​​of the three sampling positions (as shown in Table 1), the shallow area HSV is used to extract the image.

[0209] Table 1

[0210]

[0211]

[0212] 2. Morphological operation to remove noise.

[0213] The morphological operation functions in the OpenCV (cv2) library are used to process the mask in the image, eliminating small objects, filling small holes, and smoothly connecting the object boundaries.

[0214] The results are as follows Figure 4The characterization schematic diagrams of the staining intensity and range of the cell membrane are shown, wherein: (a) a characterization schematic diagram of the staining intensity and range of the cell membrane corresponding to the negative image domain; (b) a characterization schematic diagram of the staining intensity and range of the cell membrane corresponding to the +1 image domain; (c) a characterization schematic diagram of the staining intensity and range of the cell membrane corresponding to the +2 image domain; (d) a characterization schematic diagram of the staining intensity and range of the cell membrane corresponding to the +3 image domain.

[0215] S4, based on the characterization of the staining intensity and range of the cell membrane, extract and determine the membrane rate, and perform test processing on the membrane rate to obtain the membrane rate of the test set; wherein the membrane rate is the ratio of the pixels where the membrane is located to the background pixels.

[0216] As a preferred implementation, S4 includes:

[0217] S41, extracting and determining the membrane rate based on the characterization of the staining intensity and range of the cell membrane;

[0218] S42, performing normalization processing on the film rate to obtain first film rate data;

[0219] In this embodiment, since the film rate value is relatively small, the film rate conversion coefficient is used to convert the film rate into a value between 0 and 1 as the first film rate data;

[0220] S43, sorting and level merging correction are performed on the first film rate data, so as to obtain a parameter value CM_score which is positively correlated with the level as the film rate of the test set; the calculation formula of the film rate of the test set is shown in the following formula (1):

[0221] CM_score=1 / (abs(lgP)) (1);

[0222] If P=0, CM_score=0; wherein P represents the first film rate data.

[0223] S5, evaluating the test sample based on the corrected membrane rate data and pathologically grading the HER2 expression in the test sample based on a grading system.

[0224] As a preferred embodiment, S5 includes: using a grading system from 0 to 3+ to classify HER2 expression, wherein the grading criteria of the grading system focus on the staining intensity and range of the cell membrane; the evaluation criteria of the grading system are the value of the membrane rate CM_score, including:

[0225] 3+: CM_score is greater than or equal to 0.6;

[0226] 2+: The CM_score value is greater than or equal to 0.3 and less than 0.6:

[0227] 1+: The value of CM_score is greater than or equal to 0.1 and less than 0.3;

[0228] 0: Negative, the CM_score value is less than 0.1.

[0229] Example test

[0230] Test set: 24 groups, 4 levels in each group, a total of 96 IHC pathology images (including images with too dark staining).

[0231] Test Results

[0232] The grade calculation results of 96 IHC staining images are shown in Table 2 below.

[0233] Table 2

[0234]

[0235]

[0236]

[0237] The grade statistics of the 96 IHC images are shown in Table 3 below.

[0238] Table 3

[0239] grade Negative 1+ 2+ 3+ Correct number 24 23 24 24 Number of errors 0 1 0 0

[0240] It can be seen that the calculation accuracy of the method in the embodiment of the present invention is 98.96%. Among them, the accuracy deviation includes the calculation error of one picture, which is a very darkly stained sample.

[0241] As a preferred embodiment, the method further includes: correcting the membrane rate of the test set based on batch processing of the quality control films to improve the accuracy of the rating.

[0242] like Figure 5 As shown, the second aspect of the present invention is to provide a pathological grading system based on HER2 immunohistochemistry images, which is used to implement the method of the first aspect, comprising:

[0243] The sample picture set module 101 is used to collect and process a plurality of original HER2 immunohistochemistry pictures of the test sample, so as to obtain a sample picture set for the pathological grading;

[0244] A color space conversion module 102, configured to read the sample picture set and convert the RGB color space in the sample picture set into the HSV color space;

[0245] A staining parameter determination module 103, for determining the staining intensity and range of the cell membrane based on the immunohistochemistry pictures in the sample picture set converted to the HSV color space;

[0246] The membrane rate determination module 104 of the test set is used to extract and determine the membrane rate based on the characterization of the staining intensity and range of the cell membrane, and perform test processing on the membrane rate to obtain the membrane rate of the test set; wherein the membrane rate is the ratio of the pixels where the membrane is located to the background pixels;

[0247] The pathological grading module 105 is used to evaluate the test sample based on the corrected membrane rate data and perform pathological grading on the HER2 expression in the test sample based on a grading system.

[0248] The present invention also provides a memory storing a plurality of instructions, wherein the instructions are used to implement the method as in the first embodiment.

[0249] like Figure 6 As shown, the present invention also provides an electronic device, including a processor 201 and a memory 202 connected to the processor 201, the memory 202 stores a plurality of instructions, and the instructions can be loaded and executed by the processor so that the processor can execute the method as in the first embodiment.

[0250] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A pathological grading method based on HER2 immunohistochemistry images, used for pathological grading of test samples, characterized in that: include: S1, collecting and processing a plurality of original HER2 immunohistochemistry images of the test sample, thereby obtaining a sample image set for the pathological grading; S2, reading the sample picture set and converting the RGB color space in the sample picture set into the HSV color space; S3, determining the staining intensity and range of the cell membrane based on the immunohistochemistry images in the sample image set converted into the HSV color space; S4, extracting and determining a membrane rate based on the characterization of the staining intensity and range of the cell membrane, wherein the membrane rate is the ratio of pixels where the membrane is located to background pixels, and performing a test processing on the membrane rate to obtain a membrane rate of a test set; S5, evaluating the test sample based on the membrane rate of the test set and pathologically grading the HER2 expression in the test sample based on a grading system.

2. A pathological grading method based on HER2 immunohistochemistry images according to claim 1, characterized in that: The S1 includes: S11, performing immunohistochemical staining on the cells or tissues in the multiple original HER2 immunohistochemical images; S12, placing the immunohistochemical images after immunohistochemical staining under a microscope to capture images to form the sample image set.

3. A pathological grading method based on HER2 immunohistochemistry images according to claim 2, characterized in that: The S11 includes: (1) Dewaxing: After baking the quality control slides consisting of the multiple original HER2 immunohistochemistry images, the quality control slides are taken out and transferred to xylene, and then placed in xylene I for 5 minutes, then transferred to xylene II for 5 minutes, and finally transferred to xylene III for 5 minutes; (2) Hydration: Take out the quality control sheet from the xylene III, drain and transfer to ethanol with different concentration gradients for hydration, stay in anhydrous ethanol I for 5 minutes, transfer to anhydrous ethanol II for 5 minutes, transfer to 95% ethanol for 5 minutes, transfer to 85% ethanol for 5 minutes, transfer to 75% ethanol for 5 minutes, and finally transfer to 50% ethanol for 5 minutes. After the end, take out from 50% ethanol and rinse with tap water for 5-10 minutes; (3) Antigen repair: Place the quality control slide in a staining jar and add an appropriate amount of EDTA antigen repair solution for heat repair. Boil for 20 minutes, and then rinse with phosphate-tween buffer (PBST) three times for 3 minutes each time after cooling naturally. (4) Circle drawing: Drain excess PBST from the quality control slide and draw a circle around the tissue with an immunohistochemistry pen; (5) Blocking: add 3% hydrogen peroxide to the quality control plate to cover the tissue, incubate at room temperature for 20 minutes, and rinse with PBST three times, each time for 3 minutes; (6) Blocking: Add blocking solution dropwise, incubate the quality control plate in a blocking solution with a concentration of 2%-5% for 20 minutes, and rinse with PBST three times, each time for 3 minutes; (7) Primary antibody incubation: Drain excess PBST from the quality control slide, add HER2 antibody, and place in a wet box at 4°C overnight. After incubation, return to room temperature and rinse with PBST three times, 3 minutes each time. (8) Secondary antibody incubation: Drain excess PBST from the quality control slide, add secondary antibody and incubate at room temperature for 35-40 minutes, then rinse with PBST three times, 3 minutes each time; (9) DAB color development: Drain excess PBST from the quality control slide, add DAB color development solution, observe under a microscope and stop at the appropriate time, then wash with tap water; (10) Re-staining: Drain excess liquid from the quality control slide, add hematoxylin staining solution, incubate for 3-5 minutes, and rinse with tap water to turn blue; (11) Dehydration and transparent treatment: The water used to rinse the quality control sheet is dried, and the ethanol dehydration is performed in the following order: first, dehydration is performed with 75% ethanol for 5 minutes, then with 95% ethanol for 5 minutes, then with anhydrous ethanol I for 5 minutes, then with anhydrous ethanol II for 5 minutes, and then the quality control sheet taken out from anhydrous ethanol II is placed in xylene I for 5 minutes, and then placed in xylene II for 5 minutes for transparent treatment; the dehydration and transparent treatment are performed in a fume hood; (12) Sealing: Transfer the quality control slide to the sealing machine for sealing; The S12 includes: an upright microscope with a magnification of 200x to take a screenshot and obtain an immunohistochemical image after the image is captured.

4. A pathological grading method based on HER2 immunohistochemistry images according to claim 3, characterized in that: The S2 includes: S21, reading the sample picture set based on opencv; S22, based on the cvCvtColor function of opencv, convert the RGB color space to the HSV color space.

5. A pathological grading method based on HER2 immunohistochemical images according to claim 4, characterized in that: The S3 includes: S31, randomly extracting the values ​​corresponding to the hue H, saturation S and brightness V of the stained colors on the M groups of cell membranes in the sample image set; the M groups of cell membranes are classified into three levels: dark color, medium color and light color; S32, counting the values ​​corresponding to the hue H, saturation S and brightness V on the N groups of cell membranes corresponding to each level, so as to determine the maximum value range of the hue H, saturation S and brightness V; S33, extracting a membrane contour based on the maximum value range, and locking the value range of hue H, saturation S and brightness V of each level based on the membrane contour as a representation of the staining intensity and range of the cell membrane; The S33 includes: (1) By sampling and counting the values ​​of hue H, saturation S and brightness V corresponding to multiple sampling positions corresponding to the three levels, the contour image is extracted using the light area HSV; (2) processing the mask in the contour image based on the morphological operation function in the OpenCV library to eliminate small objects and fill small holes, and smoothly connect the object boundaries to obtain the film contour; (3) filtering pixels near the film outline based on the hue H, saturation S and brightness V of the color of the test sample and a filter; (4) The color parameters of the pixels near the membrane contour are quantified to form a quantified value, and the quantified value matches the staining level of HER2 pathology, so that it can be used as a representation of the staining intensity and range of the cell membrane.

6. A pathological grading method based on HER2 immunohistochemical images according to claim 5, characterized in that: The S4 includes: S41, extracting and determining the membrane rate based on the characterization of the staining intensity and range of the cell membrane; S42, performing normalization processing on the film rate to obtain first film rate data; S43, sorting and level merging correction are performed on the first film rate data, so as to obtain a parameter value CM_score which is positively correlated with the level as the film rate of the test set; the calculation formula of the film rate of the test set is shown in the following formula (1): CM_score=1 / (abs(lgP)) (1); If P=0, CM_score=0; wherein P represents the first film rate data.

7. A pathological grading method based on HER2 immunohistochemical images according to claim 6, characterized in that: The S5 includes: using a grading system from 0 to 3+ to classify HER2 expression, wherein the grading criteria of the grading system focus on the staining intensity and range of the cell membrane; the evaluation criteria of the grading system are the numerical value of the membrane rate CM_score of the test set, including: 3+: CM_score is greater than or equal to 0.6; 2+: The value of CM_score is greater than or equal to 0.3 and less than 0.6; 1+: The value of CM_score is greater than or equal to 0.1 and less than 0.3; 0: Negative, the CM_score value is less than 0.

1.

8. A pathological grading system based on HER2 immunohistochemical images, used to implement the method of any one of claims 1 to 7, characterized in that: include: A sample image set module (101), used for collecting and processing a plurality of original HER2 immunohistochemistry images of a test sample, thereby obtaining a sample image set for the pathological grading; A color space conversion module (102), used for reading the sample picture set and converting the RGB color space in the sample picture set into the HSV color space; A staining parameter determination module (103), used to determine the staining intensity and range of the cell membrane based on the immunohistochemistry pictures in the sample picture set converted into the HSV color space; A membrane rate determination module (104) for a test set is used to extract and determine the membrane rate based on the characterization of the staining intensity and range of the cell membrane, and to perform test processing on the membrane rate to obtain the membrane rate of the test set; wherein the membrane rate is the ratio of the pixels where the membrane is located to the background pixels; A pathological grading module (105) is used to evaluate the test sample based on the membrane rate of the test set and to perform pathological grading on the HER expression in the test sample based on a grading system.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a plurality of instructions, and the processor is used to read the instructions and execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the plurality of instructions can be read by a processor to execute the method according to any one of claims 1 to 7.

Images