Color-related quality assessment method for pathological sections based on staining separation

Abstract

The color-related quality assessment method of pathological sections based on staining separation of the present invention includes an image preprocessing step, a feature extraction step and a classification step. The image preprocessing step includes: firstly extracting the tissue area in the pathological section image, then removing the red blood cell area and the tissue-free area that affect the color quality in the tissue area, and only retaining the tissue area that is colored after H&E staining; the feature extraction step is to extract features from the tissue area after preprocessing. The assessment method of the present invention first removes the blank area in the center of the image, removes the blank area around the image, and removes the red blood cell area in the tissue area image; then, by obtaining 13 feature vectors, and then inputting them into a machine learning model for classification, the obtained machine learning model can effectively evaluate and classify the color quality of the slice from objective conditions, reduce the subjective intention of manual evaluation, and reduce the workload of manual evaluation.

Description

Technical Field

[0001] The present invention relates to a method for evaluating the color-related quality of pathological sections, and more specifically, to a method for evaluating the color-related quality of pathological sections based on staining separation. Background Art

[0002] Pathological section quality control is the key link and foundation of clinical pathology quality control. The quality of pathological sections will directly affect the accuracy of pathological diagnosis, the corresponding medical quality will be reduced, and medical errors may also occur. To improve the quality of medical care, it is necessary to pay attention to the quality control of pathological sections. Usually, the quality assessment of pathological sections is performed manually. At regular intervals, the pathological sections of the pathology departments of hospitals at all levels are spot-checked, and the pathology experts uniformly evaluate the quality of the batch of pathological sections manually, and give the quality of pathological section production of each hospital.

[0003] At present, this random inspection method cannot represent the overall quality of a hospital's pathological section production to a certain extent, and manual evaluation will add the subjective intention of pathologists, which cannot truly explain the production quality of the actual sections. The color quality of pathological sections mainly evaluates the standardized staining and thickness of the sections. The sections under standard staining are bright red and blue, with only one layer of cells, while non-standard staining will result in three situations: light red, light blue, and poor contrast, as well as the situation where the sections are too thick. The present invention designs a pathological section feature extraction method based on staining separation, and designs a machine learning model, which can effectively evaluate and classify the color quality of the sections from objective conditions, reduce the subjective intention of manual evaluation, and reduce the workload of manual evaluation. Summary of the invention

[0004] In order to overcome the disadvantages of the above technical problems, the present invention provides a color-related quality assessment method for pathological sections based on staining separation.

[0005] The color-related quality assessment method of pathological sections based on staining separation of the present invention comprises an image preprocessing step, a feature extraction step and a classification step, and is characterized in that the image preprocessing step comprises: firstly extracting the tissue area in the pathological section image, then removing the red blood cell area and the tissue-free area that affect the color quality in the tissue area, and only retaining the tissue area colored after H&E staining; the feature extraction step is to extract features from the tissue area after preprocessing; and the classification step is used to train a machine learning model with the obtained features, so as to judge the quality of the section color.

[0006] In the color-related quality assessment method of pathological sections based on staining separation of the present invention, the image preprocessing step is specifically implemented by the following steps:

[0007] a) Obtaining pathological images, first obtaining a sufficient number of H&E stained digital pathological images obtained by scanning with a digital scanner, and then sequentially performing steps b) to f) on the acquired pathological images;

[0008] b) Find the blank area in the center of the pathological tissue. Since the G channel in the RGB three channels of the pathological image can display the pathological tissue area at the grayscale level, the blank area is filtered out using the G channel of the pathological image, which is achieved through steps b-1) to b-3):

[0009] b-1) Obtain and process the G channel image, record the initial pathological image as Image, extract the G channel in the Image image to form the G channel image Image_G; record the pixels with gray values ​​above 200 in the Image_G image as blank points, and the remaining pixels are pathological tissue areas;

[0010] b-2). Find the blank area in the Image_G image, and find out whether there is a blank area in the center of the Image_G image with an area that meets the requirements according to the connected area characteristics;

[0011] b-3). Process the flag image, initialize a flag image Flag_center with a gray value equal to the size of Image, if there is a central blank area in step b-2), then all the pixel values ​​of the positions corresponding to the central blank area in the Flag_center image are set to 255, that is, white; if there is no central blank area in step b-2), the flag image Flag_center remains completely black;

[0012] c). Find the blank area around the pathological tissue, initialize a grayscale value flag image Flag_Image with the same size as Image, find the largest connected area in the Flag_Image that matches the pathological tissue area according to the connected area characteristics, set the pixel points in the Flag_Image corresponding to the largest connected area to 255, that is, white; the area of ​​black dots in Flag_Image is the blank area around the tissue;

[0013] d) Obtaining an image of the tissue region, subtracting the Flag_Image image obtained in step c) from the Flag_center image obtained in step b), and obtaining an image of the tissue region with the central blank area and the surrounding blank area removed, recorded as Image_cutwhite;

[0014] e) Removing the location of red blood cells in the image of the tissue area is achieved through steps e-1) to e-3):

[0015] e-1). Space conversion: first, the pathological image Image_cutwhite with blank areas removed is converted from RGB space to HSV space to obtain the hue Image_H and saturation Image_S of the whole image;

[0016] e-2). Filter out the red blood cells, and then find out the location of the red blood cells in the hue image Image_H and the saturation image Image_S and filter them out according to the hue and saturation characteristics of the red blood cells, that is, the hue of the red blood cells exceeds 220 and the saturation of the red blood cells exceeds 125;

[0017] f). Obtain the image of the pathological tissue region to be analyzed, filter out the position of the red blood cells in the toned image Image_H, and then binarize the toned image Image_H with the red blood cells filtered out to obtain Image_H_bw; the position of the white dots in Image_H_bw is the tissue region, and the pixels at the corresponding position in the corresponding color image are the pathological tissue region Image_cutred to be analyzed.

[0018] In the color-related quality assessment method of pathological sections based on staining separation of the present invention, the feature extraction step is specifically implemented by the following steps:

[0019] 1) Obtain the features of thick slices, convert the pathological tissue area image Image_cutred from RGB space to HSV space, and use the mean of hue H, saturation S and brightness V as the features of thick slices, which are recorded as hue mean mean(Image_cutred_H), saturation mean mean(Image_cutred_S) and brightness mean mean(Image_cutred_V), a total of 3 features;

[0020] 2). Obtain the majority feature of the entire image, convert the pathological tissue area image Image_cutred from RGB space to HSV space. Since the brightness information of the image is similar, only the hue and saturation information are taken into account; using the histogram of the grayscale image under the hue H channel and the saturation S channel, obtain the image hue H space majority Image_H_most and the saturation S space majority Image_S_most, a total of 2 features;

[0021] 3). Obtain the red-blue deviation characteristics of the entire image, and convert the image Image_cutred from the RGB space to the Lab space; Lab space is a device-independent color system, a color system based on physiological characteristics, which uses a digital method to describe human visual perception; since the poor contrast of color quality, red lightness, and blue lightness are all descriptions of human visual perception, this space is used to describe the deviation of red and blue; similarly, regardless of the brightness of the image, the mean a_Image_mean of the a channel represents the degree of redness, and the mean b_Image_mean of the b channel represents the degree of blueness. The mean a_Image_mean of the a channel and the mean b_Image_mean of the b channel are used as the two features of the red-blue deviation;

[0022] 4) Obtain the color deviation characteristics of the dye separation image, which is achieved through steps 4-1) to 4-3):

[0023] 4-1) Convert to optical density value. According to the Lambert-Beer LAW shown in formula (1), the RGB color value of the pathological tissue area image Image_cutred is converted to optical density value:

[0024] OD = -log 10 (I) (1)

[0025] Where, I is the value of the RGB three channels of the pathological image normalized to the range of 0 to 1, and OD is the corresponding optical density field;

[0026] 4-2) To obtain the optical density field, firstly, singular value decomposition is performed on OD to obtain the two largest singular value vectors, and a coordinate plane is established in the direction indicated by these two vectors; then, the OD value converted by the image is projected on the newly created coordinate system to obtain two optical density fields corresponding to the H component and the E component;

[0027] 4-3). Obtain the dye separation image, and finally convert the optical density field obtained in step 4-2) back to RGB space to obtain the cell nucleus image Image_seperation_H based on the H dye and the cytoplasm image Image_seperation_E based on the E dye;

[0028] 4-4). Obtain the color deviation feature. First, convert the cell nucleus image Image_seperation_H based on H dye and the cytoplasm image Image_seperation_E based on E dye from RGB space to their respective Lab spaces. Then calculate the a channel and b channel means of the corresponding images. The a channel and b channel means of the cell nucleus image Image_seperation_H based on H dye are recorded as a_H_mean and b_H_mean, respectively. The a channel and b channel means of the cytoplasm image Image_seperation_E based on E dye are recorded as a_E_mean and b_E_mean, respectively. Finally, calculate the color deviation Color_Deviation_H and Color_Deviation_E of the cell nucleus image and cytoplasm image respectively according to formula (2):

[0029] Deviation=[(aa mean ) 2 +(bb mean ) 2 ] 1 / 2 (2)

[0030] The color deviations of the cell nucleus image and cytoplasm image, Color_Deviation_H and Color_Deviation_E, are used as two color deviation features;

[0031] 5). Obtain the majority feature of the stained separation image. To increase the distinguishability of the image, convert the nucleus image Image_seperation_H based on H dye and the cytoplasm image Image_seperation_E based on E dye from RGB space to HSV space, and calculate the modes of the H hue channel and S saturation channel of the two images in HSV space, respectively, and record them as the mode of the H channel of the nucleus image H_H_most, the mode of the S channel of the nucleus image H_S_most, the mode of the H channel of the cytoplasm image E_H_most, and the mode of the S channel of the cytoplasm image E_S_most.

[0032] The color-related quality assessment method of pathological sections based on staining separation of the present invention, the implementation method of the classification step is: after processing a sufficient number of digital pathological images through steps b) to f) and steps 1) to 5), a part of them is selected as a training set, a part as a test set, and the rest as a verification set, and the images in the test set are manually labeled with five classification labels, namely thick slices, light blue, light red, poor contrast, and red and blue standard slices, and then classified by a machine learning model, the input of the machine learning model is 13 feature vectors obtained after the pathological sections have undergone the feature extraction step, and the output is the color quality grade of the pathological sections.

[0033] The beneficial effects of the present invention are as follows: the color-related quality assessment method of pathological sections based on staining separation of the present invention first removes the blank area in the center, the blank area in the periphery, and the red blood cell area in the tissue area image of the H&E stained digital pathological image obtained after scanning with a digital scanner; then, the whole image and the image separated by H staining and E staining are subjected to corresponding HSV space, Lab space conversion, and optical density field conversion to obtain the mean value mean(Image_cutred_H), mean(Image_cutred_S), mean(Image_cutred_V) of each channel in the HSV space of the whole image, and the modes Image_H_most, Image_S_most, La_most of the H space and S space The mean a_Image_mean / b_Image_mean of the a / b channels in b space, the modes H_H_most and H_S_most of the H channel and S channel in the HSV space of the cell nucleus image based on the H dye, and the color deviation Color_Deviation_H in the Lab space, the modes E_H_most and E_S_most of the H channel and S channel in the HSV space of the cytoplasm image based on the E dye, and the color deviation Color_Deviation_E in the Lab space, a total of 13 features, are used as feature vectors and then input into the machine learning model for classification. The obtained machine learning model can effectively evaluate and classify the color quality of slices based on objective conditions, reduce the subjective intention of manual evaluation, and reduce the workload of manual evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 The pixel points with grayscale values ​​above 200 in the present invention are recorded as blank points in the Image_G image and the Flag_center image;

[0035] Figure 2The image is an Image_G image in which the pixel points at the corresponding position of the pathological tissue area in the present invention are set to 255, the Image_G image after the maximum connected area processing, and the image after the difference between Flag_Image and Flag_center;

[0036] Figure 3 The Image_cutwhite image after removing the blank area in the present invention;

[0037] Figure 4 The Image_cutwhite image and its corresponding hue image Image_H and saturation image Image_S in the present invention;

[0038] Figure 5 The Image_H image, the binarized image of the Image_H image, and the pathological tissue image Image_cutred after filtering out red blood cells in the present invention;

[0039] Figure 6 It is a scatter plot with the mean S saturation as the horizontal axis and the mean V brightness as the vertical axis;

[0040] Figure 7 It is the Image_cutred image in the present invention, and the Color_Deviation_H image and Color_Deviation_E image separated therefrom. DETAILED DESCRIPTION

[0041] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0042] The purpose of the present invention is to be able to objectively evaluate the quality related to the color of the slice, and to replace manual judgment of the excellent quality of the slice color, and to propose a rapid quality evaluation method for the color of pathological slices based on staining separation. The technical method of the present invention includes an image preprocessing step, a feature extraction step and a classification step. The image preprocessing step is used to extract the tissue in the slice, and remove the red blood cell area and the tissue-free area that affect the color quality in the slice tissue, and only retain the tissue area that is colored after H&E staining; the feature extraction step is used to extract features from the tissue area after preprocessing, including mean, mode, maximum value, etc.; the classification step is used to train the machine learning model with the obtained features, so as to judge the quality of the slice color.

[0043] The data of the present invention is a digital pathological image of H&E staining at 1 times magnification obtained after scanning with a digital scanner. Since the excellent discrimination results of the color quality of the slices are 5 categories, namely thick slices, poor contrast slices, red light slices, blue light slices and red and blue standard slices, these discrimination results are judged based on the colored tissue area after H&E staining (hematoxylin-eosin staining) in the slices. In addition to the effective pathological tissue area stained with H&E stain, the digital pathological image scanned by the digital scanner also includes blank areas around the tissue, blank areas inside the tissue, and red blood cells in the tissue. In order to reduce the negative impact of blank areas and red blood cells on the image feature extraction process (blank areas will cause the brightness and grayscale information of the image to be high, and red blood cells will cause the red channel value and saturation of the image to deviate), these areas need to be identified and removed.

[0044] The present invention uses the color information of the pathological image to filter out the invalid tissue area. For the white area, since the G channel in the RGB three channels is more capable of displaying the tissue area at the grayscale level, the present invention first uses the G channel of the image to filter out the blank area. The specific method is as follows:

[0045] a) Obtaining pathological images, first obtaining a sufficient number of H&E stained digital pathological images obtained by scanning with a digital scanner, and then sequentially performing steps b) to f) on the acquired pathological images;

[0046] b) Find the blank area in the center of the pathological tissue. Since the G channel in the RGB three channels of the pathological image can display the pathological tissue area at the grayscale level, the blank area is filtered out using the G channel of the pathological image, which is achieved through steps b-1) to b-3):

[0047] b-1) Obtain and process the G channel image, record the initial pathological image as Image, extract the G channel in the Image image to form the G channel image Image_G; record the pixels with gray values ​​above 200 in the Image_G image as blank points, and the remaining pixels are pathological tissue areas;

[0048] b-2). Find the blank area in the Image_G image, and find out whether there is a blank area in the center of the Image_G image with an area that meets the requirements according to the connected area characteristics;

[0049] b-3). Process the flag image, initialize a flag image Flag_center with a gray value equal to the size of Image, if there is a central blank area in step b-2), then all the pixel values ​​of the positions corresponding to the central blank area in the Flag_center image are set to 255, that is, white; if there is no central blank area in step b-2), the flag image Flag_center remains completely black;

[0050] like Figure 1 As shown, the two pictures from left to right respectively show the Image_G image and the Flag_center image in which the pixels with gray values ​​above 200 are marked as blank points.

[0051] c). Find the blank area around the pathological tissue, initialize a grayscale flag image Flag_Image with the same size as Image, find the largest connected area in Flag_Image that matches the pathological tissue area according to the connected area characteristics, and set the pixel points in Flag_Image corresponding to the largest connected area to 255, i.e. white; the area of ​​black dots in Flag_Image is the blank area around the tissue;

[0052] d) Obtaining an image of the tissue region, subtracting the Flag_Image image obtained in step c) from the Flag_center image obtained in step b), and obtaining an image of the tissue region with the central blank area and the surrounding blank area removed, recorded as Image_cutwhite;

[0053] like Figure 2 As shown in the figure, the three pictures from left to right respectively show the Image_G image with the pixel points at the corresponding position of the pathological tissue area set to 255, the Image_G image after the maximum connected area processing, and the image after the difference between Flag_Image and Flag_center. Figure 3 As shown, the two pictures respectively show the Image_cutwhite images after removing the blank areas.

[0054] The red blood cells in the slices are because blood vessels are taken during the sampling process of pathological tissues, and red blood cells exist in the blood vessels. For red blood cells, the location of red blood cells can be screened out according to the hue and saturation characteristics of red. e) Removing the location of red blood cells in the image of the tissue area is achieved through steps e-1) to e-3):

[0055] e-1). Space conversion: first, the pathological image Image_cutwhite with blank areas removed is converted from RGB space to HSV space to obtain the hue Image_H and saturation Image_S of the whole image;

[0056] e-2). Filter out the red blood cells, and then find out the location of the red blood cells in the hue image Image_H and the saturation image Image_S and filter them out according to the hue and saturation characteristics of the red blood cells, that is, the hue of the red blood cells exceeds 220 and the saturation of the red blood cells exceeds 125;

[0057] f). Obtain the image of the pathological tissue region to be analyzed, filter out the position of the red blood cells in the toned image Image_H, and then binarize the toned image Image_H with the red blood cells filtered out to obtain Image_H_bw; the position of the white dots in Image_H_bw is the tissue region, and the pixels at the corresponding position in the corresponding color image are the pathological tissue region Image_cutred to be analyzed.

[0058] like Figure 4 As shown, the three pictures from left to right respectively show the Image_cutwhite image and its corresponding hue image Image_H and saturation image Image_S, as shown in Figure 5 As shown, the three pictures from left to right respectively show the Image_H image, the binarized image of the Image_H image, and the pathological tissue image Image_cutred after filtering out red blood cells. In summary, the image preprocessing step is completed, the input is the pathological tissue image Image, and the output is the filtered pathological tissue image Image_cutred.

[0059] The feature extraction step in the present invention is used to extract the features of the pathological tissue image Image_cutred. Since the purpose of the present invention is to classify the color quality of the slices into five categories, it is necessary to find features that can represent the image color to distinguish images of each category.

[0060] A standard pathology image is an image with only one layer of cells and a sharp contrast between red and blue. Thick sections are pathology sections with more than one layer of cells; while poor contrast sections, red-light sections, and blue-light sections are classifications for bluish-purple cell nuclei and reddish-pink cytoplasm. Standard H&E-stained pathology images contain bluish-purple cell nuclei and reddish-pink cytoplasm. If the staining quality is poor, there will be weak distinction between the cell nucleus and the cytoplasm (poor contrast), overstaining, or too weak staining (red-light, blue-light), etc.

[0061] In summary, the present invention will extract features based on the color characteristics of each type of slice. For thick slices, because they contain multiple layers of cells, poor transparency, and darker colors, thick slices can be distinguished from the other four categories by using their saturation and hue characteristics. Specifically, the pathological tissue image Image_cutred is converted to the HSV space, and the mean of H hue, S saturation, and V brightness are used as the features of the thick slices, which are recorded as features mean(Image_cutred_H), mean(Image_cutred_S), and mean(Image_cutred_V), respectively. Figure 6 This is a scatter plot with the mean saturation of S as the horizontal axis and the mean brightness of V as the vertical axis. It can be seen from the figure that the slices with poor contrast, light red, light blue and red and blue standards are more concentrated, and their brightness information is distributed between 0.8-0.9, and the corresponding saturation is between 0.2-0.55, while the brightness value of thick slices is low and the saturation is high.

[0062] In view of the poor contrast, red lightness and blue lightness caused by staining quality, the present invention introduces the mode of a single color channel to reflect the color interval of most pixels in the image, and introduces the red-blue deviation as a sign to evaluate whether the color contrast between the cell nucleus and the cytoplasm is clear. The present invention calculates the mode and the red-blue deviation for the entire pathological image and the H-stained image and the E-stained image after staining separation.

[0063] The feature extraction step is specifically implemented through the following steps:

[0064] 1) Obtain the features of thick slices, convert the pathological tissue area image Image_cutred from RGB space to HSV space, and use the mean of hue H, saturation S and brightness V as the features of thick slices, which are recorded as hue mean mean(Image_cutred_H), saturation mean mean(Image_cutred_S) and brightness mean mean(Image_cutred_V), a total of 3 features;

[0065] 2). Obtain the majority feature of the entire image, convert the pathological tissue area image Image_cutred from RGB space to HSV space. Since the brightness information of the image is similar, only the hue and saturation information are taken into account; using the histogram of the grayscale image under the hue H channel and the saturation S channel, obtain the image hue H space majority Image_H_most and the saturation S space majority Image_S_most, a total of 2 features;

[0066] 3). Obtain the red-blue deviation characteristics of the entire image, and convert the image Image_cutred from the RGB space to the Lab space; Lab space is a device-independent color system, a color system based on physiological characteristics, which uses a digital method to describe human visual perception; since the poor contrast of color quality, red lightness, and blue lightness are all descriptions of human visual perception, this space is used to describe the deviation of red and blue; similarly, regardless of the brightness of the image, the mean a_Image_mean of the a channel represents the degree of redness, and the mean b_Image_mean of the b channel represents the degree of blueness. The mean a_Image_mean of the a channel and the mean b_Image_mean of the b channel are used as the two features of the red-blue deviation;

[0067] The classification of light blue and light red involved in the slice color of the present invention is based on the dye, according to the color characteristics of the cell nucleus and cytoplasm, that is, the H dye can color the cell nucleus blue-purple, and the E dye can color the cytoplasm pink. For example, light blue means that the coloring of the H dye is not obvious while the E dye is normally colored, and the overall slice color is reddish. Light red means that the coloring of the E dye is not obvious while the H dye is normally colored, and the overall slice color is blue.

[0068] 4) Obtain the color deviation characteristics of the dye separation image, which is achieved through steps 4-1) to 4-3):

[0069] 4-1) Convert to optical density value. According to the Lambert-Beer LAW shown in formula (1), the RGB color value of the pathological tissue area image Image_cutred is converted to optical density value:

[0070] OD = -log 10 (I) (1)

[0071] Where, I is the value of the RGB three channels of the pathological image normalized to the range of 0 to 1, and OD is the corresponding optical density field;

[0072] 4-2) To obtain the optical density field, firstly, singular value decomposition is performed on OD to obtain the two largest singular value vectors, and a coordinate plane is established in the direction indicated by these two vectors; then, the OD value converted by the image is projected on the newly created coordinate system to obtain two optical density fields corresponding to the H component and the E component;

[0073] 4-3). Obtain the dye separation image, and finally convert the optical density field obtained in step 4-2) back to RGB space to obtain the cell nucleus image Image_seperation_H based on the H dye and the cytoplasm image Image_seperation_E based on the E dye;

[0074] 4-4). Obtain the color deviation feature. First, convert the cell nucleus image Image_seperation_H based on H dye and the cytoplasm image Image_seperation_E based on E dye from RGB space to their respective Lab spaces. Then calculate the a channel and b channel means of the corresponding images. The a channel and b channel means of the cell nucleus image Image_seperation_H based on H dye are recorded as a_H_mean and b_H_mean, respectively. The a channel and b channel means of the cytoplasm image Image_seperation_E based on E dye are recorded as a_E_mean and b_E_mean, respectively. Finally, calculate the color deviation Color_Deviation_H and Color_Deviation_E of the cell nucleus image and cytoplasm image respectively according to formula (2):

[0075] Deviation=[(aa mean ) 2 +(bb mean ) 2 ] 1 / 2 (2)

[0076] The color deviations of the cell nucleus image and cytoplasm image, Color_Deviation_H and Color_Deviation_E, are used as two color deviation features;

[0077] 5). Obtain the majority feature of the stained separation image. To increase the distinguishability of the image, convert the nucleus image Image_seperation_H based on H dye and the cytoplasm image Image_seperation_E based on E dye from RGB space to HSV space, and calculate the modes of the H hue channel and S saturation channel of the two images in HSV space, respectively, and record them as the mode of the H channel of the nucleus image H_H_most, the mode of the S channel of the nucleus image H_S_most, the mode of the H channel of the cytoplasm image E_H_most, and the mode of the S channel of the cytoplasm image E_S_most.

[0078] In summary, in order to distinguish thick slices, light blue, light red, poor contrast and red and blue standard slices, the present invention designs a total of 13 image features, which are the means mean(Image_cutred_H) / mean(Image_cutred_S) / mean(Image_cutred_V) of each channel in the HSV space of the whole image, the modes Image_H_most / Image_S_most of the H space and the S space, the means a_Image_mean / b_Image_mean of the a / b channels in the Lab space, the modes H_H_most and H_S_most of the H and S channels in the HSV space of the cell nucleus image based on the H dye, and the color deviation Color_Deviation_H in the Lab space, the modes E_H_most / E_S_most of the H and S channels in the HSV space of the cytoplasm image based on the E dye, and the color deviation Color_Deviation_E in the Lab space.

[0079] According to the above analysis, the color quality classification problem of the slice involved in the present invention is a five-classification problem, and the machine learning model is designed for classification. The input of the model is 13 feature vectors obtained by the pathological slice after the feature extraction module, and the output is the color quality grade of the pathological slice. Through comparative analysis, the present invention uses an integrated classifier based on RUSBoost, and its maximum split number is 20, the number of learners is 30, the learning rate is 0.021, the optimizer is a random search, and the number of iterations is 30 times. After training, the machine learning model accuracy obtained is 81.3%, which meets the use requirements.

Claims

1. A color-related quality assessment method for pathological sections based on staining separation, comprising an image preprocessing step, a feature extraction step and a classification step, characterized in that: The image preprocessing step includes: firstly extracting the tissue area in the pathological slice image, then removing the red blood cell area and the tissue-free area that affect the color quality in the tissue area, and only retaining the tissue area colored after H&E staining; the feature extraction step is to extract the features of the tissue area after preprocessing, and perform the corresponding HSV space, Lab space conversion, and optical density field conversion on the whole image and the image separated by H staining and E staining to obtain the mean of each channel of the HSV space of the whole image, mean(Image_cutred_H), mean(Image_cutred_S), mean(Image_cutred_V), the modes of H space and S space Image_H_most, Image_S_most, and the a / b channel of Lab space The mean a_Image_mean / b_Image_mean of the channel, the modes H_H_most and H_S_most of the H channel and S channel in the HSV space of the cell nucleus image based on the H dye, and the color deviation Color_Deviation_H in the Lab space, the modes E_H_most and E_S_most of the H channel and S channel in the HSV space of the cytoplasm image based on the E dye, and the color deviation Color_Deviation_E in the Lab space, a total of 13 features, are used as feature vectors and then input into the machine learning model for classification. The obtained machine learning model can effectively evaluate and classify the color quality of the slices based on objective conditions; the classification step is used to train the machine learning model with the obtained features, so as to judge the quality of the slice color.

2. The method for color-related quality assessment of pathological sections based on staining separation according to claim 1, characterized in that: The image preprocessing step is specifically implemented by the following steps: a) Obtaining pathological images, first obtaining a sufficient number of H&E stained digital pathological images obtained by scanning with a digital scanner, and then sequentially performing steps b) to f) on the acquired pathological images; b) Find the blank area in the center of the pathological tissue. Since the G channel in the RGB three channels of the pathological image can display the pathological tissue area at the grayscale level, the blank area is filtered out using the G channel of the pathological image, which is achieved through steps b-1) to b-3): b-1) Obtain and process the G channel image, record the initial pathological image as Image, extract the G channel in the Image image to form the G channel image Image_G; record the pixels with gray values ​​above 200 in the Image_G image as blank points, and the remaining pixels are pathological tissue areas; b-2). Find the blank area in the Image_G image, and find out whether there is a blank area in the center of the Image_G image with an area that meets the requirements according to the connected area characteristics; b-3). Process the flag image, initialize a flag image Flag_center with a gray value equal to the size of Image, if there is a central blank area in step b-2), then all the pixel values ​​of the positions corresponding to the central blank area in the Flag_center image are set to 255, that is, white; if there is no central blank area in step b-2), the flag image Flag_center remains completely black; c). Find the blank area around the pathological tissue, initialize a grayscale flag image Flag_Image with the same size as Image, find the largest connected area in Flag_Image that matches the pathological tissue area according to the connected area characteristics, and set the pixel points in Flag_Image corresponding to the largest connected area to 255, i.e. white; the area of ​​black dots in Flag_Image is the blank area around the tissue; d) Obtaining an image of the tissue region, subtracting the Flag_Image image obtained in step c) from the Flag_center image obtained in step b), and obtaining an image of the tissue region with the central blank area and the surrounding blank area removed, recorded as Image_cutwhite; e) Removing the location of red blood cells in the image of the tissue area is achieved through steps e-1) to e-3): e-1). Space conversion: first, the pathological image Image_cutwhite with blank areas removed is converted from RGB space to HSV space to obtain the hue Image_H and saturation Image_S of the whole image; e-2). Filter out the red blood cells, and then find out the location of the red blood cells in the hue image Image_H and the saturation image Image_S and filter them out according to the hue and saturation characteristics of the red blood cells, that is, the hue of the red blood cells exceeds 220 and the saturation of the red blood cells exceeds 125; f). Obtain the image of the pathological tissue region to be analyzed, filter out the position of the red blood cells in the toned image Image_H, and then binarize the toned image Image_H with the red blood cells filtered out to obtain Image_H_bw; the position of the white dots in Image_H_bw is the tissue region, and the pixels at the corresponding position in the corresponding color image are the pathological tissue region Image_cutred to be analyzed.

3. The method for color-related quality assessment of pathological sections based on staining separation according to claim 2, characterized in that: The feature extraction step is specifically implemented by the following steps: 1) Obtain the features of thick slices, convert the pathological tissue area image Image_cutred from RGB space to HSV space, and use the mean of hue H, saturation S and brightness V as the features of thick slices, which are recorded as hue mean mean(Image_cutred_H), saturation mean mean(Image_cutred_S) and brightness mean mean(Image_cutred_V), a total of 3 features; 2). Obtain the majority feature of the entire image, convert the pathological tissue area image Image_cutred from RGB space to HSV space. Since the brightness information of the image is similar, only the hue and saturation information are taken into account; using the histogram of the grayscale image under the hue H channel and the saturation S channel, obtain the image hue H space majority Image_H_most and the saturation S space majority Image_S_most, a total of 2 features; 3). Obtain the red-blue deviation characteristics of the entire image, and convert the image Image_cutred from the RGB space to the Lab space; Lab space is a device-independent color system, a color system based on physiological characteristics, which uses a digital method to describe human visual perception; since the poor contrast of color quality, red lightness, and blue lightness are all descriptions of human visual perception, this space is used to describe the deviation of red and blue; similarly, regardless of the brightness of the image, the mean a_Image_mean of the a channel represents the degree of redness, and the mean b_Image_mean of the b channel represents the degree of blueness. The mean a_Image_mean of the a channel and the mean b_Image_mean of the b channel are used as the two features of the red-blue deviation; 4) Obtain the color deviation characteristics of the dye separation image, which is achieved through steps 4-1) to 4-3): 4-1) Convert to optical density value. According to the Lambert-Beer LAW shown in formula (1), the RGB color value of the pathological tissue area image Image_cutred is converted to optical density value: OD=﹣log 10 (I) (1) Where, I is the value of the RGB three channels of the pathological image normalized to the range of 0 to 1, and OD is the corresponding optical density field; 4-2) To obtain the optical density field, firstly, singular value decomposition is performed on OD to obtain the two largest singular value vectors, and a coordinate plane is established in the direction indicated by these two vectors; then, the OD value converted by the image is projected on the newly created coordinate system to obtain two optical density fields corresponding to the H component and the E component; 4-3). Obtain the dye separation image, and finally convert the optical density field obtained in step 4-2) back to RGB space to obtain the cell nucleus image Image_seperation_H based on the H dye and the cytoplasm image Image_seperation_E based on the E dye; 4-4). Obtain the color deviation feature. First, convert the cell nucleus image Image_seperation_H based on H dye and the cytoplasm image Image_seperation_E based on E dye from RGB space to their respective Lab spaces. Then calculate the a channel and b channel means of the corresponding images. The a channel and b channel means of the cell nucleus image Image_seperation_H based on H dye are recorded as a_H_mean and b_H_mean, respectively. The a channel and b channel means of the cytoplasm image Image_seperation_E based on E dye are recorded as a_E_mean and b_E_mean, respectively. Finally, calculate the color deviation Color_Deviation_H and Color_Deviation_E of the cell nucleus image and cytoplasm image respectively according to formula (2): Deviation=[(a-a mean ) 2 +(b-b mean ) 2 ] 1 / 2 (2) The color deviations of the cell nucleus image and cytoplasm image, Color_Deviation_H and Color_Deviation_E, are used as two color deviation features; 5). Obtain the majority feature of the stained separation image. To increase the distinguishability of the image, convert the nucleus image Image_seperation_H based on H dye and the cytoplasm image Image_seperation_E based on E dye from RGB space to HSV space, and calculate the modes of the H hue channel and S saturation channel of the two images in HSV space, respectively, and record them as the mode of the H channel of the nucleus image H_H_most, the mode of the S channel of the nucleus image H_S_most, the mode of the H channel of the cytoplasm image E_H_most, and the mode of the S channel of the cytoplasm image E_S_most.

4. The method for color-related quality assessment of pathological sections based on staining separation according to claim 3, characterized in that: The implementation method of the classification step is: after processing a sufficient number of digital pathology images through steps b) to f) and steps 1) to 5), a part of them is selected as a training set, a part as a test set, and the rest as a verification set, and the images in the test set are manually labeled with five classification labels, namely thick slices, blue light, red light, contrast difference, and red and blue standard slices, and then classified by a machine learning model. The input of the machine learning model is 13 feature vectors obtained after the pathology slice passes through the feature extraction module, and the output is the color quality grade of the pathology slice.

Images