Breast cancer histological grading method used in breast tissue pathological image

Abstract

The invention relates to the technical field of digital pathological intelligent diagnosis, and provides a breast cancer histological grading method for a breast tissue pathological image. E-dyed mammary gland pathology pictures are used for respectively carrying out cell nucleus irregularity, glandular canal formation and scoring on cancer whole lesion images through a tissue region segmentation model, a core region classification model, a cancer region segmentation model, a cancer whole lesion model, a salient region detection model, a cell nucleus irregularity scoring model, a glandular canal formation scoring model and a nuclear division image counting model in sequence; the method can automatically identify and segment cancer cell nucleuses from the breast tissue pathology image, extract tissue structure features, and carry out accurate histological grading on breast cancer.

Description

Technical Field

[0001] The present invention relates to the technical field of digital pathology intelligent diagnosis, and in particular to a breast cancer histological grading method used in breast tissue pathology images. Background Art

[0002] Breast cancer is one of the most common and lethal malignant tumors in women worldwide. Early diagnosis and precise treatment are of great significance for improving the survival rate and quality of life of patients. In the diagnosis and treatment of breast cancer, histological grading is one of the key steps in evaluating the biological behavior of tumors and the prognosis of patients. At present, the Nottingham histological grading system is widely used internationally to grade and evaluate invasive breast cancer. This system provides a relatively objective and standardized grading system through a semi-quantitative comprehensive evaluation of three key parameters: glandular duct formation ratio, nuclear pleomorphism, and mitotic figure count.

[0003] However, the Nottingham histological grading system is highly dependent on the subjective judgment of pathologists and has a heavy workload. Therefore, the present invention proposes a breast cancer histological grading algorithm for breast tissue pathology images to achieve automated, precise and efficient breast cancer histological grading, providing a scientific basis and technical support for clinical diagnosis and treatment. Summary of the invention

[0004] The present invention proposes a method for histological grading of breast cancer in breast tissue pathology images. The method is based on computer vision and deep learning technology, and can automatically identify and segment cancer cell nuclei from breast tissue pathology images, extract tissue structure features, and accurately grade breast cancer.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] The present invention provides a breast cancer histological grading method for breast tissue pathological images, comprising:

[0007] (1) Using the tissue region segmentation model, the breast tissue region is located and segmented from the complete H&E stained breast pathology image;

[0008] (2) Using the core area classification model to remove fragmented tissue structures in the breast tissue area, and obtain the core area of ​​the breast tissue;

[0009] (3) Using the cancer region segmentation model, the cancer region in the core area of ​​the breast tissue is located and segmented;

[0010] (4) removing non-cancer areas and benign lesion areas in the cancer area through the overall cancer lesion model to obtain the malignant cancer area;

[0011] (5) Detecting the salient regions in the malignant cancer region through the salient region detection model, including rectangular salient regions and circular salient regions;

[0012] (6) Classifying the nuclear atypia of each rectangular significant area using the nuclear atypia scoring model, and averaging the scores of all rectangular significant areas as the final “nuclear atypia” score;

[0013] (7) Using the duct formation scoring model, each rectangular significant area is classified as "duct formation", and the average of all rectangular significant areas is taken as the final "duct formation" score;

[0014] (8) The number of cells with segmented mitotic figures is calculated based on the mitotic figure counting model within the circular significant area.

[0015] Furthermore, the tissue region segmentation model is a UNet model with an EfficientNet network structure as the backbone network;

[0016] The training process of the tissue region segmentation model is:

[0017] Data preprocessing: Assume that the original size of the breast tissue pathology image is H*W, and the mpp value is a.

[0018] The breast tissue pathology image is adaptively scaled to b, and the scaled size is recorded as scale H *scale W , where scale H =H*a / b, scale W =W*a / b;

[0019] Data annotation: H&E stained breast pathology images were used as training samples and test samples, and the content of each H&E stained breast pathology image was annotated separately, with the annotated breast tissue area recorded as 1 and the rest recorded as 0; specifically: 1,000 H&E stained breast pathology samples were used as training, and another H&E stained breast pathology sample was used as test;

[0020] Model training: Use cross entropy loss and Adam optimizer to train the tissue region segmentation model and perform multiple iterations;

[0021] Model testing: The segmentation results are verified through manually labeled data to ensure the accuracy of the scoring.

[0022] The specific steps of using the tissue region segmentation model are:

[0023] (11) The complete H&E stained breast pathology image is input into the tissue region segmentation model, and the image is adaptively scaled to b according to the mpp value of the H&E stained breast pathology image. The scaled size is recorded as scale H *scale W ; Among them, scale H =H*a / b, scale W =W*a / b, H and W are the original length and height of H&E stained breast pathology images, a is the mpp value;

[0024] (12) The tissue region segmentation model marks the breast tissue region in the scaled breast pathology image as 1, and the rest of the background position as 0; extracts the outer contour of the connected region where each pixel is marked as 1 and calls it a target;

[0025] (13) Filter out the targets with breast tissue area less than 500pix^2, and output the connected areas where all remaining pixels are marked as 1, i.e., the breast tissue area.

[0026] Furthermore, the core area classification model is a convolutional neural network (CNN) classification model based on the MobileNetV3 network structure, and its output channel is 2, which respectively represents the probability scores of "fragmented organizational structure" and "core area";

[0027] The specific steps for using the core area classification model are:

[0028] (21) Each breast tissue region is uniformly cut into blocks according to its extracted outer contour as the center. Specifically, the coordinates of the center point of the outer contour are recorded as (x c ,y c ), the height and width of the circumscribed rectangle of the outer contour are H 轮廓 , W 轮廓 The center of the outer contour is the height and width of the crop. H ,crop W , where crop H =H 轮廓 *1.25,crop W =W 轮廓 *1.25, the coordinates of the center point of the outer contour (x c ,y c ) as the center to cut the height and width of the crop H ,crop W The cut rectangular area is then scaled to a size of m*m to obtain a first preprocessed image, where m is a multiple of 2, and 1024, 512, 384, or 224 is generally preferred;

[0029] (22) Sending the first preprocessed image to the core area classification model to obtain the probability scores of the image being a “fragmented tissue structure” and a “core area”; according to the probability scores, if the probability score of the “core area” is higher than the probability score of the “fragmented tissue structure”, the output category is “core area”; otherwise, the output is “fragmented tissue structure”;

[0030] (23) The “core area” is retained as the classification result, and the outer contour information extracted from the tissue region segmentation model is obtained again, and its outer contour information is used as the outer contour of the core area to obtain the core area of ​​the breast tissue.

[0031] The training process is:

[0032] Data preprocessing: Each breast tissue area is uniformly cut according to its extracted outer contour as the center; specifically, the center point of the contour is denoted as center, and the specific coordinates are denoted as (x c ,y c ), the height and width of the circumscribed rectangle of the outline are H 轮廓 , W 轮廓 , remember the center of the contour and the height and width of the cut as crop H ,crop W Among them, crop H =H 轮廓 *1.25,crop W =W 轮廓 *1.25; the coordinates of the center point of the contour (x c ,y c ) as the center to cut the height and width of the crop H ,crop W The cut-out area is then scaled to a size of m*m to obtain a first preprocessed image; wherein m is a multiple of 2, and is generally preferably 1024, 512, 384, or 224;

[0033] Data labeling: The first preprocessed image is used as a training sample and a test sample. If the data is a fragmented tissue structure, it is marked as 0, and if the data is a truly valid breast tissue, it is marked as 1. Specifically, 2000 targets are used for training, and another 400 targets are used for testing.

[0034] Model training: Use cross entropy loss and Adam optimizer to train the model and iterate multiple times;

[0035] Model testing: Verify the classification results through test samples to ensure the accuracy of the scoring.

[0036] Furthermore, the cancer region segmentation model is a UNet model with an EfficientNet network structure as the backbone network, and has 3 output channels, which respectively represent the probability scores of “background”, “cancer region”, and “cancer region”;

[0037] The training steps are:

[0038] Data preprocessing: The core area of ​​the breast tissue is cut into pieces according to the circumscribed rectangle of the outline, and the pixel value of the area outside the outline is set to 0. The image is adaptively scaled to b according to the mpp value of the core area of ​​the breast tissue to obtain the second preprocessed image. The scaled size is recorded as scale H核心 *scale W核心 ; Among them, scale H核心 =H 核心 *a2 / b, scale W核心 =W 核心 *a2 / b, H 核心 and W 核心 is the original length and height of the core area of ​​breast tissue, a2 is the mpp value of the core area of ​​breast tissue;

[0039] Data labeling: The second pre-processed images were used as training samples and test samples, and the cancer area in the image was labeled as 1, and the rest of the positions were labeled as 0. Specifically, 500 second pre-processed images were used for training, and another 100 second pre-processed images were used for testing.

[0040] Model training: Cross entropy loss and Adam optimizer were used simultaneously for model training, and the training was repeated multiple times.

[0041] The specific steps of using the cancer region segmentation model are:

[0042] (31) Cut the breast tissue core area into pieces according to the circumscribed rectangle of the outline, set the pixel value of the area outside the outline to 0, and adaptively scale it to b according to the mpp value of the breast tissue core area to obtain the second preprocessed image. The scaled size is recorded as scale H核心 *scale W核心 ; Among them, scale H核心 =H 核心 *a2 / b, scale W核心 =W 核心 *a2 / b, H 核心 and W 核心 is the original length and height of the core area of ​​breast tissue, a2 is the mpp value of the core area of ​​breast tissue;

[0043] (32) Inputting the second preprocessed image into the cancer region segmentation model, obtaining the probability score of each pixel in the region being "background", "cancer region", or "normal tissue region", and according to the probability score, if the probability score of the pixel being "background" is the highest, then the pixel is marked as 0; if the probability score of the pixel being "cancer region" is the highest, then the pixel is marked as 1; if the probability score of the pixel being "normal tissue region" is the highest, then the pixel is marked as 2;

[0044] (33) The extracted “normal tissue region” is adjusted using the region growing technique, and the adjacent region images are fused to output the final normal tissue, and the pixels in the region at this position are recorded as 2;

[0045] (34) Each pixel is finally marked as a connected area of ​​1, its outer contour is extracted and called a target. After filtering the targets with an area less than 500pix^2, the remaining targets are output as the final cancer areas.

[0046] Furthermore, the overall cancer lesion model is a convolutional neural network (CNN) classification model with MobileNetV3 as the main backbone, and the output channels of the classification are 3, which respectively represent the probability scores of "non-cancer block", "benign lesion" and "malignant cancer area";

[0047] The training steps are:

[0048] Data preprocessing: The cancer area is uniformly cut into blocks according to its extracted outer contour as the center. Specifically, the coordinates of the center point of the outer contour are recorded as (x c癌 ,y c癌 ), the height and width of the circumscribed rectangle of the outer contour are H 癌轮廓 , W 癌轮廓 The center of the outer contour is the height and width of the crop. H癌 ,crop W癌 , where crop H癌 =H 癌轮廓 *1.25,crop W癌 =W 癌轮廓 *1.25, the coordinates of the center point of the outer contour (x c癌 ,y c癌 ) as the center to cut the height and width of the crop H癌 ,crop W癌 The cut rectangular area is then scaled to a size of m*m to obtain a third preprocessed image, where m is a multiple of 2, and 1024, 512, 384, or 224 is generally preferred;

[0049] Data labeling: The third pre-processed images are used as training samples and test samples. If the data is a "non-cancerous block", it is marked as 0; if the data is a "benign lesion", it is marked as 1; if the data is a "malignant cancer area", it is marked as 2. Specifically, 2000 third pre-processed images are used for training, and another 400 third pre-processed images are used for testing;

[0050] Model training: Cross entropy loss and Adam optimizer were used to train the model and repeated multiple times.

[0051] Model testing: Verify the classification results through test samples to ensure the accuracy of the scoring.

[0052] The specific steps of using the overall cancer lesion model are as follows:

[0053] (41) Each cancer region is uniformly cut into blocks according to its extracted outer contour as the center. Specifically, the coordinates of the center point of the outer contour are recorded as (x c癌 ,y c癌 ), the height and width of the circumscribed rectangle of the outer contour are H 癌轮廓 , W 癌轮廓 The center of the outer contour is the height and width of the crop. H癌 ,crop W癌 , where crop H癌 =H 癌轮廓 *1.25,crop W癌 =W 癌轮廓 *1.25, the coordinates of the center point of the outer contour (x c癌 ,y c癌 ) as the center to cut the height and width of the crop H癌 ,crop W癌 The cut rectangular area is then scaled to a size of m*m to obtain a third preprocessed image, where m is a multiple of 2, and 1024, 512, 384, or 224 is generally preferred;

[0054] (42) inputting the third preprocessed image into the overall cancer lesion model, obtaining the probability score of each pixel point in the third preprocessed image being a "non-cancer block", "benign lesion" and "malignant cancer area", and according to the score, if the "non-cancer block" has the highest score, the category output is "non-cancer block", if the "benign lesion" has the highest score, the category output is "benign lesion", and if the "malignant cancer area" has the highest score, the category output is "malignant cancer area";

[0055] (43) The malignant cancer region is retained, and the outer contour information obtained in the cancer region segmentation model is re-acquired and used as the outer contour of the malignant cancer region.

[0056] Furthermore, the salient region detection model uses the UNet model with EfficientNet as the backbone network. Its training process is consistent with the training process of the cancer region segmentation model.

[0057] The specific steps of using the salient region detection model are as follows:

[0058] (51) The malignant cancer area is cut into pieces according to the circumscribed rectangle of its outer contour, and the pixel values ​​of the area outside the contour are set to 0. The image is adaptively scaled to b according to the mpp value of the malignant cancer area to obtain the fourth preprocessed image. The scaled size is recorded as scale H恶性 *scale W恶性 ; Among them, scale H恶性 =H 恶性 *a3 / b, scale W恶性 =W 恶性 *a3 / b, H 恶性 and W 恶性 is the original length and height of the malignant cancer area, a3 is the mpp value of the malignant cancer area;

[0059] (52) Inputting the fourth preprocessed image into the salient region detection model, obtaining a probability score of each pixel in the fourth preprocessed image being a "background", a "cancer region", or a "normal tissue region", and according to the score, taking the pixel with the highest probability score of being a "cancer region" as a salient feature map, wherein the value range of each pixel is between 0 and 1, and the closer to 1, the more likely it is cancer;

[0060] (53) The salient feature map is significantly stretched to obtain a salient feature stretch map. The specific operation is: the feature value at the pixel position (i, j) is set to x ij , the eigenvalue after stretching is recorded as y ij , then y ij =x ij 2 ; The obtained graph is called the significant feature stretching graph;

[0061] (54) Obtain the coordinate point position (imax, jmax) with the highest probability score of the "cancer region" in the significant feature stretching map, as well as its corresponding eigenvalue y imax,jmax If y imax,jmax >p, then jump to step (55), otherwise jump to step (57); wherein p is preferably 0.85;

[0062] (55) Take the coordinate point position (imax, jmax) as the significant center point position, and take a rectangle with a size of R1*R1 with it as the center as the rectangular significant area; take a circle with a radius of R2 with it as the center as the circular significant area; if the number of rectangular significant areas and circular significant areas reaches q, jump to step (57), otherwise enter step (56) to update the significant feature stretching map; wherein R1>R2, q is preferably 20;

[0063] (56) Taking the obtained coordinate point position (imax, jmax) as the center and R3 as the radius, the feature value on the significant feature stretching map within the range is set to 0; then return to step (54); wherein R3 <R2,

[0064] (57) Output rectangular salient areas and circular salient areas.

[0065] Furthermore, the nuclear atypia scoring model uses MobileNetV3 as the main convolutional neural network (CNN) classification model, and the classification output channel is 3, representing the probability score values ​​of "low atypia", "medium atypia" and "high atypia" respectively;

[0066] The training steps are:

[0067] Data preprocessing: Each rectangular salient area is uniformly cut into blocks according to its circumscribed rectangle, and then the cut area is scaled to a size of m*m, where m is a multiple of 2, and 1024, 512, 384, and 224 are generally preferred;

[0068] Data labeling: The scaled images were used as training samples and test samples. If the data was "low heterogeneity", it was marked as 0, if the data was "medium heterogeneity", it was marked as 1, and if the data was "high heterogeneity", it was marked as 2. Specifically, 2,000 scaled images were used for training and another 400 scaled images were used for testing.

[0069] Model training: Cross entropy loss and Adam optimizer were used simultaneously for model training, and the training was repeated multiple times.

[0070] Model testing: Verify the classification results through test samples to ensure the accuracy of the scoring.

[0071] The specific steps of using the nuclear atypia scoring model are as follows:

[0072] (61) Cut the rectangular salient area into blocks according to its circumscribed rectangle, and then scale the cut area to a size of m*m, where m is a multiple of 2, and is generally preferably 1024, 512, 384, or 224;

[0073] (62) Input the scaled image into the nuclear atypia scoring model to obtain the probability scores of the image being "low atypia", "medium atypia" and "high atypia"; the category with the highest score is the final category of the rectangular significant area. In addition, if the rectangular significant area is judged to be "low atypia", the score is 0; if the rectangular significant area is judged to be "medium atypia", the score is 1; if the rectangular significant area is judged to be "high atypia", the score is 2;

[0074] (63) The average of all “rectangular significant area” scores was taken as the final score at the image level, and the final cell atypia score in the range of [0, 2 / 3) was set to 1, the final cell atypia score in the range of [2 / 3, 4 / 3] was set to 2, and the final cell atypia score in the range of [4 / 3, 2] was set to 3.

[0075] Furthermore, the duct formation scoring model is a convolutional neural network (CNN) classification model with MobileNetV3 as the main trunk, and the output channels of the classification are 3, which respectively represent the probability scores of "high duct formation", "medium duct formation" and "low duct formation";

[0076] The training steps are as follows:

[0077] Data preprocessing: The rectangular salient area is uniformly cut into blocks according to its circumscribed rectangle, and then the cut area is scaled to m*m size, where m is a multiple of 2, generally 1024, 512, 384, 224 are preferred;

[0078] Data labeling: The scaled images were used as training samples and test samples. If the data was "high duct formation", it was marked as 0, if the data was "medium duct formation", it was marked as 1, and if the data was "low duct formation", it was marked as 2. Specifically, 2,000 scaled images were used for training, and another 400 scaled images were used for testing.

[0079] Model training: Cross entropy loss and Adam optimizer were used to train the model and repeated multiple times.

[0080] Model testing: Verify the classification results through test samples to ensure the accuracy of the scoring.

[0081] The specific steps of using the duct formation scoring model are as follows:

[0082] (71) Cut the rectangular salient area into blocks according to its circumscribed rectangle, and then scale the cut area to a size of m*m, where m is a multiple of 2, and is generally preferably 1024, 512, 384, or 224;

[0083] (72) Input the zoomed image into the nuclear atypia scoring model to obtain the probability scores of the image being "high duct formation", "medium duct formation" and "low duct formation". The category with the highest score is the final category of the rectangular significant area. In addition, if the rectangular significant area is judged to be "high duct formation", the score is 0; if the rectangular significant area is judged to be "medium duct formation", the score is 1; if the rectangular significant area is judged to be "low duct formation", the score is 2;

[0084] (73) The average of all “rectangular salient area” scores is taken as the final score at the image level, and the final duct formation score in the range of [0, 2 / 3) is taken as 1 point, the final duct formation score in the range of [2 / 3, 4 / 3] is taken as 2 points, and the final duct formation score in the range of [4 / 3, 2] is taken as 3 points.

[0085] Furthermore, the mitotic figure counting model is a UNet segmentation model with EfficientNet as the backbone network, and the output channel of the model is 2, representing the probability scores of "background" and "mitotic figures" respectively;

[0086] The training steps are:

[0087] Data preprocessing: The circular salient area is cut into pieces according to the circumscribed rectangle of the outer contour, and the pixel value of the area outside the contour is set to 0. The circular salient area is adaptively scaled to b according to the mpp value of the circular salient area to obtain the fifth preprocessed image. The scaled size is recorded as scale H圆形 *scale W圆形 ; Among them, scale H圆形 =H 圆形 *a4 / b, scale W圆形 =W 圆形 *a4 / b, H 圆形 and W 圆形 is the original length and height of the malignant cancer area, a4 is the mpp value of the malignant cancer area;

[0088] Data labeling: The fifth preprocessed image is used as a training sample and a test sample, and the position of the nuclear division image in the image is labeled as 1, and the other positions are labeled as 0; specifically, 500 targets are used for training, and the other 100 targets are used for testing;

[0089] Model training: Cross entropy loss and Adam optimizer were used to train the model and repeated multiple times.

[0090] Model testing: Verify the segmentation results through the test set data to ensure the accuracy of the scoring.

[0091] The specific steps for using the mitotic figure counting model are as follows:

[0092] (81) The circular salient area is cut into pieces according to the circumscribed rectangle of the outer contour, and the pixel values ​​of the area outside the outer contour are set to 0. The circular salient area is adaptively scaled to b according to the mpp value of the circular salient area to obtain the fifth preprocessed image. The scaled size is recorded as scale H圆形 *scale W圆形 ; Among them, scale H圆形 =H 圆形 *a4 / b, scale W圆形 =W 圆形 *a4 / b, H 圆形 and W 圆形 is the original length and height of the malignant cancer area, a4 is the mpp value of the malignant cancer area;

[0093] (82) Input the fifth preprocessed image into the nuclear division image counting model, and output the probability scores of the pixels in the image being "background" and "nuclear division image"; if the probability score of the pixel being "background" is the highest, the pixel is marked as 0; if the probability score of the pixel being "nuclear division image" is the highest, the pixel is marked as 1;

[0094] (83) The connected regions with pixels marked as 1 are subjected to outer contour extraction, and each connected region is output as a “nuclear fission image” target;

[0095] (84) If there are less than 5 “mitotic figures” in each “circular prominent area”, it is scored as 1 point; if there are 6 to 10 “mitotic figures”, it is scored as 2 points; if there are more than 11 “mitotic figures”, it is scored as 3 points;

[0096] (85) The maximum value of the probability score of nuclear division images in each "circular significant area" is taken as the nuclear division image score of the entire image.

[0097] The present invention has the following beneficial effects:

[0098] (1) The present invention provides an efficient and accurate breast cancer histological grading algorithm, and uses computer objective and quantitative technical means to eliminate subjective judgments among different doctors for breast grading; at the same time, it can provide a technical reference for doctors who are new to the field of pathological detection.

[0099] (2) The present invention makes full use of advanced deep learning technology and eliminates the detection bias of a single model through the cascade of multiple models, making the model monitoring effect better. It provides strong support and guarantee for the clinical diagnosis and treatment of breast cancer. BRIEF DESCRIPTION OF THE DRAWINGS

[0100] Figure 1 It is the overall flow chart of the present invention.

[0101] Figure 2 It is a schematic diagram of the breast tissue region output by the tissue region segmentation model in the present invention.

[0102] Figure 3 It is a schematic diagram of the “background”, “cancer region” and “normal tissue region” output by the cancer region segmentation model in the present invention.

[0103] Figure 4 The salient region detection model in the present invention outputs a rectangular salient region and a circular salient region.

[0104] Figure 5 It is a schematic diagram of nuclear division images detected by the nuclear division image counting model in the present invention. DETAILED DESCRIPTION

[0105] The specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings. It should be pointed out that the embodiments are only specific explanations of the invention and should not be regarded as limitations of the invention. The purpose of the embodiments is to enable those skilled in the art to better understand and reproduce the technical solutions of the present invention. The protection scope of the present invention shall still be based on the scope defined by the claims.

[0106] like Figure 1 As shown, the present invention provides a breast cancer histological grading method for breast tissue pathology images, comprising:

[0107] (1) The breast tissue region is located and segmented from the complete H&E stained breast pathology image using the tissue region segmentation model. Figure 2 As shown;

[0108] The tissue region segmentation model is a UNet model with an EfficientNet network structure as the backbone network;

[0109] The training process of the tissue region segmentation model is:

[0110] Data preprocessing: Assume that the original size of the breast tissue pathology image is H*W, and the mpp value is a.

[0111] The breast tissue pathology image is adaptively scaled to b, and the scaled size is recorded as scale H *scale W , where scale H =H*a / b, scale W =W*a / b;

[0112] Data annotation: H&E stained breast pathology images were used as training samples and test samples, and the content of each H&E stained breast pathology image was annotated separately, with the annotated breast tissue area recorded as 1 and the rest recorded as 0; specifically: 1,000 H&E stained breast pathology samples were used as training, and another H&E stained breast pathology sample was used as test;

[0113] Model training: Use cross entropy loss and Adam optimizer to train the tissue region segmentation model and perform multiple iterations;

[0114] Model testing: The segmentation results are verified through manually labeled data to ensure the accuracy of the scoring.

[0115] The specific steps of using the tissue region segmentation model are:

[0116] (11) The complete H&E stained breast pathology image is input into the tissue region segmentation model, and the image is adaptively scaled to b according to the mpp value of the H&E stained breast pathology image. The scaled size is recorded as scale H *scale W ; Among them, scale H =H*a / b, scale W =W*a / b, H and W are the original length and height of H&E stained breast pathology images, a is the mpp value;

[0117] (12) The tissue region segmentation model marks the breast tissue region in the scaled breast pathology image as 1, and the rest of the background position as 0; extracts the outer contour of the connected region where each pixel is marked as 1 and calls it a target;

[0118] (13) Filter out the targets with breast tissue area less than 500pix^2, and output the connected areas where all remaining pixels are marked as 1, i.e., the breast tissue area.

[0119] Furthermore, the core area classification model is a convolutional neural network (CNN) classification model based on the MobileNetV3 network structure, and its output channel is 2, which respectively represents the probability scores of "fragmented organizational structure" and "core area";

[0120] (2) Using the core area classification model to remove fragmented tissue structures in the breast tissue area, and obtain the core area of ​​the breast tissue;

[0121] The specific steps for using the core area classification model are:

[0122] (21) Each breast tissue region is uniformly cut into blocks according to its extracted outer contour as the center. Specifically, the coordinates of the center point of the outer contour are recorded as (x c,y c ), the height and width of the circumscribed rectangle of the outer contour are H 轮廓 , W 轮廓 The center of the outer contour is the height and width of the crop. H ,crop W , where crop H =H 轮廓 *1.25,crop W =W 轮廓 *1.25, the coordinates of the center point of the outer contour (x c ,y c ) as the center to cut the height and width of the crop H ,crop W The cut rectangular area is then scaled to a size of 512*512 to obtain a first preprocessed image;

[0123] (22) Sending the first preprocessed image to the core area classification model to obtain the probability scores of the image being a “fragmented tissue structure” and a “core area”; according to the probability scores, if the probability score of the “core area” is higher than the probability score of the “fragmented tissue structure”, the output category is “core area”; otherwise, the output is “fragmented tissue structure”;

[0124] (23) The “core area” is retained as the classification result, and the outer contour information extracted from the tissue region segmentation model is obtained again, and its outer contour information is used as the outer contour of the core area to obtain the core area of ​​the breast tissue.

[0125] The training process is:

[0126] Data preprocessing: Each breast tissue area is uniformly cut according to its extracted outer contour as the center; specifically, the center point of the contour is denoted as center, and the specific coordinates are denoted as (x c ,y c ), the height and width of the circumscribed rectangle of the outline are H 轮廓 , W 轮廓 , remember the center of the contour and the height and width of the cut as crop H ,crop W Among them, crop H =H 轮廓 *1.25,crop W =W 轮廓 *1.25; the coordinates of the center point of the contour (x c ,y c ) as the center to cut the height and width of the crop H ,crop WThe cut-out area is then scaled to a size of 512*512 to obtain a first preprocessed image;

[0127] Data labeling: The first preprocessed image is used as a training sample and a test sample. If the data is a fragmented tissue structure, it is marked as 0, and if the data is a truly valid breast tissue, it is marked as 1. Specifically, 2000 targets are used for training, and another 400 targets are used for testing.

[0128] Model training: Use cross entropy loss and Adam optimizer to train the model and iterate multiple times;

[0129] Model testing: Verify the classification results through test samples to ensure the accuracy of the scoring.

[0130] (3) Using the cancer region segmentation model, the cancer region in the core area of ​​the breast tissue is located and segmented;

[0131] The cancer region segmentation model is a UNet model with the EfficientNet network structure as the backbone network, and its output channels are 3, representing the probability scores of "background", "cancer region" and "normal tissue region" respectively; Figure 2 As shown, the red area is the unlocated cancer area, the green area is the normal area, and the other parts are the background;

[0132] The training steps are:

[0133] Data preprocessing: The core area of ​​the breast tissue is cut into pieces according to the circumscribed rectangle of the outline, and the pixel value of the area outside the outline is set to 0. The image is adaptively scaled to b according to the mpp value of the core area of ​​the breast tissue to obtain the second preprocessed image. The scaled size is recorded as scale H核心 *scale W核心 ; Among them, scale H核心 =H 核心 *a2 / b, scale W核心 =W 核心 *a2 / b, H 核心 and W 核心 is the original length and height of the core area of ​​breast tissue, a2 is the mpp value of the core area of ​​breast tissue;

[0134] Data labeling: The second pre-processed images were used as training samples and test samples, and the cancer area in the image was labeled as 1, and the rest of the positions were labeled as 0. Specifically, 500 second pre-processed images were used for training, and another 100 second pre-processed images were used for testing.

[0135] Model training: Cross entropy loss and Adam optimizer were used simultaneously for model training, and the training was repeated multiple times.

[0136] The specific steps of using the cancer region segmentation model are:

[0137] (31) Cut the breast tissue core area into pieces according to the circumscribed rectangle of the outline, set the pixel value of the area outside the outline to 0, and adaptively scale it to b according to the mpp value of the breast tissue core area to obtain the second preprocessed image. The scaled size is recorded as scale H核心 *scale W核心 ; Among them, scale H核心 =H 核心 *a2 / b, scale W核心 =W 核心 *a2 / b, H 核心 and W 核心 is the original length and height of the core area of ​​breast tissue, a2 is the mpp value of the core area of ​​breast tissue;

[0138] (32) Inputting the second preprocessed image into the cancer region segmentation model, obtaining the probability score of each pixel in the region being "background", "cancer region", or "normal tissue region", and according to the probability score, if the probability score of the pixel being "background" is the highest, then the pixel is marked as 0; if the probability score of the pixel being "cancer region" is the highest, then the pixel is marked as 1; if the probability score of the pixel being "normal tissue region" is the highest, then the pixel is marked as 2;

[0139] (33) The extracted “normal tissue region” is adjusted using the region growing technique, and the adjacent region images are fused to output the final normal tissue, and the pixels in the region at this position are recorded as 2;

[0140] (34) Each pixel is finally marked as a connected area of ​​1, its outer contour is extracted and called a target. After filtering the targets with an area less than 500pix^2, the remaining targets are output as the final cancer areas.

[0141] (4) removing non-cancer areas and benign lesion areas in the cancer area through the overall cancer lesion model to obtain the malignant cancer area;

[0142] The overall cancer lesion model is a convolutional neural network (CNN) classification model based on MobileNetV3, with 3 output channels for classification, representing the probability scores of "non-cancer block", "benign lesion" and "malignant cancer area" respectively;

[0143] The training steps are:

[0144] Data preprocessing: The cancer area is uniformly cut into blocks according to its extracted outer contour as the center. Specifically, the coordinates of the center point of the outer contour are recorded as (x c癌 ,y c癌 ), the height and width of the circumscribed rectangle of the outer contour are H 癌轮廓 , W 癌轮廓 The center of the outer contour is the height and width of the crop. H癌 ,crop W癌 , where crop H癌 =H 癌轮廓 *1.25,crop W癌 =W 癌轮廓 *1.25, the coordinates of the center point of the outer contour (x c癌 ,y c癌 ) as the center to cut the height and width of the crop H癌 ,crop W癌 The cut rectangular area is then scaled to a size of 512*512 to obtain a third preprocessed image;

[0145] Data labeling: The third pre-processed images are used as training samples and test samples. If the data is a "non-cancerous block", it is marked as 0; if the data is a "benign lesion", it is marked as 1; if the data is a "malignant cancer area", it is marked as 2. Specifically, 2000 third pre-processed images are used for training, and another 400 third pre-processed images are used for testing;

[0146] Model training: Cross entropy loss and Adam optimizer were used to train the model and repeated multiple times.

[0147] Model testing: Verify the classification results through test samples to ensure the accuracy of the scoring.

[0148] The specific steps of using the overall cancer lesion model are as follows:

[0149] (41) Each cancer region is uniformly cut into blocks according to its extracted outer contour as the center. Specifically, the coordinates of the center point of the outer contour are recorded as (x c癌 ,y c癌 ), the height and width of the circumscribed rectangle of the outer contour are H 癌轮廓 , W 癌轮廓 The center of the outer contour is the height and width of the crop. H癌 ,crop W癌 , where crop H癌 =H 癌轮廓 *1.25,crop W癌 =W 癌轮廓 *1.25, the coordinates of the center point of the outer contour (x c癌 ,y c癌) as the center to cut the height and width of the crop H癌 ,crop W癌 The cut rectangular area is then scaled to a size of 512*512 to obtain a third preprocessed image;

[0150] (42) inputting the third preprocessed image into the overall cancer lesion model, obtaining the probability score of each pixel point in the third preprocessed image being a "non-cancer block", "benign lesion" and "malignant cancer area", and according to the score, if the "non-cancer block" has the highest score, the category output is "non-cancer block", if the "benign lesion" has the highest score, the category output is "benign lesion", and if the "malignant cancer area" has the highest score, the category output is "malignant cancer area";

[0151] (43) The malignant cancer region is retained, and the outer contour information obtained in the cancer region segmentation model is re-acquired and used as the outer contour of the malignant cancer region.

[0152] (5) Detecting the salient regions in the malignant cancer region through the salient region detection model, including rectangular salient regions and circular salient regions;

[0153] The salient region detection model uses EfficientNet as a UNet model as the backbone network, and its training process is consistent with the training process of the cancer region segmentation model.

[0154] The specific steps of using the salient region detection model are as follows:

[0155] (51) The malignant cancer area is cut into pieces according to the circumscribed rectangle of its outer contour, and the pixel values ​​of the area outside the contour are set to 0. The image is adaptively scaled to b according to the mpp value of the malignant cancer area to obtain the fourth preprocessed image. The scaled size is recorded as scale H恶性 *scale W恶性 ; Among them, scale H恶性 =H 恶性 *a3 / b, scale W恶性 =W 恶性 *a3 / b, H 恶性 and W 恶性 is the original length and height of the malignant cancer area, a3 is the mpp value of the malignant cancer area;

[0156] (52) Inputting the fourth preprocessed image into the salient region detection model, obtaining a probability score of each pixel in the fourth preprocessed image being a "background", a "cancer region", or a "normal tissue region", and according to the score, taking the pixel with the highest probability score of being a "cancer region" as a salient feature map, wherein the value range of each pixel is between 0 and 1, and the closer to 1, the more likely it is cancer;

[0157] (53) The salient feature map is significantly stretched to obtain a salient feature stretch map. The specific operation is: the feature value at the pixel position (i, j) is set to x ij , the eigenvalue after stretching is recorded as y ij , then y ij =x ij 2 ; The obtained graph is called the significant feature stretching graph;

[0158] (54) Obtain the coordinate point position (imax, jmax) with the highest probability score of the "cancer region" in the significant feature stretching map, as well as its corresponding eigenvalue y imax,jmax If y imax,jmax >0.85, then jump to step (55), otherwise jump to step (57);

[0159] (55) Take the coordinate point position (imax, jmax) as the significant center point position, and take a rectangle with a size of 1000*1000 with it as the center as the rectangular significant area; take a circle with a radius of 500 with it as the center as the circular significant area; if the number of rectangular significant areas and circular significant areas reaches 20, jump to step (57), otherwise enter step (56) to update the significant feature stretching map;

[0160] (56) Taking the obtained coordinate point position (imax, jmax) as the center and 100 as the radius, the feature value on the significant feature stretching map within the range is set to 0; then return to step (54);

[0161] (57) Output rectangular salient areas and circular salient areas, such as Figure 4 As shown, the red circular area is the significant circular area, and the green rectangle is the significant rectangular area.

[0162] (6) Classifying the nuclear atypia of each rectangular significant area using the nuclear atypia scoring model, and averaging the scores of all rectangular significant areas as the final “nuclear atypia” score;

[0163] The nuclear atypia scoring model uses MobileNetV3 as the main convolutional neural network (CNN) classification model, and the classification output channel is 3, which respectively represent the probability score values ​​of "low atypia", "medium atypia" and "high atypia";

[0164] The training steps are:

[0165] Data preprocessing: Each rectangular salient area is cut into blocks according to its circumscribed rectangle, and then the cut area is scaled to a size of 512*512;

[0166] Data labeling: The scaled images were used as training samples and test samples. If the data was "low heterogeneity", it was marked as 0, if the data was "medium heterogeneity", it was marked as 1, and if the data was "high heterogeneity", it was marked as 2. Specifically, 2,000 scaled images were used for training and another 400 scaled images were used for testing.

[0167] Model training: Cross entropy loss and Adam optimizer were used simultaneously for model training, and the training was repeated multiple times.

[0168] Model testing: Verify the classification results through test samples to ensure the accuracy of the scoring.

[0169] The specific steps of using the nuclear atypia scoring model are as follows:

[0170] (61) Cut the rectangular salient area into blocks according to its circumscribed rectangle, and then scale the cut area to a size of 512*512;

[0171] (62) Input the scaled image into the nuclear atypia scoring model to obtain the probability scores of the image being "low atypia", "medium atypia" and "high atypia"; the category with the highest score is the final category of the rectangular significant area. In addition, if the rectangular significant area is judged to be "low atypia", the score is 0; if the rectangular significant area is judged to be "medium atypia", the score is 1; if the rectangular significant area is judged to be "high atypia", the score is 2;

[0172] (63) The average of all “rectangular significant area” scores was taken as the final score at the image level, and the final cell atypia score in the range of [0, 2 / 3) was set to 1, the final cell atypia score in the range of [2 / 3, 4 / 3] was set to 2, and the final cell atypia score in the range of [4 / 3, 2] was set to 3.

[0173] (7) Using the duct formation scoring model, each rectangular significant area is classified as "duct formation", and the average of all rectangular significant areas is taken as the final "duct formation" score;

[0174] The duct formation scoring model is a convolutional neural network (CNN) classification model with MobileNetV3 as the main trunk, and the output channels of the classification are 3, which respectively represent the probability scores of "high duct formation", "medium duct formation" and "low duct formation";

[0175] The training steps are as follows:

[0176] Data preprocessing: The rectangular salient area is uniformly cut into blocks according to its circumscribed rectangle, and then the cut area is scaled to a size of 512*512;

[0177] Data labeling: The scaled images were used as training samples and test samples. If the data was "high duct formation", it was marked as 0, if the data was "medium duct formation", it was marked as 1, and if the data was "low duct formation", it was marked as 2. Specifically, 2,000 scaled images were used for training, and another 400 scaled images were used for testing.

[0178] Model training: Cross entropy loss and Adam optimizer were used to train the model and repeated multiple times.

[0179] Model testing: Verify the classification results through test samples to ensure the accuracy of the scoring.

[0180] The specific steps of using the duct formation scoring model are as follows:

[0181] (71) Cut the rectangular salient area into blocks according to its circumscribed rectangle, and then scale the cut area to a size of 512*512;

[0182] (72) Input the zoomed image into the nuclear atypia scoring model to obtain the probability scores of the image being "high duct formation", "medium duct formation" and "low duct formation". The category with the highest score is the final category of the rectangular significant area. In addition, if the rectangular significant area is judged to be "high duct formation", the score is 0; if the rectangular significant area is judged to be "medium duct formation", the score is 1; if the rectangular significant area is judged to be "low duct formation", the score is 2;

[0183] (73) The average of all “rectangular salient area” scores is taken as the final score at the image level, and the final duct formation score in the range of [0, 2 / 3) is taken as 1 point, the final duct formation score in the range of [2 / 3, 4 / 3] is taken as 2 points, and the final duct formation score in the range of [4 / 3, 2] is taken as 3 points.

[0184] (8) Calculating the number of cells with segmented mitotic figures based on the area of ​​the circular significant area using a mitotic figure counting model;

[0185] The mitotic figure counting model is a UNet segmentation model with EfficientNet as the backbone network. The output channel of the model is 2, representing the probability scores of "background" and "mitotic figures" respectively;

[0186] The training steps are:

[0187] Data preprocessing: The circular salient area is cut into pieces according to the circumscribed rectangle of the outer contour, and the pixel value of the area outside the contour is set to 0. The circular salient area is adaptively scaled to b according to the mpp value of the circular salient area to obtain the fifth preprocessed image. The scaled size is recorded as scale H圆形 *scale W圆形 ; Among them, scale H圆形 =H 圆形 *a4 / b, scale W圆形 =W 圆形 *a4 / b, H 圆形 and W 圆形 is the original length and height of the malignant cancer area, a4 is the mpp value of the malignant cancer area;

[0188] Data labeling: The fifth preprocessed image is used as a training sample and a test sample, and the position of the nuclear division image in the image is labeled as 1, and the other positions are labeled as 0; specifically, 500 targets are used for training, and the other 100 targets are used for testing;

[0189] Model training: Cross entropy loss and Adam optimizer were used to train the model and repeated multiple times.

[0190] Model testing: Verify the segmentation results through the test set data to ensure the accuracy of the scoring.

[0191] The specific steps for using the mitotic figure counting model are as follows:

[0192] (81) The circular salient area is cut into pieces according to the circumscribed rectangle of the outer contour, and the pixel values ​​of the area outside the outer contour are set to 0. The circular salient area is adaptively scaled to b according to the mpp value of the circular salient area to obtain the fifth preprocessed image. The scaled size is recorded as scale H圆形 *scale W圆形 ; Among them, scale H圆形 =H 圆形 *a4 / b, scale W圆形 =W 圆形 *a4 / b, H 圆形 and W 圆形 is the original length and height of the malignant cancer area, a4 is the mpp value of the malignant cancer area;

[0193] (82) Input the fifth preprocessed image into the nuclear division image counting model, and output the probability scores of the pixels in the image being "background" and "nuclear division image"; if the probability score of the pixel being "background" is the highest, the pixel is marked as 0; if the probability score of the pixel being "nuclear division image" is the highest, the pixel is marked as 1;

[0194] (83) The connected regions with pixels marked as 1 are subjected to contour extraction, and each connected region is output as a “nuclear fission image” target; Figure 5 As shown, the green circular outer contour is the fixed field of view, and the inner green polygon is the detected nuclear division image;

[0195] (84) If there are less than 5 “mitotic figures” in each “circular prominent area”, it is scored as 1 point; if there are 6 to 10 “mitotic figures”, it is scored as 2 points; if there are more than 11 “mitotic figures”, it is scored as 3 points;

[0196] (85) The maximum value of the probability score of nuclear division images in each "circular significant area" is taken as the nuclear division image score of the entire image.

[0197] The present invention provides an efficient and accurate algorithm for histological grading of breast cancer. At the same time, the objective and quantitative technical means of computers eliminate the subjective judgment between different doctors for breast grading. At the same time, it can provide a technical reference for doctors who are new to the field of pathological detection. This method makes full use of advanced deep learning technology and eliminates the detection bias of a single model through the cascade of multiple models, making the model monitoring effect better. It provides strong support and guarantee for the clinical diagnosis and treatment of breast cancer.

[0198] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0199] It should be noted that the technical features not described in detail in the present invention can be implemented by any existing technology.

Claims

1. A method for histological grading of breast cancer for breast tissue pathology images, characterized in that: include: (1) Using the tissue region segmentation model, the breast tissue region is located and segmented from the complete H&E stained breast pathology image; (2) Using the core area classification model to remove fragmented tissue structures in the breast tissue area, the core area of ​​the breast tissue is obtained; (3) Using the cancer region segmentation model, the cancer region in the core area of ​​the breast tissue is located and segmented; (4) removing non-cancer areas and benign lesion areas in the cancer area through the overall cancer lesion model to obtain the malignant cancer area; (5) Detecting the salient regions in the malignant cancer region through the salient region detection model, including rectangular salient regions and circular salient regions; (6) Classifying the nuclear atypia of each rectangular significant area using the nuclear atypia scoring model, and averaging the scores of all rectangular significant areas as the final "nuclear atypia" score; (7) Using the duct formation scoring model, each rectangular significant area is classified as "duct formation", and the average of all rectangular significant areas is taken as the final "duct formation" score; (8) The number of cells with segmented mitotic figures is calculated based on the mitotic figure counting model within the circular significant area.

2. The method for histological grading of breast cancer for breast tissue pathology images according to claim 1, characterized in that: The tissue region segmentation model is a UNet model with an EfficientNet network structure as the backbone network; The specific steps of using the tissue region segmentation model are: (11) The complete H&E stained breast pathology image is input into the tissue region segmentation model, and the image is adaptively scaled to b according to the mpp value of the H&E stained breast pathology image. The scaled size is recorded as scale H *scale W ; (12) The tissue region segmentation model marks the breast tissue region in the scaled breast pathology image as 1, and the rest of the background locations as 0; Mark each pixel as 1 in the connected area, extract its outer contour and call it a target; (13) Filter out breast tissue regions with an area less than 500 pix^2, and output the remaining connected regions where all pixels are marked as 1, i.e., the breast tissue regions.

3. The method for histological grading of breast cancer for breast tissue pathology images according to claim 2, characterized in that: The core area classification model is a convolutional neural network (CNN) classification model based on the MobileNetV3 network structure, and its output channel is 2, which respectively represents the probability scores of "fragmented organizational structure" and "core area"; The specific steps for using the core area classification model are: (21) Each breast tissue region is uniformly cut according to its extracted outer contour as the center. Specifically, the coordinates of the center point of the outer contour are recorded as (x c ,y c ), the center of the outer contour is the height and width of the crop H ,crop W , the coordinates of the center point of the outer contour (x c ,y c ) as the center to cut the height and width of the crop H ,crop W The cut rectangular area is then scaled to a size of m*m to obtain a first preprocessed image, where m is a multiple of 2; (22) Sending the first preprocessed image to the core area classification model to obtain the probability scores of the image being "fragmented tissue structure" and "core area"; according to the probability scores, if the probability score of "core area" is higher than the probability score of "fragmented tissue structure", the output category is "core area"; otherwise, the output is "fragmented tissue structure"; (23) The “core area” is retained as the classification result, and the outer contour information extracted from the tissue region segmentation model is obtained again, and its outer contour information is used as the outer contour of the core area to obtain the core area of ​​the breast tissue.

4. The method for histological grading of breast cancer for breast tissue pathology images according to claim 2, characterized in that: The cancer region segmentation model is a UNet model with an EfficientNet network structure as the backbone network, and has 3 output channels, which represent the probability scores of "background", "cancer region", and "cancer region" respectively; The specific steps of using the cancer region segmentation model are: (31) Cut the breast tissue core area into pieces according to the circumscribed rectangle of the outline, set the pixel value of the area outside the outline to 0, and adaptively scale it to b according to the mpp value of the breast tissue core area to obtain the second preprocessed image. The scaled size is recorded as scale H核心 *scale W核心 ; (32) Inputting the second preprocessed image into the cancer region segmentation model, obtaining the probability score of each pixel in the region being "background", "cancer region", and "normal tissue region", and according to the probability score, if the probability score of the pixel being "background" is the highest, then the pixel is marked as 0; if the probability score of the pixel being "cancer region" is the highest, then the pixel is marked as 1; if the probability score of the pixel being "normal tissue region" is the highest, then the pixel is marked as 2; (33) Using the region growing technique, the extracted "normal tissue region" is adjusted, and the adjacent regional images are fused to output the final normal tissue, and the pixels in the region at this position are recorded as 2; (34) Each pixel is finally marked as a connected area of ​​1, its outer contour is extracted and called a target. After filtering the targets with an area less than 500pix^2, the remaining targets are output as the final cancer areas.

5. The method for histological grading of breast cancer for breast tissue pathology images according to claim 1, characterized in that: The overall cancer lesion model is a convolutional neural network (CNN) classification model based on MobileNetV3, with 3 output channels for classification, representing the probability scores of "non-cancer block", "benign lesion" and "malignant cancer area" respectively; The specific steps of using the overall cancer lesion model are as follows: (41) Each cancer region is uniformly cut into blocks according to its extracted outer contour as the center. Specifically, the coordinates of the center point of the outer contour are recorded as (x c癌 ,y c癌 ), the center of the outer contour is the height and width of the crop H癌 ,crop W癌 , the coordinates of the center point of the outer contour (x c癌 ,y c癌 ) as the center to cut the height and width of the crop H癌 ,crop W癌 The cut rectangular area is then scaled to a size of m*m to obtain a third preprocessed image, where m is a multiple of 2; (42) Inputting the third preprocessed image into the overall cancer lesion model, obtaining the probability score of each pixel point in the third preprocessed image being a "non-cancer block", a "benign lesion" and a "malignant cancer area", and according to the score, if the "non-cancer block" has the highest score, the category output is "non-cancer block", if the "benign lesion" has the highest score, the category output is "benign lesion", and if the "malignant cancer area" has the highest score, the category output is "malignant cancer area"; (43) The malignant cancer region is retained, and the outer contour information obtained in the cancer region segmentation model is re-acquired and used as the outer contour of the malignant cancer region.

6. The method for histological grading of breast cancer for breast tissue pathology images according to claim 1, characterized in that: The salient area detection model uses the UNet model with EfficientNet as the backbone network; The specific steps of using the salient region detection model are as follows: (51) The malignant cancer area is cut into pieces according to the circumscribed rectangle of its outer contour, and the pixel values ​​of the area outside the contour are set to 0. The image is adaptively scaled to b according to the mpp value of the malignant cancer area to obtain the fourth preprocessed image. The scaled size is recorded as scale H恶性 *scale W恶性 ;; (52) Inputting the fourth preprocessed image into the salient region detection model, obtaining the probability score of each pixel in the fourth preprocessed image being "background", "cancer region", and "normal tissue region", and according to the score, taking the pixel with the highest probability score of being "cancer region" as the salient feature map, and the value range of each pixel is between 0 and 1, and the closer to 1, the more likely it is cancer; (53) The salient feature map is significantly stretched to obtain a salient feature stretch map. The specific operation is: the feature value at the pixel position (i, j) is set to x ij , the eigenvalue after stretching is recorded as y ij , then y ij =x ij 2 ; The obtained graph is called the significant feature stretching graph; (54) Obtain the coordinate point position (imax, jmax) with the highest probability score of the "cancer region" in the significant feature stretching map, as well as its corresponding eigenvalue y imax,jmax If y imax,jmax >p, then jump to step (55), otherwise jump to step (57); where p<1; (55) Take the coordinate point position (imax, jmax) as the position of the significant center point, and take a rectangle with a size of R1*R1 with it as the center as the rectangular significant area; take a circle with a radius of R2 with it as the center as the circular significant area; where R1>R2, if the number of rectangular significant areas and circular significant areas reaches q, jump to step (57), otherwise go to step (56) to update the significant feature stretching map; (56)Centering around the obtained coordinate point position (imax, jmax) with a radius of R3, where R3 < R2, take the eigenvalue on the stretched graph of significant features within this range as 0; then return to step (54); (57)Output the rectangular significant region and the circular significant region.

7. The method for histological grading of breast cancer for breast tissue pathology images according to claim 1, characterized in that: The nuclear atypia scoring model is based on the MobileNetV3 backbone convolutional neural network classification model, and the output channels of the classification are 3, representing the probability score values of "low atypia", "medium atypia", and "high atypia" respectively; The specific usage steps of the nuclear atypia scoring model are as follows: (61)Cut the rectangular significant region according to its circumscribed rectangle, and then scale the cut-out region to the size of m * m, where m is a multiple of 2; (62)Input the scaled image into the nuclear atypia scoring model to obtain the probability score values of "low atypia", "medium atypia", and "high atypia" for this image; take the category with the highest score as the final category of this rectangular significant region. Additionally, if this rectangular significant region is determined to be "low atypia", the score is recorded as 0, if it is determined to be "medium atypia", the score is recorded as 1, and if it is determined to be "high atypia", the score is recorded as 2; (63)Take the average of the scores of all "rectangular significant regions" as the final score at the image level, and take the final cell atypia score as 1 point when the final score is in the range of [0, 2 / 3), take the final cell atypia score as 2 points when the final score is in the range of [2 / 3, 4 / 3], and take the final cell atypia score as 3 points when the final score is in the range of [4 / 3, 2].

8. The method for histological grading of breast cancer for breast tissue pathology images according to claim 1, characterized in that: The glandular formation scoring model is a classification model of a convolutional neural network (CNN) with MobileNetV3 as the backbone, and the output channels of the classification are 3, representing the probability scores of "high glandular formation", "medium glandular formation", and "low glandular formation" respectively; The specific usage steps of the glandular formation scoring model are as follows: (71)Cut the rectangular significant region according to its circumscribed rectangle, and then scale the cut-out region to the size of m * m, where m is a multiple of 2; (72)Input the scaled image into the nuclear atypia scoring model to obtain the probability score values of "high glandular formation", "medium glandular formation", and "low glandular formation" for this image, take the category with the highest score as the final category of this rectangular significant region. Additionally, if this rectangular significant region is determined to be "high glandular formation", the score is recorded as 0, if it is determined to be "medium glandular formation", the score is recorded as 1, and if it is determined to be "low glandular formation", the score is recorded as 2; (73)Take the average of the scores of all "rectangular significant regions" as the final score at the image level, and take the final glandular formation score as 1 point when the final score is in the range of [0, 2 / 3), take the final glandular formation score as 2 points when the final score is in the range of [2 / 3, 4 / 3], and take the final glandular formation score as 3 points when the final score is in the range of [4 / 3, 2].

9. The method for histological grading of breast cancer for breast tissue pathology images according to claim 1, characterized in that: The mitotic figure counting model is a UNet segmentation model with EfficientNet as the backbone network. The output channels of the model are 2, representing the probability scores of "background" and "mitotic figures" respectively; The specific steps for using the mitotic figure counting model are as follows: (81) The circular salient area is cut into pieces according to the circumscribed rectangle of the outer contour, and the pixel values ​​of the area outside the outer contour are set to 0. The circular salient area is adaptively scaled to b according to the mpp value of the circular salient area to obtain the fifth preprocessed image. The scaled size is recorded as scale H圆形 *scale W圆形 ; (82) Input the fifth preprocessed image into the nuclear division image counting model, and output the probability scores of the pixels in the image being "background" and "nuclear division image"; if the probability score of the pixel being "background" is the highest, the pixel is marked as 0; if the probability score of the pixel being "nuclear division image" is the highest, the pixel is marked as 1; (83) Extract the outer contour of the connected regions with pixels marked as 1, and output each connected region as a "nuclear division image" target; (84) If there are less than 5 "mitotic figures" in each "circular prominent area", it is scored as 1 point, 6 to 10 as 2 points, and more than 11 as 3 points; (85) The maximum value of the probability score of nuclear division in each "circular significant area" is taken as the nuclear division score of the whole image.

Images