Breast cancer pathological section detection method and system
Through sliding window segmentation and optical density processing technology, combined with the AP-ViT model with adaptive patch size, the problem of inaccurate feature extraction caused by staining differences in breast cancer pathological section detection is solved, which improves detection accuracy and efficiency and reduces the The doctor's diagnostic burden provides detailed diagnostic reports.
Patent Information
- Application Number
- CN202411101299.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-12
- Publication Date
- 2024-11-29
AI Technical Summary
In the automated detection of breast cancer pathological sections, due to staining differences and inconsistent image quality, feature extraction is inaccurate, affecting the detection accuracy and efficiency.
The full-field pathological images were divided by sliding windows, and the color was standardized through optical density processing technology, and the pathological images were automatically classified by automatic patch size AP-ViT model.
It improves the detection accuracy and efficiency of pathological images, reduces the diagnostic burden of doctors, and provides detailed diagnostic reports by automatically identifying and labeling benign and malignant pathological images, enhancing clinical reference information.
Smart Images

Figure CN119048450A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of medical image recognition, and in particular to a method and system for detecting pathological sections of breast cancer. Background Art
[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.
[0003] Breast cancer is a disease in which abnormal cells in the breast grow out of control and form tumors, which may worsen later in life. If not controlled, breast cancer cells may spread to other organs throughout the body, becoming life-threatening. The earliest form of breast cancer (carcinoma in situ) is not life-threatening and can be detected at an early stage. However, the cancer cells can spread to nearby breast tissue (invasive cancer) and form a lump or thickening of tissue. In more serious cases, the cancer may spread to nearby lymph nodes or other organs (metastasis), which can be life-threatening.
[0004] As the "gold standard" for breast cancer diagnosis, pathological biopsy plays an increasingly important role in clinical medicine. With the development of optical technology, biopsy will be scanned into whole-view images (WSI). Clinicians can judge whether the patient has tissue cancer by observing digital images. Since the resolution of WSI is usually as high as 100k×100k, it has placed a great burden on doctors' clinical work. With the development of deep learning technology, especially in the field of computer vision, the automated detection of pathological images has become possible.
[0005] Common pathological tissues are usually stained with H&E stains and scanned with an electron microscope to obtain full-field images. Due to the influence of the type and amount of stains and the ambient light when scanning the images, the quality of the films will vary, which will have a great impact on the recognition of pathological images. Summary of the invention
[0006] In response to the above-mentioned defects, the present invention proposes a breast cancer pathology section detection method and system, which adopts a sliding window method for segmentation, and solves the problem of inaccurate feature extraction caused by uneven quality of pathology images through optical density processing technology. Combined with an improved lesion property detection model, automatic classification of pathology images is achieved, the accuracy and efficiency of pathology detection are improved, and the clinical diagnosis burden of doctors is reduced.
[0007] In order to achieve the above objectives, the present disclosure adopts the following technical solutions:
[0008] The first aspect of the present disclosure provides a method for detecting breast cancer pathological sections, comprising the following steps:
[0009] Acquire full-field pathological images;
[0010] Segment the full-field pathological image and perform color standardization on the segmented pathological image;
[0011] The color-normalized pathological image is input into the detection model with adaptive patch size to train the detection model;
[0012] The trained adaptive patch size detection model is used to identify and predict pathological images.
[0013] As a further implementation method, the full-field pathology image is segmented, specifically:
[0014] The acquired full-field pathology image is segmented in a sliding window manner to obtain multiple patches, and adjacent patches have overlapping areas.
[0015] As a further implementation method, the segmented pathological image is subjected to color standardization, specifically:
[0016] The image is converted from RGB space to optical density space, the main staining basis vectors are extracted, the staining concentration is calculated and normalized, and then converted back to RGB space.
[0017] As a further implementation, the adaptive patch size detection model is an AP-ViT model, which adapts to the complexity of different pathological image contents by dynamically adjusting the size of the image patch.
[0018] As a further implementation, the dynamic adjustment of the size of the image patch is specifically as follows:
[0019] The gradient amplitude of the pathological image is calculated, and the size of the image cropping is dynamically adjusted based on the comparison between the gradient amplitude and the gradient threshold.
[0020] As a further implementation, compared with fixed-size patches, variable-size patches can better capture the cell and tissue structure features in pathological images, using small patches where cell nuclei are dense and large patches where duct cavities are.
[0021] As a further implementation method, the probability that the patient has breast cancer is calculated based on the predicted probability of the pathological image.
[0022] A second aspect of the present disclosure provides a breast cancer pathology section detection system, comprising:
[0023] The data acquisition module is configured to: acquire a full-field pathological image;
[0024] The preprocessing module is configured to: segment the full-field pathology image and perform color standardization on the segmented pathology image;
[0025] The model training module is configured to: input the color-normalized pathological image into the adaptive patch size detection model to train the detection model;
[0026] The output module is configured to: use the trained adaptive patch size detection model to identify and predict the pathological image.
[0027] The third aspect of the present disclosure provides a medium on which a program is stored. When the program is executed by a processor, the steps in the method for detecting pathological sections of breast cancer described in the first aspect of the present disclosure are implemented.
[0028] A fourth aspect of the present disclosure provides an electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the method for detecting pathological sections of breast cancer described in the first aspect of the present disclosure are implemented.
[0029] Compared with the prior art, the present invention has the following beneficial effects:
[0030] The present invention discloses a method for detecting breast cancer pathology sections, which uses a sliding window to segment the full-field pathology image, and uses a color normalization algorithm based on the optical density method to perform color normalization on the pathology image. By standardizing the color distribution in different images, the consistency and comparability of the images are improved, and the problem of inaccurate feature extraction due to staining differences, which in turn leads to a decrease in the accuracy of pathology image detection, is solved. Combined with the lesion property detection model, automatic classification of pathology images is achieved, and the average prediction probability of pathology images is recorded. At the same time, the number of benign and malignant pathology images, the probability of cancer, and the comparison diagram of pathology images with a higher probability of malignancy and benign and malignant images are given in the diagnosis report, which improves the accuracy and efficiency of pathology detection and reduces the clinical diagnosis burden of doctors.
[0031] The present invention discloses a method for detecting breast cancer pathological sections. The lesion property detection model adopts the AP-ViT model, and the pathological image is feature extracted and classified through adaptive patch, block embedding and self-attention mechanism. The AP-ViT model of adaptive patch can adjust the size of the patch according to the tissue structure, improves the ability to capture key information, can effectively capture the global features and complex texture changes in the pathological image, and improves the recognition accuracy of the lesion area. The AP-ViT model is used to classify the pathological images, automatically identify and mark benign and malignant pathological images, and record the average predicted probability of the pathological images. By displaying pathological images with a higher probability of malignancy and comparison diagrams of benign and malignant images in the diagnosis report, it helps doctors to make accurate diagnoses and provide more detailed clinical reference information.
[0032] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] The accompanying drawings constituting a part of the present disclosure are used to provide a further understanding of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure.
[0034] Figure 1 It is a flow chart of the breast cancer pathological section detection method disclosed in the present invention. DETAILED DESCRIPTION
[0035] The present disclosure is further described below in conjunction with the accompanying drawings and embodiments.
[0036] It should be noted that the following detailed descriptions are all illustrative and are intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present disclosure belongs.
[0037] In the absence of conflict, the embodiments in the present disclosure and the features in the embodiments may be combined with each other.
[0038] Embodiment 1
[0039] like Figure 1 As shown, the first embodiment of the present disclosure provides a method for detecting pathological sections of breast cancer, which mainly includes the following steps:
[0040] S1 obtains full-field pathological images;
[0041] S2 segments the full-field pathological image and performs color standardization on the segmented pathological image;
[0042] S3 inputs the color-normalized pathological image into the detection model with adaptive patch size to train the detection model;
[0043] S4 uses the trained adaptive patch size detection model to identify and predict pathological images.
[0044] Based on the recognition and prediction results, a pathology detection report is given, which includes the average predicted probability of malignant pathology sections, the probability of cancer, images of key pathology sections, and comparisons of normal area images and abnormal area images.
[0045] The whole slide imaging (WSI) is obtained by scanning and the whole slide imaging (WSI) is segmented as follows:
[0046] The acquired full-view pathological image is segmented in a sliding window manner to obtain multiple patches, and there are overlapping areas between adjacent patches.
[0047] The scanned WSI is cropped into smaller patches according to a certain size and stride to facilitate computer processing and analysis. The cropped patch should cover the entire WSI image, and there should be overlapping areas between adjacent patches to ensure that no important information is lost. The patch is named according to the file name of the original WSI image, the sequence number of the image, the horizontal starting coordinate, the vertical starting coordinate, and the saving format. This method of cropping and naming ensures the uniqueness of each patch, and the location information is included in the name, which is convenient for quick positioning during analysis.
[0048] Among them, in the present invention, the size is 256×256 and the stride is 128 for cropping, and the cropped pathological images are named sample_wsi_patch1_x0_y0.png, sample_wsi_patch2_x128_y0.png, ..., sample_wsi_patch268_x0_y128.png, sample_wsi_patch269_x128_y128.png, ..., wherein patch1, patch2...patch268, patch269 are the pathological image serial numbers, x{n}, y{m} are the horizontal starting coordinates and the vertical starting coordinates, respectively, and png is the image saving format.
[0049] The segmented pathological images are color standardized as follows:
[0050] The image is converted from RGB space to optical density space, the main staining basis vectors are extracted, the staining concentration is calculated and normalized, and then converted back to RGB space.
[0051] The optical density normalization method is used for preprocessing to standardize the color without destroying the image structure. This method can reduce the interference caused by color differences and improve the accuracy of model recognition.
[0052] S21 inputs a pathological image and converts the image from RGB value to the corresponding optical density value (OD). The corresponding conversion formula is:
[0053]
[0054] Where I is the RGB color component and I0 is the intensity of the light source (usually 255).
[0055] S22 calculates the staining basis matrix, which represents the main staining direction. For the sake of stability, a threshold β is set for pixels with almost no staining (low OD), and pixels with β<0.15 are deleted.
[0056] Compute the covariance matrix:
[0057]
[0058] Where μ is the mean optical density value, OD i is the optical density value after denoising, and N is the number of samples.
[0059] Perform eigenvalue decomposition:
[0060] ∑=VΛV T
[0061] Where V is the eigenvector matrix and Λ is the eigenvalue diagonal matrix.
[0062] Select the main chromaticity basis vectors:
[0063] V=[V1,V2]
[0064] Among them, V1 and V2 are the last two columns of the feature vector V.
[0065] S23 calculates the staining concentration of each pixel, projects the optical density value of each pixel onto the staining basis vector, and obtains the staining concentration matrix:
[0066] C=ODV
[0067] Where C is the staining concentration matrix.
[0068] S24 then performs color density normalization to normalize the color density of the input image to within the color density range of the target image.
[0069] First, calculate the 99th percentile of the input image and target image staining concentration:
[0070] max source =ercentile(C source ,99,axis=0)
[0071] max target =ercentile(C target ,99,axis=0)
[0072] Then scale the color density of the input image to the range of the target image:
[0073]
[0074] S25 finally converts the standardized color concentration into RGB space.
[0075] Calculate the normalized optical density value:
[0076]
[0077] Convert the optical density value back to RGB value:
[0078] I normalized =255·exp(- normalized )
[0079] Among them, I normalized is the normalized RGB value of the image.
[0080] Through the above steps, color standardization can be achieved, so that the staining variation in pathological images is reduced and the accuracy of image analysis is improved. The core of this method is to convert the image to optical density space, extract the main staining basis vector, calculate and standardize the staining concentration, and then convert it back to RGB space.
[0081] In the related technology, image standardization based on color segmentation can distinguish different staining components and extract specific staining areas. However, this method is sensitive to differences in staining intensity and is easily affected by noise, resulting in poor image standardization effects. Image standardization based on histogram equalization can enhance image contrast, but it will amplify image noise and have inconsistent effects on different images. For breast cancer pathology images, H&E staining agents are mostly used for coloring. After staining, the nucleus is blue-purple, and the cytoplasm and connective tissue are light pink. The color difference is small. The above two methods have certain limitations.
[0082] The method proposed in this example converts the pathological image from RGB space to optical density space, and the optical density value essentially reflects the concentration of the dye. Therefore, this method can more accurately correct the color difference between different pathological images and maintain color consistency; optical density conversion will not affect the structural information of cells and can retain more image details, especially in areas with weak staining; the image after optical density normalization has more uniform staining concentration and color distribution, which reduces the analysis error caused by staining differences and improves the stability and reliability of image analysis.
[0083] The color-normalized pathological images are input into the detection model with adaptive patch size to train the detection model.
[0084] The adaptive patch size detection model is the AP-ViT (Adaptive Patch Vision Transformer) model, which dynamically adjusts the size of the image patch to adapt to the complexity of different pathological image contents. The BreakHis dataset (Breast Cancer Histopathological Image Dataset) is used for training.
[0085] BreakHis is a public dataset for breast cancer pathology image classification research. The dataset covers 7909 images from 82 patients. The images were taken at different magnifications (40×, 100×, 200×, 400×), including 2480 benign tumor images and 5429 malignant tumor images. It covers four types of benign tumor images (adenosis, tubular adenoma, fibroadenoma, phyllodes tumor) and four types of malignant tumor images (intraductal carcinoma, lobular carcinoma, papillary carcinoma, mucinous carcinoma), multi-scale magnification, and multiple types of cancer types, which are more in line with the pathological section samples to be detected in clinical situations. Therefore, the BreakHis dataset is used. First, the dataset is divided into a training set and a validation set, with a ratio of 80% and 20% to ensure that benign and malignant samples are evenly distributed in the training set and validation set. The training set is further divided into two categories: benign and malignant.
[0086] Vision Transformer (ViT) is a deep learning model that applies the Transformer architecture to computer vision tasks. Transformer initially achieved great success in the field of natural language processing (NLP), and ViT introduced it to image classification tasks. The basic idea of the ViT model is to divide the image into several patches of fixed size, and regard each patch as a sequence element, flatten and embed these patches into a high-dimensional space through linear projection, and then input them into the Transformer encoder for processing. Unlike traditional convolutional neural networks (CNNs), ViT relies on a self-attention mechanism to capture global information and long-distance dependencies. Pathological sections usually contain complex tissue structures and diverse cell morphologies. ViT uses a self-attention mechanism to capture global and local features in the image well.
[0087] This example uses an improved ViT model (AP-ViT model), which dynamically adjusts the size of image patches to adapt to the complexity of different pathological image contents. Compared with fixed-size patches, variable-size patches can better capture the cell and tissue structural features in pathological images. Smaller patches are used in densely packed cell nuclei and larger patches are used in ductal cavities, thereby improving the model's expressiveness and classification performance. The pre-trained AP-ViT model is selected for transfer learning to specifically improve the accuracy of breast cancer pathology image recognition.
[0088] Specifically:
[0089] S31 uses the Prewitt operator to calculate the gradient value of the color-normalized pathological image and then detects the edge in the image.
[0090]
[0091]
[0092] Among them, I x is the horizontal gradient, I y is the vertical gradient, I is the gray value of the image, G x , G y are the operators in the horizontal and vertical directions respectively.
[0093] Then calculate the gradient magnitude:
[0094]
[0095] Finally, the gradient threshold is set, and the size of the image patch is adjusted by comparing the gradient amplitude and the gradient threshold, and the average value of the gradient amplitude is:
[0096]
[0097] Set the gradient threshold T:
[0098]
[0099] Among them, k is the threshold coefficient. According to multiple experiments, the model shows better recognition effect when k is 1.46.
[0100] S32 adjusts the size of the patch and the step size according to the comparison between the gradient amplitude and the gradient threshold.
[0101] Specifically:
[0102] Initialize the window starting points to i and j. At each position, determine the patch size and step size based on the gradient value G. If the gradient value G[i,j] of the current pixel is greater than the threshold T, use a 16×16 window and a step size of 8. If the gradient value G[i,j] of the current pixel is less than the threshold T, use a 32×32 window and a step size of 16. After determining the window size, crop a patch of the corresponding size from the image and add it to the result list, and crop it sequentially until the entire image is covered.
[0103] According to the input principle of the Vision Transformer network model, S33 needs to convert all patches into one-dimensional arrays of the same dimension through a linear mapping layer. The entire image is flattened into a 2D sequence, and the number of patches is the length of the sequence. Therefore, the Transformer model originally used in the NLP field can be used to deal with visual problems.
[0104] The trained adaptive patch size detection model is used to identify and predict pathological images.
[0105] The probability that the patient has breast cancer is calculated based on the predicted probability of the pathological image. Generate a pathological test report, load the trained model, and cut, color normalize, and predict the model of the clinical breast cancer pathological images to finally obtain the benign and malignant probabilities of each image. In order to further enrich the content of the test report and more accurate indicators, it is necessary to calculate the probability that the patient has breast cancer based on the predicted probability of the pathological image.
[0106] The predicted probability of each pathological image is stored in an array, and then the average predicted probability of all pathological images is calculated as the probability that the patient has breast cancer.
[0107] The top six pathological images with the highest predicted probability of malignancy were screened out and given as significant samples in the pathological test report. At the same time, the two images with the highest predicted probability and the two images with the lowest predicted probability were screened out as comparison samples for reference by clinicians.
[0108] Embodiment 2
[0109] Embodiment 2 of the present disclosure provides a breast cancer pathology section detection system, including:
[0110] The data acquisition module is configured to: acquire a full-field pathological image;
[0111] The preprocessing module is configured to: segment the full-field pathology image and perform color standardization on the segmented pathology image;
[0112] The model training module is configured to: input the color-normalized pathological image into the adaptive patch size detection model to train the detection model;
[0113] The output module is configured to: use the trained adaptive patch size detection model to identify and predict the pathological image.
[0114] It also includes a report generation module and a data storage module.
[0115] Among them, the report generation module is configured to: generate a breast cancer pathology image detection report, screen out the six pathology images with the highest malignancy probability and two pairs of benign and malignant comparison images according to the malignancy probability output by the classification module, and calculate the average predicted probability through statistical means as the probability of the patient having cancer.
[0116] The data storage module is configured to store and manage all data in the system. It stores all acquired and preprocessed pathological slice images, including original images and cut pathological images; stores the classification results of each pathological slice image, including the probability values of benign and malignant; stores all statistical analysis results and generated test reports for subsequent retrieval and analysis; manages and maintains all historical data in the system, and supports data backup, recovery, and deletion operations.
[0117] The more detailed steps are the same as those in the first embodiment and will not be repeated here.
[0118] Embodiment 3
[0119] Embodiment 3 of the present disclosure provides a medium on which a program is stored. When the program is executed by a processor, the steps in the method for detecting pathological sections of breast cancer as described in Embodiment 1 of the present disclosure are implemented.
[0120] The more detailed steps are the same as those in the first embodiment and will not be repeated here.
[0121] Embodiment 4
[0122] Embodiment 4 of the present disclosure provides an electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps in the method for detecting pathological sections of breast cancer as described in Embodiment 1 of the present disclosure are implemented.
[0123] The more detailed steps are the same as those in the first embodiment and will not be repeated here.
[0124] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. For those skilled in the art, the present disclosure may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A method for detecting breast cancer pathological sections, characterized in that: The following steps are involved: Acquire full-field pathological images; Segment the full-field pathological image and perform color standardization on the segmented pathological image; The color-normalized pathological image is input into the detection model with adaptive patch size to train the detection model; The trained adaptive patch size detection model is used to identify and predict pathological images.
2. A method for detecting breast cancer pathological sections as claimed in claim 1, characterized in that: The full-field pathology image is segmented as follows: The acquired full-field pathology image is segmented in a sliding window manner to obtain multiple patches, and adjacent patches have overlapping areas.
3. A method for detecting breast cancer pathological sections as claimed in claim 1, characterized in that: The segmented pathological images are color standardized as follows: The image is converted from RGB space to optical density space, the main staining basis vectors are extracted, the staining concentration is calculated and normalized, and then converted back to RGB space.
4. A method for detecting breast cancer pathological sections as claimed in claim 1, characterized in that: The adaptive patch size detection model is the AP-ViT model, which adapts to the complexity of different pathological image contents by dynamically adjusting the size of the image patch.
5. A method for detecting breast cancer pathological sections as claimed in claim 4, characterized in that: The dynamic adjustment of the size of the image patch is specifically as follows: The gradient amplitude of the pathological image is calculated, and the size of the image cropping is dynamically adjusted based on the comparison between the gradient amplitude and the gradient threshold.
6. A method for detecting breast cancer pathological sections as claimed in claim 4, characterized in that: Compared with fixed-size patches, variable-size patches can better capture the cell and tissue structural features in pathological images, using small patches where cell nuclei are dense and large patches where duct cavities are.
7. A method for detecting breast cancer pathological sections as claimed in claim 1, characterized in that: The probability that the patient has breast cancer is calculated based on the predicted probability of the pathological image.
8. A breast cancer pathology section detection system, characterized in that: include: The data acquisition module is configured to: acquire a full-field pathological image; The preprocessing module is configured to: segment the full-field pathology image and perform color standardization on the segmented pathology image; The model training module is configured to: input the color-normalized pathological image into the adaptive patch size detection model to train the detection model; The output module is configured to: use the trained adaptive patch size detection model to identify and predict the pathological image.
9. A storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps in the method for detecting pathological sections of breast cancer as described in any one of claims 1 to 7 are implemented.
10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the method for detecting pathological sections of breast cancer as described in any one of claims 1 to 7 are implemented.
Citation Information
Patent Citations
Method for color standardization of pathological image based on low rank embedded non-negative matrix decomposition
CN109345601A
Pathological image automatic classification method based on dyeing intensity matrix
CN113538422A
Breast cancer pathology image recognition method based on key region extraction
CN114140445A
Breast cancer pathological image classification device and method based on deep learning
CN116524226A
