Method and system for interpreting expression of ER and PR in breast cancer by using artificial intelligence

Abstract

The invention provides a method and system for interpreting expression of ER and PR in breast cancer by using artificial intelligence, and relates to the technical field of artificial intelligence, the method comprises the following steps: collecting labeled breast cancer tissue pathological section images, and performing data enhancement to obtain a sample image set; performing chromatic aberration balance on the image data in the sample image set to obtain a preprocessed data set; training a preset initial segmentation network by using the preprocessed data set; inputting the breast cancer tissue pathological section image into the trained cell nucleus segmentation model to obtain a cell nucleus region image; performing morphological processing on the cell nucleus region image to obtain a processed target region; and extracting color features and morphological features in the target area, and inputting the color features and the morphological features into the classification model to obtain an interpretation result. According to the method, cell nucleus segmentation and feature extraction are carried out through the deep learning model, subjectivity and errors of manual interpretation are reduced, and the interpretation accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and system for interpreting the expression of ER and PR in breast cancer using artificial intelligence. Background Art

[0002] Breast cancer is one of the most common cancers that seriously threatens women's health and is the second leading cause of cancer death in women. According to the latest breast cancer data report in my country in 2019, the incidence of breast cancer in women in my country is about 304,000 per year, and the number of cases and deaths account for 11.2% and 9.2% of the global incidence and mortality respectively, ranking among the top in the world, and growing at a rate of about 2% per year. It can be seen that breast cancer has become the most threatening malignant tumor to the health of women in my country. In the past decade, breast cancer has made exciting progress in detection and treatment. The average 5-year survival rate of breast cancer in the United States is 90%, but in China it is only 73.1%. Therefore, improving the survival rate of breast cancer patients in my country is a huge challenge currently faced. Providing high-quality early detection of breast cancer, accurate individualized treatment services and predictive evaluation of recurrent patients are key factors in improving the 5-10 year survival rate of breast cancer patients.

[0003] In the invention patent document with application number 202110170428.0, an artificial intelligence interpretation method for breast cancer immunohistochemistry is disclosed, which includes the following steps: the first step is to use a deep convolutional neural network to complete the classification and identification of breast cancer tissue in the HE microscopic tissue pathology digital panoramic image; the second step is to quickly and accurately align the HE and IHC microscopic breast cancer tissue pathology digital panoramic images based on the pyramid model; the third step is to calculate the immunohistochemical nuclear positivity rate; this patent can realize the automatic quantitative calculation and analysis of the nuclear positivity rate of the immunohistochemical breast cancer tissue area, and on the premise of ensuring the accuracy of the calculation results of the nuclear positivity rate, it can reduce the large amount of repetitive and complicated work of doctors to manually count cells, greatly improve the calculation efficiency of the nuclear positivity rate, and efficiently assist doctors to complete the immunohistochemical nuclear positivity analysis process. However, the patent does not take into account that when counting the number of nuclear positive cells and non-nuclear positive cells, it may be affected by factors such as cell overlap and background noise, resulting in inaccurate statistical results.

[0004] In view of the defects of the above-mentioned related technologies, it is necessary to propose a new technical solution to improve the above-mentioned technical problems. Summary of the invention

[0005] In order to overcome the shortcomings of the prior art, the purpose of the present invention is to provide a method and system for interpreting the expression of ER and PR in breast cancer using artificial intelligence, which not only improves the accuracy and efficiency of interpretation, but also enhances consistency and repeatability, and has a wide range of clinical application value and research potential. Through automated and standardized processes, it can provide strong support for early diagnosis and personalized treatment of breast cancer.

[0006] To achieve the above object, the present invention provides the following solutions:

[0007] A method for interpreting the expression of ER and PR in breast cancer using artificial intelligence, comprising:

[0008] Collecting annotated breast cancer tissue pathology slice images; the breast cancer tissue pathology slice images include IHC images stained with ER and PR;

[0009] Performing data enhancement on the breast cancer tissue pathology section image to obtain a sample image set;

[0010] Performing color difference balancing on the image data in the sample image set to obtain a preprocessed data set;

[0011] Using the preprocessed data set to train a preset initial segmentation network to obtain a trained cell nucleus segmentation model;

[0012] Inputting the breast cancer tissue pathological section image into the trained cell nucleus segmentation model to obtain a cell nucleus region image;

[0013] Performing morphological processing on the cell nucleus region image to obtain a processed target region;

[0014] Color features and morphological features are extracted in the target area, and the color features and morphological features are input into the trained classification model to perform positive or negative judgment on the expression of ER and PR for each cell nucleus to obtain a judgment result.

[0015] Preferably, after extracting color features and morphological features in the target area and inputting the color features and morphological features into a trained classification model to perform positive or negative judgment on the expression of ER and PR for each cell nucleus and obtaining the judgment result, the method further comprises:

[0016] According to the interpretation results, the proportion of ER- and PR-positive cell nuclei is counted to calculate the overall positive rate;

[0017] Heat maps were generated based on the proportion of ER- and PR-positive nuclei and the overall positive rate to show the spatial distribution of ER and PR expression.

[0018] Preferably, the data enhancement includes: image rotation, image flipping and image scaling.

[0019] Preferably, performing color difference balancing on the image data in the sample image set to obtain a preprocessed data set includes:

[0020] Calculate the pixel mean of the image data in the sample image set in the three channels of RGB; wherein the pixel mean calculation formula is:

[0021]

[0022] Where D represents the pixel mean, represents the pixel mean value of the image data in the red channel of the sample image set, represents the pixel mean value of the image data in the green channel of the sample image set, represents the pixel mean value of the image data in the blue channel of the sample image set;

[0023] Calculating a balance coefficient for each color channel according to the pixel mean;

[0024] The balancing coefficient is used to perform color difference balancing on the image data in the sample image set to obtain a preprocessed data set.

[0025] Preferably, using the balance coefficient to perform color difference balance on the image data in the sample image set to obtain a preprocessed data set includes:

[0026] Using the formula:

[0027]

[0028] Performing color difference balance on the image data in the sample image set to obtain a video surveillance image after color difference balance; wherein, K R Indicates the balance coefficient of the red channel, K G Indicates the balance coefficient of the green channel, K B represents the balance coefficient of the blue channel, R′ represents the pixel value of the red channel after balance, G′ represents the pixel value of the green channel after balance, B′ represents the pixel value of the blue channel after balance, R represents the pixel value of the image data in the sample image set in the red channel, G represents the pixel value of the image data in the sample image set in the green channel, and B represents the pixel value of the image data in the sample image set in the blue channel.

[0029] Preferably, the initial segmentation network is a U-Net network.

[0030] Preferably, the cell nucleus region image is subjected to morphological processing to obtain a processed target region, including:

[0031] Performing a connected region analysis on the cell nucleus region image to delete a region in the cell nucleus region image whose area is smaller than a noise area threshold, to obtain a first region;

[0032] Using a preset first structure element to perform an expansion operation on the first region to connect the broken cell nucleus boundary to obtain a second region;

[0033] Using a preset second structure element to perform an erosion operation on the second region to restore the original shape of the cell nucleus to obtain a third region;

[0034] An adaptive boundary smoothing algorithm is applied to perform a smoothing operation on the third region to obtain a processed target region.

[0035] Preferably, the expression of the processed target area is:

[0036]

[0037] Among them, f(x,y) is the pixel value of the processed target area, indicating whether the pixel after the expansion or corrosion operation belongs to the cell nucleus area, I(x+i,y+j) is the pixel value of the cell nucleus area image at position (x+i,y+j), indicating whether the position belongs to the cell nucleus; S is the neighborhood of the first structure element and the second structure element, T connect is the preset connection threshold, x is the horizontal coordinate of the image, y is the vertical coordinate of the image, and i and j both represent the relative offset in the structural element.

[0038] A system for interpreting the expression of ER and PR in breast cancer using artificial intelligence, comprising:

[0039] An image collection module is used to collect annotated breast cancer tissue pathology slice images; the breast cancer tissue pathology slice images include IHC images stained with ER and PR;

[0040] A data enhancement module, used to perform data enhancement on the breast cancer tissue pathology section image to obtain a sample image set;

[0041] A color difference balancing module, used for performing color difference balancing on the image data in the sample image set to obtain a preprocessed data set;

[0042] A model training module, used to train a preset initial segmentation network using the preprocessed data set to obtain a trained cell nucleus segmentation model;

[0043] A detection module, used for inputting the breast cancer tissue pathology section image into a trained cell nucleus segmentation model to obtain a cell nucleus region image;

[0044] A morphological processing module, used for performing morphological processing on the cell nucleus region image to obtain a processed target region;

[0045] The judgment module is used to extract color features and morphological features in the target area, and input the color features and the morphological features into the trained classification model to perform positive or negative judgment on the expression of ER and PR in each cell nucleus to obtain a judgment result.

[0046] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0047] The present invention provides a method and system for interpreting the expression of ER and PR in breast cancer by using artificial intelligence. The method comprises: collecting annotated breast cancer tissue pathology slice images; the breast cancer tissue pathology slice images include IHC images stained with ER and PR; performing data enhancement on the breast cancer tissue pathology slice images to obtain a sample image set; performing color difference balance on the image data in the sample image set to obtain a preprocessing data set; using the preprocessing data set to train a preset initial segmentation network to obtain a trained cell nucleus segmentation model; inputting the breast cancer tissue pathology slice images into the trained cell nucleus segmentation model to obtain a cell nucleus region image; performing morphological processing on the cell nucleus region image to obtain a processed target region; extracting color features and morphological features in the target region, and inputting the color features and the morphological features into the trained classification model to interpret the positive or negative expression of ER and PR for each cell nucleus to obtain an interpretation result. The present invention uses a deep learning model to perform cell nucleus segmentation and feature extraction, which reduces the subjectivity and errors of manual interpretation and improves the accuracy of interpretation; and using the trained cell nucleus segmentation model, the cell nucleus area can be accurately identified, ensuring that the feature extraction of each cell nucleus is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0049] Figure 1 A flow chart of a method provided by an embodiment of the present invention;

[0050] Figure 2 A schematic diagram of the system structure provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] The purpose of the present invention is to provide a method and system for interpreting the expression of ER and PR in breast cancer using artificial intelligence, which not only improves the accuracy and efficiency of interpretation, but also enhances consistency and repeatability, and has a wide range of clinical application value and research potential. Through automated and standardized processes, it can provide strong support for early diagnosis and personalized treatment of breast cancer.

[0053] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0054] Figure 1 A flow chart of a method provided by an embodiment of the present invention, such as Figure 1 As shown, the present invention provides a method for interpreting the expression of ER and PR in breast cancer using artificial intelligence, comprising:

[0055] Step 100: Collect annotated breast cancer tissue pathology slice images; the breast cancer tissue pathology slice images include IHC images stained with ER and PR;

[0056] Step 200: performing data enhancement on breast cancer tissue pathology section images to obtain a sample image set;

[0057] Step 300: Perform color difference balance on the image data in the sample image set to obtain a preprocessed data set;

[0058] Step 400: using the preprocessed data set to train the preset initial segmentation network to obtain a trained cell nucleus segmentation model;

[0059] Step 500: inputting the breast cancer tissue pathology section image into the trained cell nucleus segmentation model to obtain a cell nucleus region image;

[0060] Step 600: performing morphological processing on the cell nucleus region image to obtain a processed target region;

[0061] Step 700: extracting color features and morphological features in the target area, and inputting the color features and morphological features into the trained classification model to perform positive or negative judgment on the expression of ER and PR for each cell nucleus to obtain a judgment result.

[0062] Preferably, after step 700, the method further includes:

[0063] Step 800: Counting the proportions of ER- and PR-positive cell nuclei according to the determination results to calculate the overall positive rate;

[0064] Step 900: Generate a heat map based on the proportion of ER- and PR-positive cell nuclei and the overall positive rate to show the spatial distribution of ER and PR expression.

[0065] Specifically, step 100 of this embodiment includes:

[0066] 1. Determine the data source. First, work with hospitals, pathology laboratories, or biobanks to obtain histopathological slide images of breast cancer patients. These images are slides stained with immunohistochemistry (IHC), especially for ER (estrogen receptor) and PR (progesterone receptor).

[0067] 2. Image acquisition: Use a high-resolution digital pathology scanner to scan the slices to ensure image clarity and detail retention. In this embodiment, an appropriate magnification (such as 20x or 40x) is selected during scanning to facilitate subsequent analysis.

[0068] 3. Annotation process: Use professional image annotation software (such as ImageJ, QuPath or Labelbox) to annotate images. Define the annotation standards for ER and PR positive cells according to clinical standards and research needs. Positive cells usually show specific staining intensity and distribution patterns. This embodiment requires annotating images by experienced pathologists to ensure the accuracy and consistency of the annotations. The positive or negative status of each cell nucleus should be clearly marked, and relevant metadata (such as patient information, sample source, etc.) should be recorded.

[0069] 4. Data storage: Store the annotated images and their corresponding annotation information (such as nucleus location, positive / negative status) in a database to ensure data security and accessibility. Images can be saved in standard image formats (such as TIFF or PNG), and annotation information can be stored in JSON or XML format.

[0070] 5. Data quality check: After data collection is completed, a quality check is conducted to ensure that all images meet the research standards and are accurately labeled. If necessary, a secondary review is conducted to ensure the reliability of the data.

[0071] 6. Data collation: organize the collected images and annotation information into a unified format to facilitate subsequent data enhancement and processing steps. Ensure that the structure of the data set is clear to facilitate subsequent automated processing.

[0072] Through the above steps, this embodiment successfully collects labeled breast cancer tissue pathology slice images, laying a foundation for subsequent image processing and analysis.

[0073] Furthermore, ER and PR proteins are expressed in the nuclei of tumor cells. Positive staining is brown. Tumor cell nuclei stained light brown are interpreted as positive 1+; stained brown are interpreted as positive 2+, stained dark brown are interpreted as positive 3+, and unstained are interpreted as negative.

[0074] Furthermore, the present embodiment first determines the staining intensity (1 to 3+), and then calculates the percentage of positive cells, not the sum of the three positives. The positive internal control may select normal breast duct epithelial cells adjacent to the cancer, whose ER and PR staining show different intensities. The negative internal control may select myoepithelial cells and stromal cells in the breast tissue, which do not express ER and PR. Exemplarily, the present embodiment also calculates the percentage of positively expressed cells in tumor cells (e.g., ER (3+, 80%)).

[0075] Preferably, the data enhancement includes: image rotation, image flipping and image scaling.

[0076] Specifically, in step 200, the specific process of data enhancement for breast cancer tissue pathology slice images is as follows: First, a portion of images are randomly selected from the annotated breast cancer tissue pathology slice images collected in step 100 as the basis for data enhancement. Then, a variety of data enhancement techniques are applied to each image using an image processing library (such as OpenCV or PIL) to increase the diversity and quantity of samples. Specific enhancement methods include: rotating images (such as randomly rotating 0 to 30 degrees), horizontal and vertical flipping, random cropping (maintaining the integrity of the cell nucleus), adjusting brightness and contrast, adding Gaussian noise, and performing color jittering. These enhancement operations can simulate different shooting conditions and sample variability, thereby improving the generalization ability of the model. After each enhancement, the generated new image will be saved together with the original image, and the corresponding annotation information will be attached to ensure that the enhanced sample still maintains the correct label. Finally, through these enhancement operations, an image set containing diverse samples is generated, providing a rich data foundation for subsequent model training.

[0077] Furthermore, in this embodiment, a series of data enhancement operations are defined to improve the robustness and generalization ability of the model based on the characteristics of breast cancer tissue pathology slice images. Common enhancement methods include: rotation, simulating different slice directions by randomly rotating the image (such as 0 to 30 degrees) to ensure that the model can adapt to different perspectives; flipping, including horizontal and vertical flipping, to increase the diversity of samples, especially when the symmetry of the cell nucleus is strong; cropping, randomly cropping a part of the image while maintaining the integrity of the cell nucleus to enhance the model's learning ability for local features; brightness and contrast adjustment, simulating different staining and imaging conditions by randomly changing the brightness and contrast of the image; adding noise, such as Gaussian noise, to improve the model's robustness to image noise; color jittering, randomly adjusting the hue, saturation and brightness of the image to enhance the model's adaptability to different staining intensities; affine transformation, further increasing the diversity of samples through operations such as translation, scaling and tilting. These enhancement strategies are designed to simulate various variations that may be encountered in the real world, ensuring that the trained model can maintain good performance under different conditions.

[0078] Preferably, performing color difference balancing on the image data in the sample image set to obtain a preprocessed data set includes:

[0079] Calculate the pixel mean of the image data in the sample image set in the three channels of RGB; wherein the pixel mean calculation formula is:

[0080]

[0081] Where D represents the pixel mean, represents the pixel mean value of the image data in the red channel of the sample image set, represents the pixel mean value of the image data in the green channel of the sample image set, represents the pixel mean value of the image data in the blue channel of the sample image set;

[0082] Calculating a balance coefficient for each color channel according to the pixel mean;

[0083] The balancing coefficient is used to perform color difference balancing on the image data in the sample image set to obtain a preprocessed data set.

[0084] Preferably, using the balance coefficient to perform color difference balance on the image data in the sample image set to obtain a preprocessed data set includes:

[0085] Using the formula:

[0086]

[0087] Performing color difference balance on the image data in the sample image set to obtain a video surveillance image after color difference balance; wherein, K R Indicates the balance coefficient of the red channel, K G Indicates the balance coefficient of the green channel, K B represents the balance coefficient of the blue channel, R′ represents the pixel value of the red channel after balance, G′ represents the pixel value of the green channel after balance, B′ represents the pixel value of the blue channel after balance, R represents the pixel value of the image data in the sample image set in the red channel, G represents the pixel value of the image data in the sample image set in the green channel, and B represents the pixel value of the image data in the sample image set in the blue channel.

[0088] Specifically, this embodiment considers that when the external lighting conditions change, the surface color of the cell remains unchanged, but the color of the object captured by the imaging device will deviate with the change of lighting, gradually deviating from the true color of the object. The present invention uses a color difference balance algorithm to compensate for this defect of the imaging device and correct the color difference of the image.

[0089] Preferably, the initial segmentation network is a U-Net network.

[0090] Optionally, the specific steps of using the preprocessed data set to train the preset initial segmentation network U-Net in this embodiment are as follows: First, prepare a preprocessed data set that has been processed by data enhancement and color difference balance, including an input image and a corresponding annotation mask. Then, use a deep learning framework (such as TensorFlow or PyTorch) to build a U-Net model, which consists of an encoder and a decoder, and can effectively extract image features and perform pixel-level segmentation. During the training process, the preprocessed data set is divided into a training set and a validation set to monitor the performance of the model. Then, a loss function (such as cross entropy loss or Dice loss) and an optimizer (such as Adam or SGD) are defined to optimize the model parameters. Next, the U-Net is iteratively trained using the training set, the output is calculated by forward propagation, and compared with the true annotation, and the model weights are updated by back propagation. During the training process, the model performance is regularly evaluated on the validation set, and hyperparameters (such as learning rate, batch size, etc.) are adjusted to improve the segmentation accuracy. After the training is completed, the trained U-Net model is saved so that it can be used for the segmentation of the cell nucleus region in subsequent steps. Through this series of steps, the U-Net model can effectively learn the nucleus features in breast cancer tissue pathology slice images and achieve high-quality nucleus segmentation.

[0091] Preferably, the cell nucleus region image is subjected to morphological processing to obtain a processed target region, including:

[0092] Performing a connected region analysis on the cell nucleus region image to delete a region in the cell nucleus region image whose area is smaller than a noise area threshold, to obtain a first region;

[0093] Using a preset first structure element to perform an expansion operation on the first region to connect the broken cell nucleus boundary to obtain a second region;

[0094] Using a preset second structure element to perform an erosion operation on the second region to restore the original shape of the cell nucleus to obtain a third region;

[0095] An adaptive boundary smoothing algorithm is applied to perform a smoothing operation on the third region to obtain a processed target region.

[0096] Specifically, in step 400, the specific implementation process of performing morphological processing on the cell nucleus region image to obtain the processed target region is as follows: First, the cell nucleus region image is analyzed using a connected region analysis algorithm (such as the connectedComponents function in OpenCV), all connected regions are identified, and the area of ​​each region is calculated. According to a preset noise area threshold, the region with an area less than the threshold is deleted to obtain a first region. Next, a preset first structural element (such as a 3x3 circular or cross structural element) is used to perform an expansion operation (using the cv2.dilate function) on the first region to connect the broken cell nucleus boundary to form a more complete cell nucleus shape to obtain a second region. Then, a preset second structural element (usually a shape smaller than the first structural element) is applied to perform an erosion operation (using the cv2.erode function) on the second region to remove excess noise and restore the original shape of the cell nucleus to obtain a third region. Finally, an adaptive boundary smoothing algorithm (such as a smoothing method based on Gaussian blur or median filtering) is applied to smooth the third region to reduce the irregularity of the boundary to obtain the final processed target region. Through this series of morphological operations, the cell nucleus region can be effectively extracted and optimized, providing a clear basis for subsequent feature extraction and classification.

[0097] Optionally, the first structural element in this embodiment is used for expansion operation, the purpose of which is to connect the broken cell nucleus boundaries. In this embodiment, a small circular or elliptical structural element is selected because these shapes can effectively fill the gaps between cell nuclei. The size and shape of the structural element are adjusted according to the average size and shape of the cell nuclei.

[0098] Further, the second structural element of this embodiment is used for the corrosion operation, the purpose of which is to restore the original shape of the cell nucleus. This embodiment is the same as or slightly smaller than the structural element of the expansion operation to ensure that the connection effect is not destroyed while restoring the shape. Selecting the appropriate structural element size is key, and this embodiment is determined through experiments to achieve the best effect.

[0099] Furthermore, this embodiment selects a suitable noise area threshold by analyzing the distribution of cell nucleus sizes in the training data set. This embodiment sets it to a ratio (such as 10%) smaller than the average cell nucleus area to remove noise without affecting the real cell nucleus.

[0100] Preferably, the expression of the processed target area is:

[0101]

[0102] Among them, f(x,y) is the pixel value of the processed target area, indicating whether the pixel after the expansion or corrosion operation belongs to the cell nucleus area, I(x+i,y+j) is the pixel value of the cell nucleus area image at position (x+i,y+j), indicating whether the position belongs to the cell nucleus; S is the neighborhood of the first structure element and the second structure element, T connect is the preset connection threshold, x is the horizontal coordinate of the image, y is the vertical coordinate of the image, and i and j both represent the relative offset in the structural element.

[0103] Optionally, the connection threshold of this embodiment is used for dynamic adjustment and is set according to the characteristics of the image and the density of the cell nucleus. This embodiment selects a threshold that can effectively connect the broken boundaries by analyzing the connection effects of different images.

[0104] Furthermore, the method for determining the preset connection threshold is specifically as follows:

[0105] 1. Image feature analysis,First, the feature analysis of the cell nucleus region image to be processed is performed, including the shape, size, distribution density and boundary characteristics of the cell nucleus, etc. By calculating the histogram, edge intensity and texture features of the image, preliminary information about the distribution and connection status of the cell nucleus can be obtained.

[0106] 2. Preliminary threshold setting: a preliminary connection threshold is set according to the image features. The preliminary threshold of this embodiment is based on empirical rules or obtained through statistical analysis of existing labeled data. For example, the average or median of the cell nucleus area in the image is selected as a reference for the preliminary threshold.

[0107] 3. Connection effect evaluation: use the initially set connection threshold to dilate the cell nucleus region image and evaluate the connection effect. This embodiment determines the connection effect by calculating the number of cell nuclei after connection, area change, and boundary integrity.

[0108] 4. Dynamically adjust the threshold. According to the evaluation results, dynamically adjust the connection threshold. If the connection effect is not ideal (for example, too many or too few cell nuclei are connected), the threshold can be gradually increased or decreased until an optimal value that can effectively connect the broken boundaries is found. This process can be performed by cross-validation, that is, testing different thresholds on different images and selecting the threshold that performs best in most images.

[0109] 5. Final confirmation: After determining the optimal connection threshold, final confirmation is performed. By comparing with the expert annotation results, it is ensured that the selected threshold can effectively connect the cell nucleus boundaries and keep the morphological characteristics of the cell nucleus from being excessively changed.

[0110] Through the above steps, the connection threshold selection of this embodiment not only takes into account the characteristics of the image and the density of the cell nucleus, but also combines the actual connection effect evaluation to ensure that the best cell nucleus connection effect can be achieved under different image conditions. The dynamic adjustment method of this embodiment improves the adaptability and accuracy of the algorithm and provides a solid foundation for subsequent cell nucleus analysis.

[0111] Optionally, in step 700, the specific implementation process steps for feature extraction and classification of the processed target area image are as follows: First, color features and morphological features are extracted from the target area image obtained after morphological processing. The color features are obtained by calculating the average color value, standard deviation and color histogram of each cell nucleus region. Commonly used color spaces include RGB and HSV to better capture the staining intensity and distribution of the cell nucleus. The morphological features include the area, perimeter, shape factor (such as roundness, aspect ratio) and complexity of the boundary of the cell nucleus, etc. These features are calculated by the contour analysis function in the image processing library (such as OpenCV). Next, the extracted color features and morphological features are combined into a feature vector and input into a pre-trained classification model (such as a support vector machine, a random forest or a deep learning model) to perform a positive or negative judgment of ER and PR expression for each cell nucleus. The classification model predicts based on the feature vector and outputs the classification result of each cell nucleus. Finally, the judgment result is associated with the original image to generate annotation information containing the expression status of ER and PR in each cell nucleus, providing a basis for subsequent analysis and clinical decision-making. Through this series of steps, the classification and interpretation of cell nuclei can be effectively achieved, supporting in-depth research on breast cancer tissue pathology.

[0112] Further, taking convolutional neural network (CNN) as an example, this embodiment constructs a classification model for nucleus classification, and its structure generally includes multiple convolutional layers, pooling layers and fully connected layers. Specifically, the input layer of the model receives the extracted nucleus feature vector, and then extracts local features through a series of convolutional layers (such as 3x3 convolution kernels), and each convolutional layer is followed by an activation function (such as ReLU) to introduce nonlinearity. Subsequently, a pooling layer (such as maximum pooling) is used for downsampling to reduce the feature dimension and enhance the translation invariance of the model. After several convolutional and pooling layers, the feature map is flattened and input to the fully connected layer, and finally classified by an output layer (usually using a softmax activation function), and the ER and PR expression status (positive or negative) of each nucleus is output. When implementing classification, this embodiment first inputs the processed nucleus image into the model, calculates the feature representation of each nucleus through forward propagation, and then predicts through the trained model to output the classification result. The training process of the model uses a labeled data set and optimizes the network parameters through the back propagation algorithm to minimize the loss between the predicted results and the true labels (such as cross entropy loss), thereby improving the accuracy and robustness of classification. Through this structure, CNN can effectively capture the complex characteristics of the cell nucleus and achieve efficient classification tasks.

[0113] Furthermore, the interpretation result of this embodiment may include the following information:

[0114] 1. Cell nucleus classification results, each cell nucleus is classified as positive or negative for ER (estrogen receptor) and PR (progesterone receptor) expression. This classification result of the present embodiment is based on the extracted feature vectors and predicted by the trained classification model, reflecting the response of the cell nucleus to the corresponding hormones.

[0115] 2. Confidence score: For each classification result, the model will output a confidence score, which indicates the confidence of the model in the classification result. Usually, this score is calculated by the softmax layer, and the value range is between 0 and 1. The closer to 1, the more confident the model is that the cell nucleus belongs to a certain category.

[0116] 3. Morphological characteristics of the cell nucleus. The interpretation results may also include morphological characteristic information of each cell nucleus, such as area, perimeter, shape factor (such as roundness, aspect ratio), etc. These characteristics of this embodiment are helpful to further analyze the morphological changes of the cell nucleus and its biological significance.

[0117] 4. Image annotation information: the classification results and confidence scores of each cell nucleus are annotated on the original image. Positive and negative cell nuclei are usually represented by different colors or symbols to facilitate intuitive observation and analysis.

[0118] 5. Statistical analysis results: statistical analysis is performed on the classification results of the cell nuclei in the entire image, including the number and proportion of positive cell nuclei, and the distribution of different types of cell nuclei. These statistical data of this embodiment can provide important references for clinical decision-making.

[0119] Through these interpretation results, researchers and clinicians can better understand the biological characteristics of the cell nucleus and evaluate the hormone receptor status of breast cancer tissue, thereby providing a basis for the formulation of subsequent treatment plans.

[0120] Optionally, in step 800, the specific steps of calculating the overall positive rate by counting the ratio of ER and PR positive nuclei according to the interpretation results are as follows: First, the present embodiment extracts the classification information of all nuclei from the nucleus classification results obtained in step 700, including the ER and PR expression status of each nucleus. Next, the number of ER positive nuclei and PR positive nuclei is counted respectively, and the number of total nuclei is calculated. The ratio of ER positive nuclei is obtained by dividing the number of ER positive nuclei by the total number of nuclei; similarly, the ratio of PR positive nuclei is calculated. Finally, the overall positive rate can be obtained by adding the number of ER positive nuclei and PR positive nuclei and dividing it by the total number of nuclei. This embodiment uses the NumPy or Pandas library in Python to efficiently process data to ensure the accuracy and reliability of the statistical results.

[0121] Further, in step 900, a heat map is generated based on the ratio of ER and PR positive nuclei and the overall positive rate to show the specific steps of the spatial distribution of ER and PR expression as follows: First, a two-dimensional grid is constructed using the spatial position information (such as coordinates) of the nuclei and the classification results, and each unit of the grid represents an area in the image. Next, the ratio of ER and PR positive nuclei in the area is calculated based on the number of nuclei in each grid unit and their positive status. Then, a heat map generation tool (such as Matplotlib or Seaborn library) is used to map these ratios to color values ​​to generate a heat map, in which the color depth represents the intensity of ER and PR expression. Finally, the heat map is superimposed on the original image to intuitively display the spatial distribution of ER and PR, helping researchers and clinicians to better understand the hormone receptor expression pattern of the nucleus and its distribution characteristics in the tissue.

[0122] Corresponding to the above method, such as Figure 2 As shown, this embodiment also provides a system for interpreting the expression of ER and PR in breast cancer using artificial intelligence, including:

[0123] An image collection module is used to collect annotated breast cancer tissue pathology slice images; the breast cancer tissue pathology slice images include IHC images stained with ER and PR;

[0124] A data enhancement module, used to perform data enhancement on the breast cancer tissue pathology section image to obtain a sample image set;

[0125] A color difference balancing module, used for performing color difference balancing on the image data in the sample image set to obtain a preprocessed data set;

[0126] A model training module, used to train a preset initial segmentation network using the preprocessed data set to obtain a trained cell nucleus segmentation model;

[0127] A detection module, used for inputting the breast cancer tissue pathology section image into a trained cell nucleus segmentation model to obtain a cell nucleus region image;

[0128] A morphological processing module, used for performing morphological processing on the cell nucleus region image to obtain a processed target region;

[0129] The judgment module is used to extract color features and morphological features in the target area, and input the color features and the morphological features into the trained classification model to perform positive or negative judgment on the expression of ER and PR in each cell nucleus to obtain a judgment result.

[0130] The beneficial effects of the present invention are as follows:

[0131] (1) The present invention uses a deep learning model to perform cell nucleus segmentation and feature extraction, which reduces the subjectivity and errors of manual interpretation and improves the accuracy of interpretation; using the trained cell nucleus segmentation model, the cell nucleus area can be accurately identified, ensuring that the feature extraction of each cell nucleus is more accurate.

[0132] (2) Compared with traditional manual interpretation, the automated AI method of the present invention can process a large number of images in a short period of time, significantly improving work efficiency; it can analyze multiple samples at the same time, meet clinical needs, and support large-scale screening and research.

[0133] (3) The present invention ensures consistency in interpretation results between different samples and different operators through unified algorithms and models, reducing the impact of human factors; the training and application process of the AI ​​model can be repeated to ensure similar results under different experimental conditions.

[0134] (4) By extracting color features and morphological features, the present invention can comprehensively evaluate the state of the cell nucleus and provide richer information for interpretation; by utilizing the feature learning ability of the deep learning model, it can automatically discover and utilize complex feature relationships and improve the sensitivity and specificity of the interpretation.

[0135] (5) Through data enhancement technology, the model of the present invention can adapt to different sample variability and improve the generalization ability of the model; the color difference balance processing ensures the consistency of image data under different staining conditions, so that the model can still maintain good performance under different experimental conditions.

[0136] (6) The present invention provides pathologists with reliable interpretation results, assists clinical decision-making, and helps doctors formulate personalized treatment plans; through rapid and accurate interpretation, it can help achieve early diagnosis of breast cancer and improve the survival rate of patients.

[0137] (7) The application of the present invention can promote the development of breast cancer-related research and promote the development of new drugs and treatment plans; as more data are accumulated, the model can be continuously optimized to improve its applicability in different clinical scenarios.

[0138] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0139] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for interpreting the expression of ER and PR in breast cancer using artificial intelligence, characterized in that: include: Collecting annotated breast cancer tissue pathology slice images; the breast cancer tissue pathology slice images include IHC images stained with ER and PR; Performing data enhancement on the breast cancer tissue pathology section image to obtain a sample image set; Performing color difference balancing on the image data in the sample image set to obtain a preprocessed data set; Using the preprocessed data set to train a preset initial segmentation network to obtain a trained cell nucleus segmentation model; Inputting the breast cancer tissue pathological section image into the trained cell nucleus segmentation model to obtain a cell nucleus region image; Performing morphological processing on the cell nucleus region image to obtain a processed target region; Color features and morphological features are extracted in the target area, and the color features and morphological features are input into the trained classification model to perform positive or negative judgment on the expression of ER and PR for each cell nucleus to obtain a judgment result.

2. The method for interpreting the expression of ER and PR in breast cancer using artificial intelligence according to claim 1, characterized in that: After extracting color features and morphological features in the target area and inputting the color features and morphological features into the trained classification model to perform positive or negative judgment on the expression of ER and PR for each cell nucleus and obtaining the judgment result, the method further includes: According to the interpretation results, the proportion of ER- and PR-positive cell nuclei was counted to calculate the overall positive rate; Heat maps were generated based on the proportion of ER- and PR-positive nuclei and the overall positive rate to show the spatial distribution of ER and PR expression.

3. The method for interpreting the expression of ER and PR in breast cancer using artificial intelligence according to claim 1, characterized in that: The data enhancement includes: image rotation, image flipping and image scaling.

4. The method for interpreting the expression of ER and PR in breast cancer using artificial intelligence according to claim 1, characterized in that: Performing color difference balancing on the image data in the sample image set to obtain a preprocessed data set includes: Calculate the pixel mean of the image data in the sample image set in the three channels of RGB; wherein the pixel mean calculation formula is: Where D represents the pixel mean, represents the pixel mean value of the image data in the red channel of the sample image set, represents the pixel mean value of the image data in the green channel of the sample image set, represents the pixel mean value of the image data in the blue channel of the sample image set; Calculating a balance coefficient for each color channel according to the pixel mean; The balancing coefficient is used to perform color difference balancing on the image data in the sample image set to obtain a preprocessed data set.

5. The method for interpreting the expression of ER and PR in breast cancer using artificial intelligence according to claim 4, characterized in that: Using the balance coefficient to perform color difference balance on the image data in the sample image set to obtain a preprocessed data set, including: Using the formula: Performing color difference balance on the image data in the sample image set to obtain a video surveillance image after color difference balance; wherein, K R Indicates the balance coefficient of the red channel, K G Indicates the balance coefficient of the green channel, K B represents the balance coefficient of the blue channel, R′ represents the pixel value of the red channel after balance, G′ represents the pixel value of the green channel after balance, B′ represents the pixel value of the blue channel after balance, R represents the pixel value of the image data in the sample image set in the red channel, G represents the pixel value of the image data in the sample image set in the green channel, and B represents the pixel value of the image data in the sample image set in the blue channel.

6. The method for interpreting the expression of ER and PR in breast cancer using artificial intelligence according to claim 1, characterized in that: The initial segmentation network is a U-Net network.

7. The method for interpreting the expression of ER and PR in breast cancer using artificial intelligence according to claim 1, characterized in that: Performing morphological processing on the cell nucleus region image to obtain a processed target region includes: Performing a connected region analysis on the cell nucleus region image to delete a region in the cell nucleus region image whose area is smaller than a noise area threshold, to obtain a first region; Using a preset first structure element to perform an expansion operation on the first region to connect the broken cell nucleus boundary to obtain a second region; Using a preset second structure element to perform an erosion operation on the second region to restore the original shape of the cell nucleus to obtain a third region; An adaptive boundary smoothing algorithm is applied to perform a smoothing operation on the third region to obtain a processed target region.

8. The method for interpreting the expression of ER and PR in breast cancer using artificial intelligence according to claim 1, characterized in that: The expression of the processed target area is: Among them, f(x,y) is the pixel value of the processed target area, indicating whether the pixel after the expansion or corrosion operation belongs to the cell nucleus area, I(x+i,y+j) is the pixel value of the cell nucleus area image at position (x+i,y+j), indicating whether the position belongs to the cell nucleus; S is the neighborhood of the first structure element and the second structure element, T connect is the preset connection threshold, x is the horizontal coordinate of the image, y is the vertical coordinate of the image, and i and j both represent the relative offset in the structural element.

9. A system for interpreting the expression of ER and PR in breast cancer using artificial intelligence, characterized in that: include: An image collection module is used to collect annotated breast cancer tissue pathology slice images; the breast cancer tissue pathology slice images include IHC images stained with ER and PR; A data enhancement module, used to perform data enhancement on the breast cancer tissue pathology section image to obtain a sample image set; A color difference balancing module, used for performing color difference balancing on the image data in the sample image set to obtain a preprocessed data set; A model training module, used to train a preset initial segmentation network using the preprocessed data set to obtain a trained cell nucleus segmentation model; A detection module, used for inputting the breast cancer tissue pathology section image into a trained cell nucleus segmentation model to obtain a cell nucleus region image; A morphological processing module, used for performing morphological processing on the cell nucleus region image to obtain a processed target region; The judgment module is used to extract color features and morphological features in the target area, and input the color features and the morphological features into the trained classification model to perform positive or negative judgment on the expression of ER and PR for each cell nucleus to obtain a judgment result.

Images