Artificial intelligence-based breast cancer nat efficacy prediction method and system
By combining image registration and boundary reconstruction techniques with deep learning models, the problem of early and accurate prediction of the efficacy of NAT in breast cancer has been solved, improving prediction accuracy and support for personalized treatment.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- FOSHAN NANHAI DISTRICT PEOPLES HOSPITAL
- Filing Date
- 2025-09-01
- Publication Date
- 2026-07-21
AI Technical Summary
In existing technologies, it is difficult to achieve early, accurate, and non-invasive prediction of the efficacy of neoadjuvant therapy (NAT) for breast cancer. Insufficient information fusion from imaging methods leads to low assessment accuracy and makes it difficult to meet the needs of individualized treatment.
By acquiring MRI image datasets of breast cancer patients before and after NAT, image registration and annotation were performed to locate the core region of the tumor, identify tumor shrinkage sub-images, reconstruct tumor boundaries, and use a pre-trained deep learning model to predict treatment efficacy.
It improves the accuracy of predicting the efficacy of NAT in breast cancer, enables early identification of tumor regression, enhances the recognition ability of deep learning models, and provides a basis for personalized treatment.
Smart Images
Figure CN121075650B_ABST
Abstract
Description
Technical Field
[0001] This disclosure belongs to the field of medical information processing technology, specifically relating to an artificial intelligence-based method and system for predicting the efficacy of breast cancer NAT. Background Technology
[0002] In the field of cancer treatment, especially in neoadjuvant therapy (NAT) for breast cancer, imaging methods are playing an increasingly important role in efficacy assessment and the development of individualized treatment strategies.
[0003] Breast cancer is a highly heterogeneous malignant tumor, and its response to treatment-associated breast cancer (NAT) varies significantly among individuals. Achieving pathologic complete response (pCR) during treatment is considered a key indicator for evaluating NAT effectiveness and predicting patient prognosis. However, current accuracy in early pCR prediction remains unsatisfactory, failing to meet the needs of personalized and precise clinical treatment. Therefore, how to utilize imaging methods to predict the efficacy of NAT in breast cancer patients early, accurately, and non-invasively is a crucial problem that urgently needs to be addressed in clinical and research fields.
[0004] Currently, clinical assessment of NAT efficacy primarily relies on radiologists' subjective judgment based on observing post-NAT MRI images. This assessment method is easily influenced by physician experience, lacks stability, and fails to systematically reflect the dynamic evolution of tumor morphology, function, and microscopic tissue characteristics during treatment. Particularly in multimodal MRI images, such as dynamic contrast-enhanced MRI (DCE-MRI) and diffusion-weighted imaging (DWI), the information fusion between different modalities has not been fully explored, severely limiting the accuracy of predicting NAT efficacy in breast cancer and its clinical application.
[0005] During NAT treatment, the tumor regresses. However, under the influence of NAT drugs, the regressing tumor undergoes several changes. First, reduced blood supply to the tumor tissue leads to significantly decreased enhancement on dynamic contrast-enhanced MRI (DCE-MRI) images, resulting in unclear boundaries between the tumor and surrounding normal breast tissue. Second, due to intratumoral cell necrosis, stromal fibrosis, and inflammatory cell infiltration, the tumor signal is reduced on T2-weighted imaging (T2WI) and diffusion-weighted imaging (DWI), further blurring the contrast with surrounding normal breast tissue and obscuring the tumor's boundaries. Therefore, the pathological and histological changes occurring in breast cancer during NAT can affect the identification of the tumor's regressed boundaries, reducing the accuracy of existing AI-based breast cancer NAT efficacy prediction systems. Summary of the Invention
[0006] The purpose of this invention is to propose a method and system for predicting the efficacy of breast cancer NAT based on artificial intelligence, so as to solve one or more technical problems existing in the prior art, and at least provide a beneficial option or create conditions.
[0007] To achieve the above objectives, according to one aspect of this disclosure, an artificial intelligence-based method for predicting the efficacy of breast cancer treatment using neonatal naïve breast cancer is provided, the method comprising the following steps: S100: Obtain MRI images of breast cancer patients' lesions before NAT treatment to form a pre-treatment image dataset, and obtain MRI images of the early stage of NAT treatment to form a post-treatment image dataset. S200: After image registration of the pre-treatment image dataset and the post-treatment image dataset, the corresponding lesion locations are labeled as regions of interest. S300: Locate the core region of the tumor in the region of interest of the pre-treatment image dataset as the dark target region, and identify the tumor regression sub-image in the post-treatment image dataset based on the dark target region; S400, the boundaries of the tumor regression sub-images in the post-treatment image dataset are reconstructed to obtain the reconstructed image dataset; S500, based on a reconstructed image dataset, uses a pre-trained deep learning model to predict the efficacy of NAT in breast cancer treatment.
[0008] Definitions: Pre-treatment: Pre-treatment refers to the period before NAT treatment; Post-treatment: Post-treatment refers to the early stage of NAT treatment.
[0009] Furthermore, in S100, the method for obtaining MRI images of breast cancer patients' lesions before NAT treatment to form a pre-treatment image dataset is as follows: MRI images of breast cancer patients' lesions before NAT treatment are obtained using a SIMENSE 3.0 MRI scanner to form a pre-treatment image dataset.
[0010] Furthermore, in S100, the method for obtaining MRI images of the early stage of NAT treatment to form a post-treatment image dataset is as follows: MRI images of the early stage of NAT treatment are obtained using a SIMENSE 3.0 MRI scanner to form a post-treatment image dataset.
[0011] Furthermore, in S100, the number of images in the pre-treatment image dataset and the post-treatment image dataset are the same, and their locations are the same.
[0012] The grayscale values of different MRI sequences are related to the water content, cell density, and blood supply of tumor tissue. Areas of tumor tissue with high water content appear as high signal on T2WI, while areas with low water content appear as low signal (high signal refers to brighter areas on the MRI image, while low signal appears as darker or blacker areas). Areas with high cell density within the tumor appear as high signal on DWI images due to restricted free diffusion of water molecules; conversely, areas with low cell density appear as low signal on DWI. Areas of the tumor with rich blood supply show high enhancement (i.e., high signal) on DCE-MRI, while areas with poor blood supply show low signal. See reference: Brown RW, Cheng YCN, Haacke E, et al. Magnetic Resonance Imaging: Physical Principles and Sequence Design[J]. 1999.DOI:10.1002 / 9781118633953.
[0013] Furthermore, in S300, the method for locating the tumor core region of the region of interest in the pre-treatment image dataset as the dark target region is as follows: the region of interest is grayscaled to obtain a grayscale image, the grayscale image is divided into sub-regions, and the sub-region with the smallest average grayscale value of each pixel in each sub-region is recorded as the tumor core region, and the tumor core region is recorded as the dark target region.
[0014] Preferably, the grayscale image is divided into sub-regions by performing a cap transformation and then extracting the edge lines of the grayscale image using an edge detection operator to form multiple sub-regions.
[0015] Preferably, a superpixel algorithm is used to automatically divide the tumor into 100 subregions.
[0016] Among them, the superpixel algorithm is either the Linear Iterative Clustering (SLIC) superpixel algorithm or the SEEDS (Superpixels Extracted via Energy-Driven Sampling) algorithm.
[0017] Preferably, the number of subregions is 100; Furthermore, in S200, the method for marking the corresponding lesion location as the region of interest is as follows: based on the second phase of DCE enhancement as the main reference, a rectangular box is manually drawn by a professional physician to cover and mark the entire tumor area, and then the edge detection operator is used to identify the edge of the lesion to obtain the precise outline of the lesion as the region of interest.
[0018] The edge detection operators include the Roberts operator and the Prewitt operator, with the Roberts operator being preferred because the extracted dark target region is used in subsequent steps to accurately locate and correct the tumor edge. Due to the strong invasiveness of tumors, some tumor tissue extends into normal breast tissue, resulting in a blurred interface where tumor and normal tissue coexist. While the Prewitt operator suppresses noise through pixel averaging, this is equivalent to low-pass filtering the image. The Roberts operator, on the other hand, typically produces a wider response in the region near the tumor edge, better identifying the blurred interface caused by tumor invasiveness. Therefore, the Prewitt operator is not as effective as the Roberts operator in locating tumor edges.
[0019] Furthermore, in S300, the method for identifying tumor regression sub-images in the post-treatment image dataset based on dark target regions is as follows: The corresponding position of the dark target region in the pre-treatment image dataset within the region of interest in the post-treatment image dataset is denoted as the anchor region; thus, each dark target region corresponds to an anchor region in the post-treatment image dataset. The maximum gray value of each point in the dark target region is obtained as TaMax; the minimum gray value of each point in the anchor region corresponding to the dark target region is obtained as AnMin; the absolute value of the difference between TaMax and AnMin is recorded as the NAT gray value difference (the NAT gray value difference reflects the degree of extreme difference in gray value caused by signal changes in MRI imaging before and after NAT treatment in the tumor center). Multiple edge lines are detected in the region of interest in the post-treatment image dataset using an edge detection operator. The surface of the region of interest is divided into multiple test regions by each edge line. The average gray value of each pixel in the test region is recorded as the gray value of the test region. The grayscale of each test area is judged sequentially. If the absolute value of the difference between the grayscale of the test area and the average grayscale value of all points on the anchored area is less than the NAT grayscale difference, then the test area corresponding to the grayscale of the test area is marked as a tumor shrinkage sub-image.
[0020] The above methods screen for areas exhibiting tumor regression by dynamically calculating the grayscale difference before and after NAT treatment. However, after NAT treatment, the blood supply to the tumor tissue is significantly reduced, leading to cell necrosis, stromal fibrosis, and inflammatory cell infiltration. This results in a decrease in the grayscale difference between the tumor area and surrounding normal breast tissue on various MRI sequences, blurring the boundary between the tumor and surrounding tissue. Since tumor regression is affected by interference from adjacent regions, leading to unclear tumor boundaries, simply comparing grayscale differences will result in low accuracy in identifying tumor regression sub-images of the region to be identified.
[0021] Preferably, in S300, the method for identifying tumor regression sub-images in the post-treatment image dataset based on the dark target region is as follows: The corresponding position of the dark target region in the pre-treatment image dataset within the region of interest in the post-treatment image dataset is denoted as the anchor region; thus, each dark target region has a corresponding anchor region within the region of interest. The point with the highest gray value in the dark target region is located at position TaP in the region of interest; the point with the lowest gray value in the anchor region corresponding to the dark target region is located at position AnP in the region of interest; the direction from point TaP to point AnP is denoted as the retreat direction; multiple edge lines are detected in the region of interest using the edge detection operator, and the surface of the region of interest is divided into multiple regions to be tested using each edge line. The tumor regression pathways for each region to be tested were determined as follows: In each test region, the straight line connecting the point with the smallest gray value on the boundary of the test region and the point with the smallest gray value in the anchored region is the hydration tendency line; the straight line connecting the point with the largest gray value on the boundary of the test region and the point with the largest gray value in the anchored region is the retreat tendency line; the area between the hydration tendency line and the retreat tendency line is the tumor retreat channel of the test region; (the tumor retreat channel is the retreat direction along the tumor center between each test region and the anchored region. One side of its boundary is the area with abnormal water content after tumor shrinkage (hydration tendency line), and the other side is the boundary of retreat (retreat tendency line). Because the tumor will retreat during the treatment process, especially after neoadjuvant chemotherapy (NAT), the retreat area will be affected by the significant reduction in tumor blood supply, cell death, stromal fibrosis and inflammatory cell infiltration, which will affect the appearance in MRI imaging. The gray value of this area is relatively discrete and irregular, and the tumor boundary is not clear. Existing deep learning models have difficulty distinguishing it from the surrounding tissue, resulting in low recognition accuracy). The determination of whether the current region to be tested is a tumor regression sub-image is based on the tumor regression pathway, specifically: Each region to be tested that intersects with the tumor regression channel of the current region to be tested is recorded as a feature region. Each feature region is judged in turn. If the gray value of a feature region is less than the gray value of the feature region before the regression direction and the gray value of the feature region is less than the gray value of the feature region after the regression direction, then the current region to be tested is recorded as a tumor regression sub-image.
[0022] Beneficial effects: Based on the gray values of all feature regions along the entire path to the tumor area, the changing trend of hydrogen proton content in the fine tissue of key areas can be accurately monitored. From MRI images, tumor retraction sub-images caused by NAT treatment can be accurately identified, which greatly improves the recognition accuracy of deep learning models in subsequent steps.
[0023] In particular, if the size of the current test area and the adjacent test areas are relatively similar in the regression direction of the tumor regression channel, discontinuities and alternating jumps between dark and light regions can easily occur in the gray-scale gradient of the regression direction. Therefore, the above linear comparison can still easily lead to mis-selection, thereby reducing the recognition accuracy of the deep learning model in subsequent steps. This application proposes the following method to address the above problems. The specific method is as follows: Preferably, determining whether the current region to be tested is a tumor regression sub-image based on the tumor regression channel specifically involves: Each region intersects with the tumor regression channel of the current test region and is recorded as a feature region. The regression grayscale difference value of each feature region in the tumor regression channel is calculated, specifically: The feature region with the largest average gray value in the tumor retreat channel is designated as the peak feature region; the feature region with the smallest average gray value in the tumor retreat channel is designated as the valley feature region; the feature regions in the tumor retreat channel are sorted in order of retreat direction, and the index of the peak feature region in the tumor retreat channel is designated as UpI; the index of the valley feature region in the tumor retreat channel is designated as LowI. The average gray value of all feature regions from the current test area to UpI in the tumor retreat channel is used as the front retreat gray value; the average gray value of all feature regions from the current test area to LowI in the tumor retreat channel is used as the back retreat gray value; the difference between the front retreat gray value and the back retreat gray value is used as the retreat gray value difference value. For each feature region in the tumor regression channel, if the regression grayscale difference value of a feature region is less than the regression grayscale difference value of the feature region before the regression direction, and the regression grayscale difference value of the feature region is less than the regression grayscale difference value of the feature region after the regression direction, then the current test region is determined to be a tumor regression sub-image.
[0024] After locating all the tumor regression sub-images, since these images represent the insignificant volume reduction produced in the early stages of NAT treatment, and pathological changes such as reduced tumor blood supply after NAT treatment lead to unclear boundaries between the tumor and surrounding normal tissue, making them difficult to distinguish, the tumor regression sub-images need to be adjusted according to the following method to enable the deep learning model to accurately identify them: Furthermore, in S400, the method for reconstructing the boundaries of the tumor regression sub-images in the post-treatment image dataset to obtain the reconstructed image dataset is as follows: Let P1 be the point with the smallest gray value on the boundary line of the tumor regression sub-image in the post-treatment image dataset, and P2 be the point with the largest gray value. Then, P1 and P2 divide the boundary line of the tumor regression sub-image into two curve segments. The shorter curve segment is taken as the fuzzy boundary line, and the longer curve segment is taken as the stable boundary line. Let PA1 be the point with the smallest absolute value of the difference between the gray value of each point on the anchored region boundary line and P1, and PA2 be the point with the smallest absolute value of the difference between the gray value of each point on the anchored region boundary line and P2. Then, PA1 and PA2 divide the anchored region boundary line into two curve segments. The shorter curve segment is taken as the mapping boundary line. Copying the mapping boundary line yields a replacement curve segment. Scaling the replacement curve segment to the same size as the blurred boundary line yields a clear boundary line. Deleting the blurred boundary line of the tumor shrinkage sub-image leaves a stable boundary line. After moving the clear boundary line to the positions where its two endpoints coincide with points P1 and P2 respectively, the clear boundary line and the stable boundary line together form the new boundary line of the tumor regression sub-image. The reconstructed image dataset consists of tumor shrinkage sub-images obtained from each new boundary line.
[0025] Furthermore, in S500, the method for obtaining the prediction results of the efficacy of breast cancer NAT based on the reconstructed image dataset using a pre-trained deep learning model is as follows: feature vectors are obtained by using the ResNeXt network model to extract features from the reconstructed image dataset. By comparing the differences between the pre-treatment image dataset and the reconstructed image dataset, the corresponding difference comparison feature vector is obtained; The difference comparison feature vectors are fused to obtain the fused feature vector; Based on the fused feature vector, the classification and prediction results of the NAT efficacy of breast cancer by artificial intelligence are obtained.
[0026] Preferably, the deep learning model is the ResNeXt network model.
[0027] Preferably, the AI-based breast cancer NAT efficacy prediction system of the present invention can not only predict the efficacy of targeted therapy for deep temporal lobe tumors (tumors of brain structures such as the amygdala and hippocampus), benign brain tumors, and brain diseases, using pre-treatment and post-treatment image datasets from the early stages of treatment.
[0028] This invention also provides an AI-based breast cancer NAT efficacy prediction system. The AI-based breast cancer NAT efficacy prediction system includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps in the AI-based breast cancer NAT efficacy prediction method. The AI-based breast cancer NAT efficacy prediction system can run on computing devices such as desktop computers, laptops, handheld computers, and cloud data centers. The runnable system may include, but is not limited to, processors, memory, and server clusters. The processor executes the computer program within the following system units: The image data acquisition unit is used to acquire MRI images of breast cancer patients before NAT treatment to form a pre-treatment image dataset, and to acquire MRI images of the early stage of NAT treatment to form a post-treatment image dataset. The data registration and annotation unit is used to register the pre-treatment image dataset and the post-treatment image dataset respectively, and then annotate the corresponding lesion locations as regions of interest. The tumor regression recognition unit is used to locate the core region of the tumor in the region of interest of the pre-treatment image dataset as the dark target region, and to identify the tumor regression sub-image in the post-treatment image dataset based on the dark target region. The regression boundary reconstruction unit is used to reconstruct the boundaries of the tumor regression sub-images in the post-treatment image dataset to obtain the reconstructed image dataset. The efficacy prediction unit is used to obtain the prediction results of the efficacy of breast cancer NAT based on the reconstructed image dataset and a pre-trained deep learning model.
[0029] The beneficial effects of this invention are as follows: This invention provides a method and system for predicting the efficacy of NAT in breast cancer based on artificial intelligence. By correcting the large number of tumor image boundaries caused by interference from adjacent regions due to tumor regression, this invention eliminates the problem of indistinguishability between the tumor location and surrounding tissue caused by regression. This allows the tumor regression sub-images of the region to be identified to be used in deep learning networks, improving the reliability of the extracted data for prediction models. Attached Figure Description
[0030] The above and other features of this disclosure will become more apparent from the detailed description of the embodiments illustrated in conjunction with the accompanying drawings. In the accompanying drawings, the same reference numerals denote the same or similar elements. Obviously, the drawings described below are merely some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained from these drawings without any creative effort. In the drawings: Figure 1 The flowchart shown is a breast cancer NAT efficacy prediction method based on artificial intelligence; Figure 2 The image shown is a pattern of the post-treatment image dataset before reconstruction. Figure 3 The image shown is a diagram of the reconstructed image dataset. Figure 4 The diagram shows the structure of an AI-based breast cancer NAT efficacy prediction system. Detailed Implementation
[0031] The following will provide a clear and complete description of the concept, specific structure, and resulting technical effects of this disclosure in conjunction with the embodiments and accompanying drawings, so as to fully understand the purpose, solution, and effects of this disclosure. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.
[0032] Example 1 like Figure 1 The diagram shows a flowchart of an AI-based method for predicting the efficacy of breast cancer treatment using NAT. The following section will combine... Figure 1 This invention describes an artificial intelligence-based method for predicting the efficacy of breast cancer NAT treatment according to an embodiment of the present invention. The method includes the following steps: S100: Obtain MRI images of breast cancer patients' lesions before NAT treatment to form a pre-treatment image dataset, and obtain MRI images of the early stage of NAT treatment to form a post-treatment image dataset. S200: After image registration of the pre-treatment image dataset and the post-treatment image dataset, the corresponding lesion locations are labeled as regions of interest. S300: Locate the core region of the tumor in the region of interest of the pre-treatment image dataset as the dark target region, and identify the tumor regression sub-image in the post-treatment image dataset based on the dark target region; S400, the boundaries of the tumor regression sub-images in the post-treatment image dataset are reconstructed to obtain the reconstructed image dataset; S500, based on a reconstructed image dataset, uses a pre-trained deep learning model to predict the efficacy of NAT in breast cancer treatment.
[0033] Furthermore, in S100, the method for obtaining MRI images of breast cancer patients' lesions before NAT treatment to form a pre-treatment image dataset is as follows: MRI images of breast cancer patients' lesions before NAT treatment are obtained using a SIMENSE 3.0 MRI scanner to form a pre-treatment image dataset.
[0034] Furthermore, in S100, the method for obtaining MRI images of the early stage of NAT treatment to form a post-treatment image dataset is as follows: MRI images of the early stage of NAT treatment are obtained using a SIMENSE 3.0 MRI scanner to form a post-treatment image dataset.
[0035] Furthermore, in S100, the pre-treatment image dataset and the post-treatment image dataset have the same number of images and are located in the same positions. The pre-treatment image dataset and the post-treatment image dataset are multimodal MRI images of breast cancer patients collected at two time points: before NAT (C0) and in the early treatment stage (C2), respectively, covering three sequences (T2WI, DWI, and T1+C). N4 bias field correction, three-dimensional isotropic resampling, grayscale normalization, and discretization are employed to unify imaging characteristics and reduce cross-device and cross-center interference.
[0036] Furthermore, in S300, the method for locating the tumor core region of the region of interest in the pre-treatment image dataset as the dark target region is as follows: the region of interest is grayscaled to obtain a grayscale image, the grayscale image is divided into sub-regions, and the sub-region with the smallest average grayscale value of each pixel in each sub-region is recorded as the tumor core region, and the tumor core region is recorded as the dark target region.
[0037] Preferably, a superpixel algorithm is used to automatically divide the tumor into 100 sub-regions and remove invalid voxel clusters.
[0038] The superpixel algorithm is the Linear Iterative Clustering (SLIC) superpixel algorithm.
[0039] Furthermore, in S200, the method for marking the corresponding lesion location as the region of interest is as follows: based on the second phase of DCE enhancement as the main reference, a rectangular frame covering the entire tumor area is manually drawn by a professional physician as the region of interest; each region of interest is adjusted to 256×256 pixels.
[0040] Furthermore, in S300, the method for identifying tumor regression sub-images in the post-treatment image dataset based on dark target regions is as follows: The corresponding position of the dark target region in the pre-treatment image dataset within the region of interest in the post-treatment image dataset is denoted as the anchor region; thus, each dark target region corresponds to an anchor region in the post-treatment image dataset. The maximum gray value of each point in the dark target region is obtained as TaMax; the minimum gray value of each point in the anchor region corresponding to the dark target region is obtained as AnMin; the absolute value of the difference between TaMax and AnMin is recorded as the NAT gray value difference (the NAT gray value difference reflects the degree of extreme difference in gray value caused by signal changes in MRI imaging before and after NAT treatment in the tumor center). Multiple edge lines are detected in the region of interest in the post-treatment image dataset using an edge detection operator. The surface of the region of interest is divided into multiple test regions by each edge line. The average gray value of each pixel in the test region is recorded as the gray value of the test region. The grayscale of each test area is judged sequentially. If the absolute value of the difference between the grayscale of the test area and the average grayscale value of all points on the anchored area is less than the NAT grayscale difference, then the test area corresponding to the grayscale of the test area is marked as a tumor shrinkage sub-image.
[0041] The key C# source code describing the specific implementation of the method for identifying tumor regression sub-images in the post-treatment image dataset based on dark target regions is as follows: public class ImageProcessor / / Main processing method public void ProcessNATImages( Image<Gray, byte> preTreatmentImage, / / Pretreatment Image Image<Gray, byte> postTreatmentImage, / / Post-treatment image Rectangle[] darkTargetRegions, / / Dark target regions in the pretreatment image Rectangle roi) / / Region of Interest in post-treatment images { / / 1. Establish the correspondence between anchored areas var anchorRegions = MapAnchorRegions(preTreatmentImage,postTreatmentImage, darkTargetRegions, roi); / / 2. Calculate the NAT grayscale difference var natDifferences = CalculateNatDifferences(preTreatmentImage, postTreatmentImage, anchorRegions); / / 3. Edge detection and region segmentation var edgeRegions = DetectAndDivideEdgeRegions(postTreatmentImage,roi); / / 4. Identify tumor retraction sub-images var hydrationRetreatSubImages = IdentifyHydrationRetreatSubImages( postTreatmentImage, edgeRegions anchorRegions, natDifferences);} / / 1. Establish the correspondence between the dark target area and the anchoring area. private Dictionary<Rectangle, Rectangle> MapAnchorRegions( Image<Gray, byte> preImage, Image<Gray, byte> postImage, Rectangle[] darkRegions, Rectangle roi) {var anchorMap = new Dictionary<Rectangle, Rectangle> (); foreach (var region in darkRegions) / / Simplified correspondence - mapping based on relative position / / (In practice, more complex registration algorithms may be required) var anchorRegion = new Rectangle( roi.X + (region.X - preImage.Width / 2) * roi.Width / preImage.Width, roi.Y + (region.Y - preImage.Height / 2) * roi.Height / preImage.Height, region.Width * roi.Width / preImage.Width, region.Height * roi.Height / preImage.Height); / / Ensure the anchored area is within the ROI range anchorRegion.Intersect(roi); anchorMap.Add(region, anchorRegion);} return anchorMap;} / / 2. Calculate the NAT grayscale difference private Dictionary<Rectangle, double> CalculateNatDifferences( Image<Gray, byte> preImage, Image<Gray, byte> postImage, Dictionary<Rectangle, Rectangle> anchorRegions) {var natDiffs = new Dictionary<Rectangle, double> (); foreach (var pair in anchorRegions) {var darkRegion = pair.Key; var anchorRegion = pair.Value; / / Get the maximum grayscale value of the dark target area byte taMax = GetRegionMaxGrayValue(preImage, darkRegion); / / Get the minimum grayscale value of the anchored area byte anMin = GetRegionMinGrayValue(postImage, anchorRegion); / / Calculate NAT grayscale difference double natDiff = Math.Abs(taMax - anMin); natDiffs.Add(darkRegion, natDiff); }return natDiffs;} / / 3. Edge detection and region segmentation private List <Image<Gray, byte> >DetectAndDivideEdgeRegions( Image<Gray, byte> postImage, Rectangle roi) / / Extract ROI region var roiImage = postImage.Copy(roi); / / Using Canny edge detection var edges = new Image<Gray, byte> (roiImage.Size); CvInvoke.Canny(roiImage, edges, 50, 150); / / Find the outline var contours = new VectorOfVectorOfPoint(); CvInvoke.FindContours(edges, contours, null,Emgu.CV.CvEnum.RetrType.List, Emgu.CV.CvEnum.ChainApproxMethod.ChainApproxSimple); / / Divide the region to be tested (simplified processing; in practice, a more complex region segmentation algorithm may be required). var dividedRegions = new List <Image<Gray, byte> >(); for (int i = 0; i <contours.Size; i++) {var contour = contours[i]; var rect = CvInvoke.BoundingRectangle(contour); rect.Intersect(roi); dividedRegions.Add(postImage.Copy(rect)); } return dividedRegions;} / / 4. Identify tumor retraction sub-images private List <rectangle>IdentifyHydrationRetreatSubImages( Image<Gray, byte>postImage, List<Image<Gray, byte>>edgeRegions, Dictionary<Rectangle, Rectangle>anchorRegions, Dictionary<Rectangle, double>natDifferences) { var result = new List <rectangle>(); foreach (var region in edgeRegions) / / Calculate the average gray level of the area to be measured double regionMean = CalculateMeanGrayValue(region); foreach (var anchorPair in anchorRegions) {var anchorRegion = anchorPair.Value; double natDiff = natDifferences[anchorPair.Key]; / / Calculate the average gray level of the anchored area double anchorMean = CalculateMeanGrayValue(postImage.Copy(anchorRegion)); / / Conditions if (Math.Abs(regionMean - anchorMean) <natDiff) {result.Add(region.ROI); break; / / Once a match is found, the inspection of other anchored regions can be stopped. }}} return result; / / Auxiliary method: Get the maximum grayscale value of the region private byte GetRegionMaxGrayValue(Image<Gray, byte> image, Rectangleregion) {byte max = 0; var roi = image.Copy(region); for (int y = 0; y <roi.Height; y++) {for (int x = 0; x <roi.Width; x++) {byte value = roi.Data[y, x, 0]; If (value > max), then max = value; }}return max;} / / Helper method: Get the minimum grayscale value of the region private byte GetRegionMinGrayValue(Image<Gray, byte> image, Rectangleregion) {byte min = 255; var roi = image.Copy(region); for (int y = 0; y <roi.Height; y++) {for (int x = 0; x <roi.Width; x++) {byte value = roi.Data[y, x, 0]; if (value <min) min = value; }}return min;} / / Auxiliary method: Calculate the average gray value of the region private double CalculateMeanGrayValue(Image<Gray, byte> image) {var mean = new MCvScalar(); mean = CvInvoke.Mean(image); return mean.V0;}}.
[0042] Furthermore, in S400, the method for reconstructing the boundaries of the tumor regression sub-images in the post-treatment image dataset to obtain the reconstructed image dataset is as follows: Let P1 be the point with the smallest gray value on the boundary line of the tumor regression sub-image in the post-treatment image dataset, and P2 be the point with the largest gray value. Then, P1 and P2 divide the boundary line of the tumor regression sub-image into two curve segments. The shorter curve segment is taken as the fuzzy boundary line, and the longer curve segment is taken as the stable boundary line. Let PA1 be the point with the smallest absolute value of the difference between the gray value of each point on the anchored region boundary line and P1, and PA2 be the point with the smallest absolute value of the difference between the gray value of each point on the anchored region boundary line and P2. Then, PA1 and PA2 divide the anchored region boundary line into two curve segments. The shorter curve segment is taken as the mapping boundary line. Copying the mapping boundary line yields a replacement curve segment. Scaling the replacement curve segment to the same size as the blurred boundary line yields a clear boundary line. Deleting the blurred boundary line of the tumor shrinkage sub-image leaves a stable boundary line. After moving the clear boundary line to the positions where its two endpoints coincide with points P1 and P2 respectively, the clear boundary line and the stable boundary line together form the new boundary line of the tumor regression sub-image. The reconstructed image dataset consists of tumor shrinkage sub-images obtained from each new boundary line.
[0043] Note: The case study in the post-treatment image dataset is: a 47-year-old female breast cancer patient with right-sided breast cancer; for example... Figure 2 The image shown is a pattern of the post-treatment image dataset before reconstruction. The boundary between the tumor and the surrounding normal breast tissue is unclear. Figure 2 The white arrow in the middle indicates the location of the tumor. Figure 3 The image shown is a pattern of the reconstructed image dataset; the tumor is clearly demarcated from the surrounding normal breast tissue. Figure 3 (The white arrow in the middle indicates the location of the tumor). Using subsequent deep learning models to extract features and predict the efficacy of NAT, it can be clearly seen that the boundary area is clearer and more significant, making it easier for the deep learning model to identify.
[0044] The key C# source code describing the method for reconstructing the boundaries of tumor regression sub-images in the post-treatment image dataset is as follows: public class BoundaryRefiner {public List <Image<Gray, byte> >ReconstructHydrationImages( Image<Gray, byte> postTreatmentImage, List <rectangle>hydrationSubImages, Dictionary<Rectangle, Rectangle> anchorRegions) {var reconstructedImages = new List <Image<Gray, byte> >(); foreach (var subImageRect in hydrationSubImages) / / 1. Extract tumor regression sub-images and corresponding anchoring regions var subImage = postTreatmentImage.Copy(subImageRect); var correspondingAnchor = anchorRegions.FirstOrDefault(x => x.Value.IntersectsWith(subImageRect)).Value; if (correspondingAnchor.IsEmpty) continue? / / 2. Handling Boundaries var processedSubImage = ProcessBoundary( postTreatmentImage, subImage, correspondingAnchor); reconstructedImages.Add(processedSubImage);} return reconstructedImages; private Image<Gray, byte> ProcessBoundary( Image<Gray, byte> fullImage, Image<Gray, byte> subImage, Rectangle anchorRegion) / / Obtain the contour of the tumor regression sub-image var subImageContour = GetImageContour(subImage); var anchorContour = GetImageContour(fullImage.Copy(anchorRegion)); / / Find key points on the boundary var (p1, p2) = FindBoundaryExtremePoints(subImageContour); var (pa1, pa2) = FindAnchorBoundaryPoints(anchorContour, p1, p2); / / Divide the boundary curve segment var (fuzzyBoundary, stableBoundary) = SplitBoundary(subImageContour,p1, p2); var (mappingBoundary, _) = SplitBoundary(anchorContour, pa1, pa2); / / Create clear boundaries var clearBoundary = CreateClearBoundary(mappingBoundary, fuzzyBoundary); / / Reconstruct the boundary var newBoundary = ReconstructBoundary(stableBoundary, clearBoundary,p1, p2); / / Apply new boundaries return ApplyNewBoundary(subImage, newBoundary);} private VectorOfPoint GetImageContour(Image<Gray, byte> image) {var edges = new Image<Gray, byte> (image.Size); CvInvoke.Canny(image, edges, 50, 150); var contours = new VectorOfVectorOfPoint(); CvInvoke.FindContours(edges, contours, null, Emgu.CV.CvEnum.RetrType.External, Emgu.CV.CvEnum.ChainApproxMethod.ChainApproxSimple); return contours.Size>0 ? contours[0] : new VectorOfPoint();} private (Point p1, Point p2) FindBoundaryExtremePoints(VectorOfPointcontour) {if (contour.Size == 0) return (Point.Empty, Point.Empty); byte minVal = 255, maxVal = 0; Point p1 = new Point(), p2 = new Point(); for (int i = 0; i <contour.Size; i++) {var point = contour[i]; / / Assuming the grayscale value of the pixel at this location has already been saved byte val = GetGrayValueAtPoint(point); / / This method needs to be actually implemented. if (val <minVal) { minVal = val; p1 = point;} if (val>maxVal) { maxVal = val; p2 = point;} }return (p1, p2);} private (Point pa1, Point pa2) FindAnchorBoundaryPoints(VectorOfPointcontour, Point p1, Point p2) {if (contour.Size == 0) return (Point.Empty, Point.Empty); byte p1Val = GetGrayValueAtPoint(p1); byte p2Val = GetGrayValueAtPoint(p2); int minDiff1 = int.MaxValue, minDiff2 = int.MaxValue; Point pa1 = new Point(), pa2 = new Point(); for (int i = 0; i<contour.Size; i++) {var point = contour[i]; byte val = GetGrayValueAtPoint(point); int diff1 = Math.Abs(val - p1Val); int diff2 = Math.Abs(val - p2Val); if (diff1<minDiff1) { minDiff1 = diff1; pa1 = point;} if (diff2<minDiff2) { minDiff2 = diff2; pa2 = point;} }return (pa1, pa2);} private (VectorOfPoint shorter, VectorOfPoint longer) SplitBoundary( VectorOfPoint contour, Point p1, Point p2) { / / Find the index positions of p1 and p2 in the contour int idx1 = -1, idx2 = -1; for (int i = 0; i<contour.Size; i++) {if (contour[i].Equals(p1)) idx1 = i; if (contour[i].Equals(p2)) idx2 = i;} if (idx1 == -1 || idx2 == -1 || idx1 == idx2) return (new VectorOfPoint(), new VectorOfPoint()); / / Ensure idx1 <idx2 if (idx1>idx2) {(idx1, idx2) = (idx2, idx1); (p1, p2) = (p2, p1);} / / Create two curve segments var segment1 = new VectorOfPoint(); var segment2 = new VectorOfPoint(); / / Divide the outline into two parts for (int i = 0; i <contour.Size; i++) {if (i>= idx1&&i<= idx2) segment1.Push(new Point[] { contour[i]}); else segment2.Push(new Point[] { contour[i]});} / / Return the shorter and longer portions return segment1.Size <segment2.Size ? (segment1, segment2) :(segment2, segment1);} private VectorOfPoint CreateClearBoundary( VectorOfPoint mappingBoundary, VectorOfPoint fuzzyBoundary / / Calculate scaling based on length double scaleFactor = (double)fuzzyBoundary.Size / mappingBoundary.Size; / / Copy and scale the map boundaries var scaledMapping = new VectorOfPoint(); for (int i = 0; i <mappingBoundary.Size; i++) {var scaledPoint = new Point( (int)(mappingBoundary[i].X * scaleFactor), (int)(mappingBoundary[i].Y * scaleFactor)); scaledMapping.Push(new Point[] { scaledPoint}); }return scaledMapping; }private VectorOfPoint ReconstructBoundary( VectorOfPoint stableBoundary VectorOfPoint clearBoundary, Point p1, Point p2) Move clearBoundary to a position that overlaps with p1 and p2. var movedClearBoundary = TranslateBoundary(clearBoundary, p1, p2); / / Merge stability boundary and clear boundary var newBoundary = new VectorOfPoint(); newBoundary.Push(stableBoundary.ToArray()); newBoundary.Push(movedClearBoundary.ToArray()); return newBoundary; private VectorOfPoint TranslateBoundary( VectorOfPoint boundary, Point targetP1, Point targetP2) {if (boundary.Size == 0) return boundary. / / Assume the first point is the original p1 and the last point is the original p2. Point originalP1 = boundary[0]; Point originalP2 = boundary[boundary.Size - 1]; / / Calculate the movement vector int dx1 = targetP1.X - originalP1.X; int dy1 = targetP1.Y - originalP1.Y; int dx2 = targetP2.X - originalP2.X; int dy2 = targetP2.Y - originalP2.Y; / / Average moving vector int avgDx = (dx1 + dx2) / 2; int avgDy = (dy1 + dy2) / 2; / / Create the boundary after the move var movedBoundary = new VectorOfPoint(); for (int i = 0; i <boundary.Size; i++) {movedBoundary.Push(new Point[] { new Point boundary[i].X + avgDx, boundary[i].Y + avgDy) });}return movedBoundary;} private Image<Gray, byte> ApplyNewBoundary( Image<Gray, byte> image, VectorOfPoint boundary) {if (boundary.Size == 0) Return image; var mask = new Image<Gray, byte> (image.Size); CvInvoke.DrawContours(mask, new VectorOfVectorOfPoint(boundary), -1, new MCvScalar(255), -1); var result = image.Copy(); result.SetValue(0, mask.Not()); return result;}}.
[0045] Furthermore, in S500, the method for obtaining prediction results of breast cancer NAT efficacy using a pre-trained deep learning model based on the reconstructed image dataset is as follows: Feature vectors were obtained by using the ResNeXt network model to extract features from the reconstructed image dataset. By comparing the differences between the pre-treatment image dataset and the reconstructed image dataset, the corresponding difference comparison feature vector is obtained; The difference comparison feature vectors are fused to obtain the fused feature vector; Based on the fused feature vector, the classification and prediction results of the NAT efficacy of breast cancer by artificial intelligence are obtained.
[0046] Example 2 This embodiment 2 replaces the method of identifying tumor regression sub-images from the post-treatment image dataset based on dark target regions, as described in embodiment 1. Preferably, in S300, the method for identifying tumor regression sub-images in the post-treatment image dataset based on the dark target region is as follows: The corresponding position of the dark target region in the pre-treatment image dataset within the region of interest in the post-treatment image dataset is denoted as the anchor region; thus, each dark target region has a corresponding anchor region within the region of interest. The point with the highest gray value in the dark target region is located at position TaP in the region of interest; the point with the lowest gray value in the anchor region corresponding to the dark target region is located at position AnP in the region of interest; the direction from point TaP to point AnP is denoted as the retreat direction; multiple edge lines are detected in the region of interest using the edge detection operator, and the surface of the region of interest is divided into multiple regions to be tested using each edge line. The tumor regression pathways for each region to be tested were determined as follows: In each test region, the straight line connecting the point with the smallest gray value on the boundary of the test region and the point with the smallest gray value in the anchored region is the hydration tendency line; the straight line connecting the point with the largest gray value on the boundary of the test region and the point with the largest gray value in the anchored region is the retreat tendency line; the area between the hydration tendency line and the retreat tendency line is the tumor retreat channel of the test region; (the tumor retreat channel is the retreat direction along the tumor center between each test region and the anchored region. During the treatment process, especially after neoadjuvant chemotherapy (NAT), the tumor will retreat. Due to the significant reduction in tumor blood supply, cell death, stromal fibrosis, and inflammatory cell infiltration, the retreat area affects the appearance in MRI imaging. The gray values in this area are relatively discrete and irregular, and the tumor boundary is unclear. Existing deep learning models have difficulty distinguishing it from the surrounding tissue, resulting in low recognition accuracy). The determination of whether the current region to be tested is a tumor regression sub-image is based on the tumor regression pathway, specifically: Each region to be tested that intersects with the tumor regression channel of the current region to be tested is recorded as a feature region. Each feature region is judged in turn. If the gray value of a feature region is less than the gray value of the feature region before the regression direction and the gray value of the feature region is less than the gray value of the feature region after the regression direction, then the current region to be tested is recorded as a tumor regression sub-image.
[0047] Beneficial effects: Based on the gray values of all feature regions along the entire path to the tumor area, the changes in hydrogen proton content, blood supply, cell density, etc. in the fine tissues of key areas can be accurately monitored. From MRI images, tumor regression sub-images caused by NAT treatment can be accurately identified, which greatly improves the recognition accuracy of deep learning models in subsequent steps.
[0048] Example 3 This embodiment 3 replaces the method in embodiment 2 that determines whether the current region to be tested is a tumor regression sub-image based on the tumor regression channel. Specifically: Preferably, determining whether the current region to be tested is a tumor regression sub-image based on the tumor regression channel specifically involves: Each region intersects with the tumor regression channel of the current test region and is recorded as a feature region. The regression grayscale difference value of each feature region in the tumor regression channel is calculated, specifically: The feature region with the largest average gray value in the tumor retreat channel is designated as the peak feature region; the feature region with the smallest average gray value in the tumor retreat channel is designated as the valley feature region; the feature regions in the tumor retreat channel are sorted in order of retreat direction, and the index of the peak feature region in the tumor retreat channel is designated as UpI; the index of the valley feature region in the tumor retreat channel is designated as LowI. The average gray value of all feature regions from the current test area to UpI in the tumor retreat channel is used as the front retreat gray value; the average gray value of all feature regions from the current test area to LowI in the tumor retreat channel is used as the back retreat gray value; the difference between the front retreat gray value and the back retreat gray value is used as the retreat gray value difference value. For each feature region in the tumor regression channel, if the regression grayscale difference value of a feature region is less than the regression grayscale difference value of the feature region before the regression direction, and the regression grayscale difference value of the feature region is less than the regression grayscale difference value of the feature region after the regression direction, then the current test region is determined to be a tumor regression sub-image.
[0049] The key C# source code describing the specific implementation of the method for identifying tumor regression sub-images in the post-treatment image dataset based on dark target regions is as follows: public class HydrationRetreatAnalyzer {public List <rectangle>AnalyzeHydrationRetreat( Image<Gray, byte>retreatChannelImage, List <rectangle>testRegions, int retreatDirection = 0) / / Retreat direction (0 - horizontal to the right, 1 - vertical downward) / / 1. Identify feature regions List <featureregion>featureRegions = IdentifyFeatureRegions(retreatChannelImage, testRegions); / / 2. Sort feature regions by retreat direction SortFeatureRegions(featureRegions, retreatDirection); / / 3. Calculate the grayscale difference value of each feature region. CalculateRetreatGrayDifferences(featureRegions); / / 4. Identify tumor retraction sub-images List <rectangle>hydrationSubImages = IdentifyHydrationSubImages(featureRegions); return hydrationSubImages;} / / 1. Identify feature regions private List <featureregion>IdentifyFeatureRegions( Image<Gray, byte>retreatChannelImage, List <rectangle>testRegions) {var featureRegions = new List <featureregion>(); foreach (var region in testRegions) / / Calculate the average gray value of the region var roi = retreatChannelImage.Copy(region); var mean = CvInvoke.Mean(roi); featureRegions.Add(new FeatureRegion {Region = region, AverageGrayValue = mean.V0});} return featureRegions;} / / 2. Sort feature regions by retreat direction private void SortFeatureRegions(List <featureregion>regions, intretreatDirection) {if (retreatDirection == 0) / / Horizontal to the right {regions.Sort((a, b) =>a.Region.X.CompareTo(b.Region.X));} else / / Vertically downward {regions.Sort((a, b) =>a.Region.Y.CompareTo(b.Region.Y));} / / Set serial number for (int i = 0; i <regions.Count; i++) {regions[i].SequenceIndex = i; }} / / 3. Calculate the grayscale difference value of each feature region. private void CalculateRetreatGrayDifferences(List <featureregion>featureRegions) {if (featureRegions.Count == 0) return; / / Identify peak and trough characteristic regions var peakRegion = featureRegions.OrderByDescending(r =>r.AverageGrayValue).First(); var valleyRegion = featureRegions.OrderBy(r =>r.AverageGrayValue).First(); int peakIndex = peakRegion.SequenceIndex; int valleyIndex = valleyRegion.SequenceIndex; / / Calculate the shrinkage grayscale difference value for each feature region foreach (var currentRegion in featureRegions) {int currentIndex = currentRegion.SequenceIndex; / / Calculate the front shrinkage grayscale double frontRetreatGray = CalculateAverageGrayBetween( featureRegions, currentIndex, peakIndex); / / Calculate the grayscale value after shrinkage double rearRetreatGray = CalculateAverageGrayBetween( featureRegions, currentIndex, valleyIndex); / / Calculate the grayscale difference value currentRegion.RetreatGrayDiff = frontRetreatGray - rearRetreatGray; }} / / Auxiliary method: Calculate the average gray value between two serial numbers private double CalculateAverageGrayBetween( List <featureregion>regions, int startIndex, int endIndex) {if (regions.Count == 0) return 0; if (startIndex == endIndex) return regions[startIndex].AverageGrayValue; int step = startIndex <endIndex ? 1 : -1; int count = 0; double sum = 0; for (int i = startIndex; i != endIndex; i += step) {sum += regions[i].AverageGrayValue; count++;} / / Grayscale value including end position sum += regions[endIndex].AverageGrayValue; count++; return sum / count;} / / 4. Identify tumor retraction sub-images private List <rectangle>IdentifyHydrationSubImages(List <featureregion>featureRegions) {var result = new List <rectangle>(); for (int i = 1; i <featureRegions.Count - 1; i++) {var current = featureRegions[i]; var prev = featureRegions[i - 1]; var next = featureRegions[i + 1]; / / Judgment condition: The current difference value is less than the difference value of the adjacent regions before and after it. if (current.RetreatGrayDiff <prev.RetreatGrayDiff&& current.RetreatGrayDiff <next.RetreatGrayDiff) {result.Add(current.Region); }} return result;}}.
[0050] To evaluate the applicability of the model in clinical practice, with the consent and authorization of the breast cancer patients, MRI images of lesions before NAT treatment were randomly selected from 50 anonymous breast cancer patients to form a pre-treatment image dataset, and MRI images of lesions during the early stages of NAT treatment were selected to form a post-treatment image dataset.
[0051] The pre-treatment image dataset and post-treatment image dataset are multimodal MRI images collected from various breast cancer patients at two time points: before NAT (C0) and in the early treatment stage (C2), respectively, covering three sequences (T2WI, DWI, and T1+C). N4 bias field correction, three-dimensional isotropic resampling, grayscale normalization, and discretization were employed to unify imaging characteristics and reduce cross-device and cross-center interference. A dataset of pre-treatment images of 100 randomly selected breast cancer patients was used for data comparison in subsequent application example 1 and comparative example 1. Application Example 1: Input the pre-treatment image dataset and the reconstructed image dataset generated in Example 1 into the ResNeXt network model for prediction, generating a prediction result for each sample. Record the prediction result and corresponding probability value for each sample.
[0052] Comparative Example 1: The pre-treatment and post-treatment image datasets were input into the ResNeXt network model for prediction, generating prediction results for each sample. The prediction result and corresponding probability value for each sample were recorded.
[0053] Experimental evaluation: This invention uses a pre-trained ResNeXt network model to predict the reconstructed image dataset and post-treatment image dataset in Application Example 1 and Comparative Example 1, respectively, and compares the results with the subjective visual evaluation of clinicians. When extracting breast cancer lesions using the unprocessed original post-treatment image dataset, some pixels at the tumor boundary are misclassified as non-tumor pixels. Tumor regression is affected by interference from adjacent regions, leading to blurred tumor boundaries and making it impossible to distinguish the tumor from surrounding tissues. This reduces the accuracy of predicting the boundaries of some breast cancer lesions. In Application Example 1 and Comparative Example 1, the two-sided statistical significance level was set to 0.05.
[0054] The reconstructed image dataset, due to its boundary correction, enables the ResNeXt network model to learn and extract rich feature information, resulting in better boundary prediction performance for breast cancer lesions.
[0055] The accuracy, sensitivity, and specificity of application example 1 were 0.662 (0.612-0.876), 0.562 (0.476-0.787), and 0.756 (0.718-0.956), respectively. The accuracy, sensitivity, and specificity of Comparative Example 1 were 0.742 (0.612-0.876), 0.692 (0.476-0.787), and 0.824 (0.718-0.956), respectively.
[0056] Note: Accuracy refers to the proportion of samples correctly predicted by the classification model out of the total number of samples; the higher the accuracy, the better the overall performance of the model.
[0057] Sensitivity, also known as recall or true positive rate, refers to the proportion of samples correctly predicted as positive by a classification model out of all positive samples. The higher the sensitivity, the stronger the model's ability to identify positive samples.
[0058] Specificity refers to the proportion of samples that the classification model correctly predicts as negative examples out of all negative examples. The higher the specificity, the stronger the model's ability to identify negative examples.
[0059] Therefore, it can be seen that, for the same ResNeXt network model, the segmentation performance of the reconstructed image dataset is better than that of the post-treatment image dataset, and all performance indicators of the reconstructed image dataset are better than those of the post-treatment image dataset.
[0060] This disclosure presents a structural diagram of the post-treatment image dataset AT efficacy prediction system. The artificial intelligence-based breast cancer NAT efficacy prediction system of this embodiment includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps described in the above-described artificial intelligence-based breast cancer NAT efficacy prediction system embodiment.
[0061] The system includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program in units of the following system: The image data acquisition unit is used to acquire MRI images of breast cancer patients before NAT treatment to form a pre-treatment image dataset, and to acquire MRI images of the early stage of NAT treatment to form a post-treatment image dataset. The data registration and annotation unit is used to register the pre-treatment image dataset and the post-treatment image dataset respectively, and then annotate the corresponding lesion locations as regions of interest. The tumor regression recognition unit is used to locate the core region of the tumor in the region of interest of the pre-treatment image dataset as the dark target region, and to identify the tumor regression sub-image in the post-treatment image dataset based on the dark target region. The regression boundary reconstruction unit is used to reconstruct the boundaries of the tumor regression sub-images in the post-treatment image dataset to obtain the reconstructed image dataset. The efficacy prediction unit is used to obtain the prediction results of the efficacy of breast cancer NAT based on the reconstructed image dataset and a pre-trained deep learning model.
[0062] The AI-based breast cancer NAT efficacy prediction system can run on computing devices such as desktop computers, laptops, PDAs, and cloud servers. The system that can run on the AI-based breast cancer NAT efficacy prediction system may include, but is not limited to, processors and memory. Those skilled in the art will understand that the examples described are merely illustrations of an AI-based breast cancer NAT efficacy prediction system and do not constitute a limitation on the system. It may include more or fewer components, or a combination of certain components, or different components. For example, the AI-based breast cancer NAT efficacy prediction system may also include input / output devices, network access devices, buses, etc.
[0063] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. This processor is the control center of the AI-based breast cancer NAT efficacy prediction system, connecting all parts of the system via various interfaces and lines.
[0064] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the AI-based breast cancer NAT efficacy prediction system by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.
[0065] Although the description of this disclosure has been quite detailed and particularly of several described embodiments, it is not intended to limit itself to any of these details or embodiments or any particular embodiments, thereby effectively covering the intended scope of this disclosure. Furthermore, the disclosure has been described above with respect to embodiments foreseeable by the inventors for the purpose of providing a useful description, and non-substantial modifications to this disclosure that have not yet been foreseen may still represent equivalent modifications.< / rectangle> < / featureregion> < / rectangle> < / featureregion> < / featureregion> < / featureregion> < / featureregion> < / rectangle> < / featureregion> < / rectangle> < / featureregion> < / rectangle> < / rectangle> < / rectangle> < / rectangle> < / rectangle>
Claims
1. An artificial intelligence-based method for predicting the efficacy of breast cancer treatment using NAT (Non-Analogous Tolerance) therapy, characterized in that, The method includes the following steps: S100: Obtain MRI images of breast cancer patients' lesions before NAT treatment to form a pre-treatment image dataset, and obtain MRI images of the early stage of NAT treatment to form a post-treatment image dataset. S200: After image registration of the pre-treatment image dataset and the post-treatment image dataset, the corresponding lesion locations are labeled as regions of interest. S300: Locate the core region of the tumor in the region of interest of the pre-treatment image dataset as the dark target region, and identify the tumor regression sub-image in the post-treatment image dataset based on the dark target region; S400, the boundaries of the tumor regression sub-images in the post-treatment image dataset are reconstructed to obtain the reconstructed image dataset; S500, based on a reconstructed image dataset, uses a pre-trained deep learning model to obtain prediction results of the efficacy of NAT in breast cancer. In S300, the method for identifying tumor regression sub-images in the post-treatment image dataset based on dark target regions is as follows: The corresponding position of the dark target region in the pre-treatment image dataset within the region of interest in the post-treatment image dataset is denoted as the anchor region; thus, each dark target region has a corresponding anchor region within the region of interest. The point with the highest gray value in the dark target region is designated as TaP in the region of interest; the point with the lowest gray value in the anchored region corresponding to the dark target region is designated as AnP in the region of interest; the direction from point TaP to point AnP is denoted as the retreat direction; multiple edge lines are detected in the region of interest using an edge detection operator, and the surface of the region of interest is divided into multiple test regions by each edge line; the tumor retreat channel of each test region is determined; and the current test region is determined as a tumor retreat sub-image based on the tumor retreat channel. In S400, the method for reconstructing the boundary of the tumor regression sub-image in the post-treatment image dataset to obtain the reconstructed image dataset is as follows: P1 is the point with the smallest gray value on the boundary line of the tumor regression sub-image in the post-treatment image dataset, and P2 is the point with the largest gray value. P1 and P2 divide the boundary line of the tumor regression sub-image into two curve segments. The shorter curve segment is taken as the fuzzy boundary line, and the longer curve segment is taken as the stable boundary line. PA1 is the point with the smallest absolute value of the difference between the gray values of each point on the anchored region boundary line and point P1. The point with the smallest absolute value is PA2. PA1 and PA2 divide the anchored region boundary line into two curve segments. The shorter curve segment is taken as the mapping boundary line. The mapping boundary line is copied to obtain the replacement curve segment. The replacement curve segment is scaled to the same size as the blurred boundary line to obtain the clear boundary line. The blurred boundary line of the tumor shrinkage sub-image is deleted to leave the stable boundary line. The clear boundary line is moved to the position where the two endpoints coincide with the points P1 and P2 respectively. The clear boundary line and the stable boundary line constitute the new boundary line of the tumor shrinkage sub-image. The tumor shrinkage sub-images with each of the obtained new boundary lines form the reconstructed image dataset.
2. According to the artificial intelligence-based breast cancer NAT efficacy prediction method according to claim 1, in S300, the method of locating the tumor core region of the region of interest in the pretreatment image dataset as the dark target region is as follows: the region of interest is grayscaled to obtain a grayscale image, the grayscale image is divided into sub-regions, the sub-region with the smallest average grayscale value of each pixel in each sub-region is recorded as the tumor core region, and the tumor core region is recorded as the dark target region.
3. In the artificial intelligence-based breast cancer NAT efficacy prediction method according to claim 1, in S300, the method of identifying tumor regression sub-images in the post-treatment image dataset based on dark target regions is replaced by: The corresponding position of the dark target region in the pre-treatment image dataset within the region of interest in the post-treatment image dataset is denoted as the anchor region; thus, each dark target region corresponds to an anchor region in the post-treatment image dataset. The maximum grayscale value of each point in the dark target area is obtained as TaMax; The minimum grayscale value of each point in the anchored region corresponding to the dark target region is AnMin; the absolute value of the difference between TaMax and AnMin is recorded as the NAT grayscale difference. Multiple edge lines are obtained by detecting the region of interest in the post-treatment image dataset using an edge detection operator. The surface of the region of interest is then divided into multiple regions to be tested using each edge line. The average gray level of each pixel in the area to be measured is recorded as the gray level of the area to be measured. The grayscale of each test area is judged sequentially. If the absolute value of the difference between the grayscale of the test area and the average grayscale value of all points on the anchored area is less than the NAT grayscale difference, then the test area corresponding to the grayscale of the test area is marked as a tumor shrinkage sub-image.
4. The method for predicting the efficacy of breast cancer NAT based on artificial intelligence according to claim 1, wherein the tumor regression channel of each test region is determined as follows: in each test region, the straight line connecting the point with the smallest gray value on the boundary of the test region and the point with the smallest gray value in the anchor region is the hydration tendency line; the straight line connecting the point with the largest gray value on the boundary of the test region and the point with the largest gray value in the anchor region is the regression tendency line; the area between the hydration tendency line and the regression tendency line is the tumor regression channel of the test region.
5. The artificial intelligence-based method for predicting the efficacy of NAT in breast cancer according to claim 1, wherein determining whether the current test area is a tumor regression sub-image based on the tumor regression channel specifically involves: recording each test area that intersects with the tumor regression channel of the current test area as a feature area; judging each feature area sequentially; if there exists a feature area whose gray value is less than the gray value of the feature area before the regression direction, and whose gray value is less than the gray value of the feature area after the regression direction, then the current test area is recorded as a tumor regression sub-image.
6. The artificial intelligence-based breast cancer NAT efficacy prediction method according to claim 1, specifically determining whether the current test region is a tumor regression sub-image based on the tumor regression channel, is as follows: Each region intersects with the tumor regression channel of the current test region and is recorded as a feature region. The regression grayscale difference value of each feature region in the tumor regression channel is calculated, specifically: The feature region with the largest average gray value in the tumor retreat channel is designated as the peak feature region; the feature region with the smallest average gray value in the tumor retreat channel is designated as the valley feature region; the feature regions in the tumor retreat channel are sorted in order of retreat direction, and the index of the peak feature region in the tumor retreat channel is designated as UpI; the index of the valley feature region in the tumor retreat channel is designated as LowI. The average gray value of all feature regions from the current test area to UpI in the tumor retreat channel is used as the front retreat gray value; the average gray value of all feature regions from the current test area to LowI in the tumor retreat channel is used as the back retreat gray value; the difference between the front retreat gray value and the back retreat gray value is used as the retreat gray value difference value. For each feature region in the tumor regression channel, if the regression grayscale difference value of a feature region is less than the regression grayscale difference value of the feature region before the regression direction, and the regression grayscale difference value of the feature region is less than the regression grayscale difference value of the feature region after the regression direction, then the current test region is determined to be a tumor regression sub-image.
7. The method for predicting the efficacy of breast cancer NAT based on artificial intelligence according to claim 1, in S500, the method for obtaining the prediction result of breast cancer NAT efficacy using a pre-trained deep learning model based on the reconstructed image dataset is as follows: feature vectors are obtained by using the ResNeXt network model to extract features from the reconstructed image dataset. By comparing the differences between the pre-treatment image dataset and the reconstructed image dataset, the corresponding difference comparison feature vector is obtained; The difference comparison feature vectors are fused to obtain the fused feature vector; Based on the fused feature vector, the classification and prediction results of the NAT efficacy of breast cancer by artificial intelligence are obtained.
8. An artificial intelligence-based NAT efficacy prediction system for breast cancer, characterized in that, The AI-based breast cancer NAT efficacy prediction system includes: a processor, a memory, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of the AI-based breast cancer NAT efficacy prediction method of claim 1.