Breast cancer NAT curative effect prediction method and system based on artificial intelligence

Abstract

The invention discloses a breast cancer NAT curative effect prediction method and system based on artificial intelligence, and belongs to the technical field of medical information processing, and the method comprises the steps: carrying out the image registration of a pre-treatment image data set and a post-treatment image data set, and marking a corresponding focus position as a region of interest; positioning a tumor body core region of a region of interest of the pre-treatment image data set as a dark target region, and identifying a tumor recession sub-image of the post-treatment image data set according to the dark target region; reconstructing the boundary of the tumor recession sub-image of the post-treatment image data set to obtain a reconstructed image data set; utilizing a pre-trained deep learning model to obtain a prediction result of the breast cancer NAT curative effect; the problem that the boundary between the tumor and the surrounding normal mammary tissue is not clear due to the recession phenomenon is eliminated, so that the tumor recession sub-image of the to-be-recognized area can be used for a deep learning network, and the accuracy and reliability of a prediction model are improved.

Description

TECHNICAL FIELD

[0001] The present disclosure belongs to the technical field of medical information processing, and particularly relates to a breast cancer NAT efficacy prediction method and system based on artificial intelligence. BACKGROUND

[0002] In the field of tumor treatment, especially in the neoadjuvant therapy (NAT) of breast cancer, imaging methods play an increasingly important role in efficacy evaluation and individualized treatment strategy development.

[0003] Breast cancer is a highly heterogeneous malignant tumor, and the response to NAT varies significantly among individuals. Whether achieving pathologic complete response (pCR) during treatment is considered a key indicator for measuring the effectiveness of NAT and predicting patient prognosis. However, the current accuracy of early pCR prediction is still not ideal, making it difficult to meet the needs of individualized precision treatment in the clinic. Therefore, how to use imaging methods to predict the efficacy of breast cancer NAT early, accurately, and non-invasively during the NAT process is an important problem that needs to be solved in the current clinical and research fields.

[0004] Currently, clinical NAT efficacy evaluation mainly relies on radiologists observing MRI images after NAT treatment to make subjective judgments. This evaluation method is easily influenced by the experience of doctors, has poor stability, and is difficult to systematically reflect the dynamic evolution process of tumor morphology, function, and microscopic tissue characteristics during treatment. In particular, in multi-modal MRI images such as dynamic contrast-enhanced MRI (DCE-MRI) and diffusion-weighted imaging (DWI), the information fusion between different modalities has not been fully explored, which severely limits the accuracy of breast cancer NAT efficacy prediction and its clinical promotion.

[0005] During the NAT treatment process, the tumor will shrink. However, under the action of NAT drugs, the shrinking tumor will have the following changes: on one hand, the blood supply to the tumor tissue decreases, and the degree of enhancement in dynamic contrast-enhanced MRI (DCE-MRI) images decreases significantly, making it difficult to distinguish the tumor from the surrounding normal breast tissue; on the other hand, due to cell necrosis, interstitial fibrous tissue proliferation, and inflammatory cell infiltration within the tumor, the tumor signal decreases on T2-weighted imaging (T2WI) and diffusion-weighted imaging (DWI), reducing the contrast between the tumor and the surrounding normal breast tissue, further blurring the boundary between the tumor and the surrounding tissue. Therefore, the pathological histological changes that occur in breast cancer during NAT affect the identification of the boundary of the tumor after shrinkage, reducing the accuracy of existing artificial intelligence-based breast cancer NAT efficacy prediction systems. SUMMARY

[0006] The present application aims to provide an artificial intelligence-based breast cancer NAT efficacy prediction method and system to solve one or more technical problems in the prior art and at least provide a beneficial option or create conditions.

[0007] To achieve the above-mentioned purpose, according to an aspect of the present disclosure, an artificial intelligence-based breast cancer NAT efficacy prediction method is provided, the method comprising the following steps: S100, acquiring MRI images of a breast cancer patient's lesion before NAT treatment to form a pre-treatment image data set, and acquiring MRI images of an early stage of NAT treatment to form a post-treatment image data set; S200, after image registration of the pre-treatment image data set and the post-treatment image data set, respectively, labeling the corresponding lesion position as a region of interest; S300, positioning the core region of the region of interest of the pre-treatment image data set as a dark target region, and identifying a tumor shrinkage sub-image of the post-treatment image data set according to the dark target region; S400, reconstructing the boundary of the tumor shrinkage sub-image of the post-treatment image data set to obtain a reconstructed image data set; S500, based on the reconstructed image data set, using a pre-trained deep learning model to obtain a prediction result of the efficacy of breast cancer NAT.

[0008] Definition: Pre-treatment: pre-treatment refers to before NAT treatment; post-treatment: post-treatment refers to the early stage of NAT treatment.

[0009] Further, in S100, the method for acquiring the MRI images of the breast cancer patient's lesion before NAT treatment to form the pre-treatment image data set is: acquiring the MRI images of the breast cancer patient's lesion before NAT treatment to form the pre-treatment image data set through a SIMENSE 3.0 nuclear magnetic resonance instrument.

[0010] Further, in S100, the method for acquiring the MRI images of the early stage of NAT treatment to form the post-treatment image data set is: acquiring the MRI images of the early stage of NAT treatment to form the post-treatment image data set through a SIMENSE 3.0 nuclear magnetic resonance instrument.

[0011] Further, in S100, the pre-treatment image data set and the post-treatment image data set have the same number of images and the same position.

[0012] The gray value of each sequence of MRI image is related to the water content, cell density and blood supply of tumor tissue. The area with high water content in tumor tissue shows high signal on T2WI, and the area with low water content shows low signal (high signal refers to the bright area on the magnetic resonance image, and low signal shows a darker or black area). The area with high cell density in the tumor shows high signal on the DWI image due to the limited free diffusion of water molecules; on the contrary, the area with low cell density shows low signal on the DWI. The area with rich blood supply in the tumor shows high enhancement (i.e. high signal) on DCE-MRI, and the area with poor blood supply shows low signal. See reference: Brown R W, Cheng Y C N, Haacke E, et al. Magnetic Resonance Imaging: Physical Principles and Sequence Design [J]. 1999. DOI:10.1002 / 9781118633953.

[0013] Further, in S300, the method of positioning the tumor core region of the pre-treatment image data set of the region of interest as the dark target region is: gray-scale the region of interest to obtain a gray-scale image, divide the gray-scale image into sub-regions, and take the sub-region with the minimum average gray value of each pixel in each sub-region as the tumor core region, and record the tumor core region as the dark target region.

[0014] Preferably, dividing the gray-scale image into sub-regions is to extract the edge lines of the gray-scale image by an edge detection operator after bottom-hat transformation to form a plurality of sub-regions.

[0015] Preferably, the tumor is automatically divided into 100 sub-regions by using a superpixel algorithm.

[0016] Preferably, the superpixel algorithm is a linear iterative clustering (SLIC) superpixel algorithm or a SEEDS (Superpixels Extracted via Energy-Driven Sampling) algorithm.

[0017] Preferably, the number of sub-regions is 100. Further, in S200, the method of labeling the corresponding lesion position as the region of interest is: based on the DCE enhanced second phase as the main reference, manually drawing a rectangular frame to cover the tumor whole by a professional physician, and then identifying the lesion edge by an edge detection operator to obtain the accurate contour of the lesion as the region of interest.

[0018] The edge detection operator includes Roberts operator and Prewitt operator, and the Roberts operator is preferred because the dark target area extracted above is used to accurately locate and correct the tumor edge in the following steps. Due to the strong invasiveness of the tumor, part of the tumor tissue extends into the normal breast tissue, and the tumor and the normal tissue exist alternately in the boundary area, resulting in a blurred boundary. The Prewitt operator has a noise suppression function, and the principle of noise suppression is to average the pixels. However, the pixel averaging is equivalent to low-pass filtering of the image, and the Roberts operator usually produces a wider response in the area near the tumor edge of the image, which can better identify the blurred boundary area caused by tumor invasiveness. Therefore, the Prewitt operator is not as good as the Roberts operator in locating the tumor edge.

[0019] Further, in S300, the method for identifying the tumor regression sub-image of the post-treatment image data set according to the dark target area is: The corresponding position of the dark target area in the pre-treatment image data set in the region of interest in the post-treatment image data set is recorded as an anchor region; each dark target area corresponds to an anchor region in the post-treatment image data set; The maximum gray value of each point in the dark target area is obtained as TaMax; the minimum gray value of each point in the anchor region corresponding to the dark target area is obtained as AnMin; the absolute value of the difference between TaMax and AnMin is recorded as NAT gray difference (NAT gray difference reflects the extreme difference degree of gray caused by the signal change of the tumor center before and after NAT treatment in MRI imaging); An edge detection operator is used to detect a plurality of edge lines in the region of interest in the post-treatment image data set, and the surface of the region of interest is divided into a plurality of to-be-tested regions by each edge line; the average gray value of each pixel in the to-be-tested region is recorded as to-be-tested region gray; The gray values of each to-be-tested region are sequentially judged, and if the absolute value of the difference between the to-be-tested region gray and the average gray value of all points on the anchor region is less than the NAT gray difference, the to-be-tested region corresponding to the to-be-tested region gray is marked as a tumor regression sub-image.

[0020] The above method screens the region where the tumor regression phenomenon occurs by dynamically calculating the NAT gray difference degree before and after NAT treatment. However, after NAT treatment, the blood supply of the tumor tissue is significantly reduced, and cell necrosis, interstitial fibrous tissue proliferation and inflammatory cell infiltration occur inside, resulting in a decrease in the gray value difference between the tumor region and the surrounding normal breast tissue on the MRI sequence images, and a blurred boundary between the tumor and the surrounding tissue. The tumor boundary is not clear due to the interference of adjacent regions, and a simple comparison of the gray value difference will result in a low accuracy of the tumor regression sub-image in the to-be-identified region.

[0021] Preferably, in S300, the method for identifying the tumor shrinkage sub-image of the post-treatment image dataset according to the dark target region is: The corresponding position of the dark target region in the pre-treatment image dataset in the region of interest in the post-treatment image dataset is recorded as an anchor region; each dark target region has a corresponding anchor region in the region of interest; The corresponding position of the point with the maximum gray value in the dark target region on the region of interest is recorded as TaP; the corresponding position of the point with the minimum gray value in the anchor region corresponding to the dark target region on the region of interest is recorded as AnP; the direction from point TaP to point AnP is recorded as the shrinkage direction; the surface of the region of interest is divided into a plurality of to-be-tested regions by a plurality of edge lines detected by an edge detection operator; Determine the tumor shrinkage channel of each to-be-tested region, specifically: In each to-be-tested region, the straight line connecting the point with the minimum gray value on the boundary of the to-be-tested region and the point with the minimum gray value in the anchor region is the hydration tendency line; the straight line connecting the point with the maximum gray value on the boundary of the to-be-tested region and the point with the maximum gray value in the anchor region is the shrinkage tendency line; the region between the hydration tendency line and the shrinkage tendency line is the tumor shrinkage channel of the to-be-tested region; (The tumor shrinkage channel is the shrinkage direction between each to-be-tested region and the anchor region along the center of the tumor, one side boundary of which is the region of abnormal water content after tumor shrinkage (hydration tendency line), and the other side boundary is the shrinkage boundary (shrinkage tendency line). During the treatment of the tumor, especially after neoadjuvant chemotherapy (NAT), the tumor will shrink, and the shrinkage region will have a significant decrease in tumor blood supply, cell death, interstitial fibrous tissue proliferation, and inflammatory cell infiltration, which will affect the performance in MRI imaging. The gray value of this region is relatively discrete and irregular, and the tumor boundary is not clear. Existing deep learning models are difficult to distinguish it from the surrounding tissue, and the recognition accuracy is low.); According to the tumor shrinkage channel, determine whether the current to-be-tested region is a tumor shrinkage sub-image, specifically: Record each to-be-tested region that has an intersection with the tumor shrinkage channel of the current to-be-tested region as a feature region, and sequentially determine each feature region. If the gray value of a feature region is less than the gray value of a feature region before the shrinkage direction, and the gray value of the feature region is less than the gray value of a feature region after the shrinkage direction, record the current to-be-tested region as a tumor shrinkage sub-image.

[0022] Beneficial effects: According to the gray value of all feature regions on the entire path to the tumor area, the change trend of the hydrogen proton content in the key region of the fine tissue can be accurately monitored, and the tumor shrinkage sub-image changed due to the NAT treatment can be accurately identified from the MRI image, greatly improving the identification accuracy of the deep learning model in the subsequent step.

[0023] If the shrinkage direction on the tumor shrinkage channel is discontinuous, the interval between the dark and light regions alternately jumps, and the size of the current to-be-detected region and the adjacent to-be-detected region is close, the above linear comparison is still prone to misjudgment, thereby reducing the recognition accuracy of the deep learning model in the subsequent step. The present application proposes the following method to solve the above problem. The specific method is: Preferably, whether the current to-be-detected region is a tumor shrinkage sub-image is determined according to the tumor shrinkage channel, specifically: Each to-be-detected region intersecting with the tumor shrinkage channel of the current to-be-detected region is recorded as a feature region, and the shrinkage gray difference value of each feature region in the tumor shrinkage channel is calculated, specifically: The feature region with the maximum average gray value in the tumor shrinkage channel is recorded as a peak feature region, the feature region with the minimum average gray value in the tumor shrinkage channel is recorded as a valley feature region, and each feature region in the tumor shrinkage channel is sequentially sorted according to the shrinkage direction, and the serial number of the peak feature region in each feature region in the tumor shrinkage channel is recorded as UpI, and the serial number of the valley feature region in each feature region in the tumor shrinkage channel is recorded as LowI. The average value of the gray values of all feature regions in the tumor shrinkage channel from the current to-be-detected region position to UpI is taken as the front shrinkage gray value, the average value of the gray values of all feature regions in the tumor shrinkage channel from the current to-be-detected region position to LowI is taken as the rear shrinkage gray value, and the difference between the front shrinkage gray value and the rear shrinkage gray value is taken as the shrinkage gray difference value. For each feature region in the tumor shrinkage channel, if the shrinkage gray difference value of the feature region is less than the shrinkage gray difference value of the feature region before it in the shrinkage direction, and the shrinkage gray difference value of the feature region is less than the shrinkage gray difference value of the feature region after it in the shrinkage direction, the current to-be-detected region is determined as a tumor shrinkage sub-image.

[0024] After locating all the tumor regression sub-images, since these images represent the insignificant volume reduction produced in the early stages of NAT treatment, and pathological changes such as reduced tumor blood supply after NAT treatment lead to unclear boundaries between the tumor and surrounding normal tissue, making them difficult to distinguish, the tumor regression sub-images need to be adjusted according to the following method to enable the deep learning model to accurately identify them: Furthermore, in S400, the method for reconstructing the boundaries of the tumor regression sub-images in the post-treatment image dataset to obtain the reconstructed image dataset is as follows: Let P1 be the point with the smallest gray value on the boundary line of the tumor regression sub-image in the post-treatment image dataset, and P2 be the point with the largest gray value. Then, P1 and P2 divide the boundary line of the tumor regression sub-image into two curve segments. The shorter curve segment is taken as the fuzzy boundary line, and the longer curve segment is taken as the stable boundary line. Let PA1 be the point with the smallest absolute value of the difference between the gray value of each point on the anchored region boundary line and P1, and PA2 be the point with the smallest absolute value of the difference between the gray value of each point on the anchored region boundary line and P2. Then, PA1 and PA2 divide the anchored region boundary line into two curve segments. The shorter curve segment is taken as the mapping boundary line. Copying the mapping boundary line yields a replacement curve segment. Scaling the replacement curve segment to the same size as the blurred boundary line yields a clear boundary line. Deleting the blurred boundary line of the tumor shrinkage sub-image leaves a stable boundary line. After moving the clear boundary line to the positions where its two endpoints coincide with points P1 and P2 respectively, the clear boundary line and the stable boundary line together form the new boundary line of the tumor regression sub-image. The reconstructed image dataset consists of tumor shrinkage sub-images obtained from each new boundary line.

[0025] Furthermore, in S500, the method for obtaining the prediction results of the efficacy of breast cancer NAT based on the reconstructed image dataset using a pre-trained deep learning model is as follows: feature vectors are obtained by using the ResNeXt network model to extract features from the reconstructed image dataset. By comparing the differences between the pre-treatment image dataset and the reconstructed image dataset, the corresponding difference comparison feature vector is obtained; The difference comparison feature vectors are fused to obtain the fused feature vector; Based on the fused feature vector, the classification and prediction results of the NAT efficacy of breast cancer by artificial intelligence are obtained.

[0026] Preferably, the deep learning model is the ResNeXt network model.

[0027] Preferably, the breast cancer NAT efficacy prediction system based on artificial intelligence of the present application can not only be used for breast cancer, but also can be used for deep temporal lobe tumors (tumors of brain structures such as amygdala and hippocampus), benign brain tumors, and brain tumor targeted therapy of brain diseases.

[0028] The present application also provides a breast cancer NAT efficacy prediction system based on artificial intelligence, which comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to realize the steps in the breast cancer NAT efficacy prediction method based on artificial intelligence, and the breast cancer NAT efficacy prediction system based on artificial intelligence can run in a computing device such as a desktop computer, a notebook computer, a palm computer and a cloud data center, and the executable system can comprise, but is not limited to, a processor, a memory, a server cluster, and the processor executes the computer program to run in the following system units: An image data acquisition unit is configured to acquire MRI images of a breast cancer patient before NAT treatment to form a pre-treatment image data set, and acquire MRI images in an early stage of NAT treatment to form a post-treatment image data set. A data registration and labeling unit is configured to perform image registration on the pre-treatment image data set and the post-treatment image data set respectively, and label the corresponding lesion positions as regions of interest. A tumor shrinkage recognition unit is configured to locate a tumor core region of the region of interest in the pre-treatment image data set as a dark target region, and recognize a tumor shrinkage sub-image of the post-treatment image data set according to the dark target region. A shrinkage boundary reconstruction unit is configured to reconstruct the boundary of the tumor shrinkage sub-image of the post-treatment image data set to obtain a reconstructed image data set. An efficacy result prediction unit is configured to obtain a prediction result of breast cancer NAT efficacy based on the reconstructed image data set and using a pre-trained deep learning model.

[0029] The present application provides a breast cancer NAT efficacy prediction method and system based on artificial intelligence, which can correct the tumor shrinkage phenomenon, eliminate the problem that the tumor position and the surrounding tissue cannot be distinguished due to the tumor shrinkage phenomenon, and make the tumor shrinkage sub-image of the region to be recognized available for a deep learning network, thereby improving the reliability of the extracted data for the prediction model. BRIEF DESCRIPTION OF DRAWINGS

[0030] The above and other features of this disclosure will become more apparent from the detailed description of the embodiments illustrated in conjunction with the accompanying drawings. In the accompanying drawings, the same reference numerals denote the same or similar elements. Obviously, the drawings described below are merely some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained from these drawings without any creative effort. In the drawings: Figure 1 The flowchart shown is a breast cancer NAT efficacy prediction method based on artificial intelligence; Figure 2 The image shown is a pattern of the post-treatment image dataset before reconstruction. Figure 3 The image shown is a diagram of the reconstructed image dataset. Figure 4 The diagram shows the structure of an AI-based breast cancer NAT efficacy prediction system. Detailed Implementation

[0031] The following will provide a clear and complete description of the concept, specific structure, and resulting technical effects of this disclosure in conjunction with the embodiments and accompanying drawings, so as to fully understand the purpose, solution, and effects of this disclosure. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0032] Example 1 like Figure 1 The diagram shows a flowchart of an AI-based method for predicting the efficacy of breast cancer treatment using NAT. The following section will combine... Figure 1 This invention describes an artificial intelligence-based method for predicting the efficacy of breast cancer NAT treatment according to an embodiment of the present invention. The method includes the following steps: S100: Obtain MRI images of breast cancer patients' lesions before NAT treatment to form a pre-treatment image dataset, and obtain MRI images of the early stage of NAT treatment to form a post-treatment image dataset. S200: After image registration of the pre-treatment image dataset and the post-treatment image dataset, the corresponding lesion locations are labeled as regions of interest. S300: Locate the core region of the tumor in the region of interest of the pre-treatment image dataset as the dark target region, and identify the tumor regression sub-image in the post-treatment image dataset based on the dark target region; S400, the boundaries of the tumor regression sub-images in the post-treatment image dataset are reconstructed to obtain the reconstructed image dataset; S500, based on a reconstructed image dataset, uses a pre-trained deep learning model to predict the efficacy of NAT in breast cancer treatment.

[0033] Furthermore, in S100, the method for obtaining MRI images of breast cancer patients' lesions before NAT treatment to form a pre-treatment image dataset is as follows: MRI images of breast cancer patients' lesions before NAT treatment are obtained using a SIMENSE 3.0 MRI scanner to form a pre-treatment image dataset.

[0034] Furthermore, in S100, the method for obtaining MRI images of the early stage of NAT treatment to form a post-treatment image dataset is as follows: MRI images of the early stage of NAT treatment are obtained using a SIMENSE 3.0 MRI scanner to form a post-treatment image dataset.

[0035] Furthermore, in S100, the pre-treatment image dataset and the post-treatment image dataset have the same number of images and are located in the same positions. The pre-treatment image dataset and the post-treatment image dataset are multimodal MRI images of breast cancer patients collected at two time points: before NAT (C0) and in the early treatment stage (C2), respectively, covering three sequences (T2WI, DWI, and T1+C). N4 bias field correction, three-dimensional isotropic resampling, grayscale normalization, and discretization are employed to unify imaging characteristics and reduce cross-device and cross-center interference.

[0036] Furthermore, in S300, the method for locating the tumor core region of the region of interest in the pre-treatment image dataset as the dark target region is as follows: the region of interest is grayscaled to obtain a grayscale image, the grayscale image is divided into sub-regions, and the sub-region with the smallest average grayscale value of each pixel in each sub-region is recorded as the tumor core region, and the tumor core region is recorded as the dark target region.

[0037] Preferably, a superpixel algorithm is used to automatically divide the tumor into 100 sub-regions and remove invalid voxel clusters.

[0038] The superpixel algorithm is the Linear Iterative Clustering (SLIC) superpixel algorithm.

[0039] Furthermore, in S200, the method for marking the corresponding lesion location as the region of interest is as follows: based on the second phase of DCE enhancement as the main reference, a rectangular frame covering the entire tumor area is manually drawn by a professional physician as the region of interest; each region of interest is adjusted to 256×256 pixels.

[0040] Furthermore, in S300, the method for identifying tumor regression sub-images in the post-treatment image dataset based on dark target regions is as follows: The corresponding position of the dark target region in the pre-treatment image dataset within the region of interest in the post-treatment image dataset is denoted as the anchor region; thus, each dark target region corresponds to an anchor region in the post-treatment image dataset. A maximum value of the gray scale of each point in the dark target region is obtained as TaMax; a minimum value of the gray scale of each point in the anchor region corresponding to the dark target region is obtained as AnMin; an absolute value of the difference between TaMax and AnMin is taken as NAT gray scale difference (the NAT gray scale difference reflects the extreme difference degree of the gray scale caused by the signal change of the tumor center before and after the NAT treatment in the MRI imaging); A plurality of edge lines are detected in the region of interest in the post-treatment image data set by using an edge detection operator, and the surface of the region of interest is divided into a plurality of to-be-measured regions by using each edge line; an average gray scale value of each pixel in the to-be-measured region is recorded as to-be-measured region gray scale; Each to-be-measured region gray scale is sequentially judged, and if the absolute value of the difference between the to-be-measured region gray scale and the average gray scale value of all points on the anchor region is less than the NAT gray scale difference, the to-be-measured region corresponding to the to-be-measured region gray scale is marked as a tumor shrinkage sub-image.

[0041] Among them, the key source code of the C# language description of the specific implementation of the method for identifying the tumor shrinkage sub-image of the post-treatment image data set according to the dark target region is as follows: public class ImageProcessor { / / Main processing method public void ProcessNATImages( Image<Gray, byte>preTreatmentImage, / / Pre-treatment image Image<Gray, byte>postTreatmentImage, / / Post-treatment image Rectangle[] darkTargetRegions, / / Dark target region in pre-treatment image Rectangle roi) / / Region of interest in post-treatment image { / / 1. Establish anchor region correspondence var anchorRegions = MapAnchorRegions(preTreatmentImage,postTreatmentImage, darkTargetRegions, roi); / / 2. Calculate NAT gray scale difference var natDifferences = CalculateNatDifferences(preTreatmentImage,postTreatmentImage, anchorRegions); / / 3. Edge detection and region division var edgeRegions = DetectAndDivideEdgeRegions(postTreatmentImage, roi); / / 4. Identify tumor retreat sub-images var hydrationRetreatSubImages = IdentifyHydrationRetreatSubImages( postTreatmentImage, edgeRegions, anchorRegions, natDifferences); / / 1. Establish the correspondence between dark target regions and anchor regions private Dictionary<Rectangle, Rectangle> MapAnchorRegions( Image<Gray, byte> preImage, Image<Gray, byte> postImage, Rectangle[] darkRegions, Rectangle roi) { var anchorMap = new Dictionary<Rectangle, Rectangle>(); foreach (var region in darkRegions) { / / Simplified correspondence - mapping based on relative position / / (In practice, more complex registration algorithms may be needed) var anchorRegion = new Rectangle( roi.X + (region.X - preImage.Width / 2) * roi.Width / preImage.Width, roi.Y + (region.Y - preImage.Height / 2) * roi.Height / preImage.Height, anchorRegion.Width = region.Width; anchorRegion.Height = region.Height; anchorMap.Add(region, anchorRegion);} return anchorMap;}region.Width * roi.Width / preImage.Width, region.Height * roi.Height / preImage.Height); / / Ensure the anchor region is within the ROI range anchorRegion.Intersect(roi); anchorMap.Add(region, anchorRegion);} return anchorMap;} / / 2. Calculate NAT gray difference private Dictionary<Rectangle, double>CalculateNatDifferences( Image<Gray, byte>preImage, Image<Gray, byte>postImage, Dictionary<Rectangle, Rectangle>anchorRegions) {var natDiffs = new Dictionary<Rectangle, double>(); foreach (var pair in anchorRegions) {var darkRegion = pair.Key; var anchorRegion = pair.Value; / / Get the maximum gray value of the dark target region byte taMax = GetRegionMaxGrayValue(preImage, darkRegion); / / Get the minimum gray value of the anchor region byte anMin = GetRegionMinGrayValue(postImage, anchorRegion); / / Calculate the NAT gray difference double natDiff = Math.Abs(taMax - anMin); natDiffs.Add(darkRegion, natDiff); } return natDiffs; / / 3. Edge detection and region division private List<Image<Gray, byte>> DetectAndDivideEdgeRegions( Image<Gray, byte> postImage, Rectangle roi) { / / Extract ROI region var roiImage = postImage.Copy(roi); / / Use Canny edge detection var edges = new Image<Gray, byte>(roiImage.Size); CvInvoke.Canny(roiImage, edges, 50, 150); / / Find contours var contours = new VectorOfVectorOfPoint(); CvInvoke.FindContours(edges, contours, null,Emgu.CV.CvEnum.RetrType.List, Emgu.CV.CvEnum.ChainApproxMethod.ChainApproxSimple); / / Divide the test region (simplified processing, actual may need more complex region segmentation algorithm) var dividedRegions = new List<Image<Gray, byte>>(); for (int i = 0; i<contours.Size; i++) {var contour = contours[i]; var rect = CvInvoke.BoundingRectangle(contour); rect.Intersect(roi); dividedRegions.Add(postImage.Copy(rect)); } return dividedRegions;} / / 4. Identify tumor retraction sub-images private List <rectangle>IdentifyHydrationRetreatSubImages( Image<Gray, byte> postImage, List<Image<Gray, byte>> edgeRegions, Dictionary<Rectangle, Rectangle> anchorRegions, Dictionary<Rectangle, double> natDifferences) { var result = new List <rectangle>(); foreach (var region in edgeRegions) / / Calculate the average gray level of the area to be measured double regionMean = CalculateMeanGrayValue(region); foreach (var anchorPair in anchorRegions) {var anchorRegion = anchorPair.Value; double natDiff = natDifferences[anchorPair.Key]; / / Calculate the average gray level of the anchored area double anchorMean = CalculateMeanGrayValue(postImage.Copy(anchorRegion)); / / Conditions if (Math.Abs(regionMean - anchorMean) <natDiff) {result.Add(region.ROI); break; / / Once a match is found, the inspection of other anchored regions can be stopped. }}} return result; / / Auxiliary method: Get the maximum grayscale value of the region private byte GetRegionMaxGrayValue(Image<Gray, byte> image, Rectangleregion) {byte max = 0; var roi = image.Copy(region); for (int y = 0; y <roi.Height; y++) {for (int x = 0; x <roi.Width; x++) {byte value = roi.Data[y, x, 0]; If (value > max), then max = value; }}return max;} / / Auxiliary method: Get the minimum gray value of the region private byte GetRegionMinGrayValue(Image<Gray, byte> image, Rectangle region) {byte min = 255; var roi = image.Copy(region); for (int y = 0; y<roi.Height; y++) {for (int x = 0; x<roi.Width; x++) {byte value = roi.Data[y, x, 0]; if (value<min) min = value; }}return min;} / / Auxiliary method: Calculate the average gray value of the region private double CalculateMeanGrayValue(Image<Gray, byte> image) {var mean = new MCvScalar(); mean = CvInvoke.Mean(image); return mean.V0;}}.

[0042] Further, in S400, the method for reconstructing the boundary of the tumor regression sub-image of the post-treatment image dataset to obtain the reconstructed image dataset is: Taking the point with the minimum gray value on the boundary line of the tumor regression sub-image of the post-treatment image dataset as P1 and the point with the maximum gray value as P2, P1 and P2 divide the boundary line of the tumor regression sub-image into two curve segments, and the shorter curve segment is taken as the fuzzy boundary line and the longer curve segment is taken as the stable boundary line; taking the point with the minimum absolute value of the difference between the gray value of P1 and the gray value of each point on the boundary line of the anchor region as PA1 and the point with the minimum absolute value of the difference between the gray value of P2 and the gray value of each point on the boundary line of the anchor region as PA2, PA1 and PA2 divide the boundary line of the anchor region into two curve segments, and the shorter curve segment is taken as the mapping boundary line; The mapping boundary line is copied to obtain a replacement curve segment, the replacement curve segment is scaled to the same size as the fuzzy boundary line to obtain a clear boundary line, and the fuzzy boundary line of the tumor regression sub-image is deleted to leave the stable boundary line; After moving the clear boundary line to the positions where its two endpoints coincide with points P1 and P2 respectively, the clear boundary line and the stable boundary line together form the new boundary line of the tumor regression sub-image. The reconstructed image dataset consists of tumor shrinkage sub-images obtained from each new boundary line.

[0043] Note: The case study in the post-treatment image dataset is: a 47-year-old female breast cancer patient with right-sided breast cancer; for example... Figure 2 The image shown is a pattern of the post-treatment image dataset before reconstruction. The boundary between the tumor and the surrounding normal breast tissue is unclear. Figure 2 The white arrow in the middle indicates the location of the tumor. Figure 3 The image shown is a pattern of the reconstructed image dataset; the tumor is clearly demarcated from the surrounding normal breast tissue. Figure 3 (The white arrow in the middle indicates the location of the tumor). Using subsequent deep learning models to extract features and predict the efficacy of NAT, it can be clearly seen that the boundary area is clearer and more significant, making it easier for the deep learning model to identify.

[0044] The key C# source code describing the method for reconstructing the boundaries of tumor regression sub-images in the post-treatment image dataset is as follows: public class BoundaryRefiner {public List <Image<Gray, byte> >ReconstructHydrationImages( Image<Gray, byte> postTreatmentImage, List <rectangle>hydrationSubImages, Dictionary<Rectangle, Rectangle> anchorRegions) {var reconstructedImages = new List <Image<Gray, byte> >(); foreach (var subImageRect in hydrationSubImages) / / 1. Extract tumor regression sub-images and corresponding anchoring regions var subImage = postTreatmentImage.Copy(subImageRect); var correspondingAnchor = anchorRegions.FirstOrDefault(x => x.Value.IntersectsWith(subImageRect)).Value; if (correspondingAnchor.IsEmpty) continue? / / 2. Handling Boundaries var processedSubImage = ProcessBoundary( postTreatmentImage, subImage, correspondingAnchor); reconstructedImages.Add(processedSubImage);} return reconstructedImages; private Image<Gray, byte> ProcessBoundary( Image<Gray, byte> fullImage, Image<Gray, byte> subImage, Rectangle anchorRegion) / / Obtain the contour of the tumor regression sub-image var subImageContour = GetImageContour(subImage); var anchorContour = GetImageContour(fullImage.Copy(anchorRegion)); / / Find key points on the boundary var (p1, p2) = FindBoundaryExtremePoints(subImageContour); var (pa1, pa2) = FindAnchorBoundaryPoints(anchorContour, p1, p2); / / Split the boundary curve var (fuzzyBoundary, stableBoundary) = SplitBoundary(subImageContour, p1, p2); var (mappingBoundary, _) = SplitBoundary(anchorContour, pa1, pa2); / / Create clear boundary var clearBoundary = CreateClearBoundary(mappingBoundary, fuzzyBoundary); / / Reconstruct boundary var newBoundary = ReconstructBoundary(stableBoundary, clearBoundary, p1, p2); / / Apply new boundary return ApplyNewBoundary(subImage, newBoundary); private VectorOfPoint GetImageContour(Image<Gray, byte> image) { var edges = new Image<Gray, byte>(image.Size); CvInvoke.Canny(image, edges, 50, 150); var contours = new VectorOfVectorOfPoint(); CvInvoke.FindContours(edges, contours, null, Emgu.CV.CvEnum.RetrType.External, Emgu.CV.CvEnum.ChainApproxMethod.ChainApproxSimple); return contours.Size>0 ? contours[0] : new VectorOfPoint();} private (Point p1, Point p2) FindBoundaryExtremePoints(VectorOfPointcontour) {if (contour.Size == 0) return (Point.Empty, Point.Empty); byte minVal = 255, maxVal = 0; Point p1 = new Point(), p2 = new Point(); for (int i = 0; i <contour.Size; i++) {var point = contour[i]; / / Assuming the grayscale value of the pixel at this location has already been saved byte val = GetGrayValueAtPoint(point); / / This method needs to be actually implemented. if (val <minVal) { minVal = val; p1 = point;} if (val>maxVal) { maxVal = val; p2 = point;} }return (p1, p2);} private (Point pa1, Point pa2) FindAnchorBoundaryPoints(VectorOfPointcontour, Point p1, Point p2) {if (contour.Size == 0) return (Point.Empty, Point.Empty); byte p1Val = GetGrayValueAtPoint(p1); byte p2Val = GetGrayValueAtPoint(p2); int minDiff1 = int.MaxValue, minDiff2 = int.MaxValue; Point pa1 = new Point(), pa2 = new Point(); for (int i = 0; i < contour.Size; i++) {var point = contour[i]; byte val = GetGrayValueAtPoint(point); int diff1 = Math.Abs(val - p1Val); int diff2 = Math.Abs(val - p2Val); if (diff1 < minDiff1) { minDiff1 = diff1; pa1 = point;} if (diff2 < minDiff2) { minDiff2 = diff2; pa2 = point;}<00003�7>}return (pa1, pa2);} private (VectorOfPoint shorter, VectorOfPoint longer) SplitBoundary( VectorOfPoint contour, Point p1, Point p2)<0̀000350>{ / / Find the index positions of p1 and p2 in the contour int idx1 = -1, idx2 = -1; for (int i = 0; i < contour.Size; i++) {if (contour[i].Equals(p1)) idx1 = i; if (contour[i].Equals(p2)) idx2 = i;} if (idx1 == -1 || idx2 == -1 || idx1 == idx2) return (new VectorOfPoint(), new VectorOfPoint()); / / Ensure idx1<idx2 if (idx1>idx2) {(idx1, idx2) = (idx2, idx1); (p1, p2) = (p2, p1);} / / Build two segments var segment1 = new VectorOfPoint(); var segment2 = new VectorOfPoint(); / / Split the contour into two parts for (int i = 0; i<contour.Size; i++) {if (i>= idx1&&i<= idx2) segment1.Push(new Point[] { contour[i]}); else segment2.Push(new Point[] { contour[i]});} / / Return the shorter and longer parts return segment1.Size<segment2.Size? (segment1, segment2) :(segment2, segment1);} private VectorOfPoint CreateClearBoundary( VectorOfPoint mappingBoundary, VectorOfPoint fuzzyBoundary) { / / Calculate scaling ratio based on length double scaleFactor = (double)fuzzyBoundary.Size / mappingBoundary.Size; / / Copy and scale the mapping boundary var scaledMapping = new VectorOfPoint(); for (int i = 0; i<mappingBoundary.Size; i++) {var scaledPoint = new Point( (int)(mappingBoundary[i].X * scaleFactor), (int)(mappingBoundary[i].Y * scaleFactor)); scaledMapping.Push(new Point[] { scaledPoint}); }return scaledMapping; }private VectorOfPoint ReconstructBoundary( VectorOfPoint stableBoundary, VectorOfPoint clearBoundary, Point p1, Point p2) { / / Move the clear boundary to coincide with p1 and p2 var movedClearBoundary = TranslateBoundary(clearBoundary, p1, p2); / / Merge the stable and clear boundaries var newBoundary = new VectorOfPoint(); newBoundary.Push(stableBoundary.ToArray()); newBoundary.Push(movedClearBoundary.ToArray()); return newBoundary;} private VectorOfPoint TranslateBoundary( VectorOfPoint boundary, Point targetP1, Point targetP2) {if (boundary.Size == 0) return boundary. / / Assume the first point is the original p1 and the last point is the original p2. Point originalP1 = boundary[0]; Point originalP2 = boundary[boundary.Size - 1]; / / Calculate the movement vector int dx1 = targetP1.X - originalP1.X; int dy1 = targetP1.Y - originalP1.Y; int dx2 = targetP2.X - originalP2.X; int dy2 = targetP2.Y - originalP2.Y; / / Average moving vector int avgDx = (dx1 + dx2) / 2; int avgDy = (dy1 + dy2) / 2; / / Create the boundary after the move var movedBoundary = new VectorOfPoint(); for (int i = 0; i <boundary.Size; i++) {movedBoundary.Push(new Point[] { new Point boundary[i].X + avgDx, boundary[i].Y + avgDy) });}return movedBoundary;} private Image<Gray, byte>ApplyNewBoundary( Image<Gray, byte>image, VectorOfPoint boundary) {if (boundary.Size == 0) return image; var mask = new Image<Gray, byte>(image.Size); CvInvoke.DrawContours(mask, new VectorOfVectorOfPoint(boundary), -1, new MCvScalar(255), -1); var result = image.Copy(); result.SetValue(0, mask.Not()); return result;}}.

[0045] Further, in S500, based on the reconstructed image dataset, the method for obtaining the prediction result of the breast cancer NAT efficacy by using the pre-trained deep learning model is: using the ResNeXt network model to extract features from the reconstructed image dataset to obtain a feature vector; by comparing the difference features between the pre-treatment image dataset and the reconstructed image dataset, a corresponding difference comparison feature vector is obtained; fuse the difference comparison feature vectors to obtain a fused feature vector; based on the fused feature vector, a classification prediction result of the breast cancer NAT efficacy by artificial intelligence is obtained.

[0046] Embodiment 2 This embodiment 2 is to replace the method of identifying the tumor regression sub-image of the post-treatment image dataset according to the dark target area based on embodiment 1, specifically: Preferably, in S300, the method for identifying the tumor regression sub-image of the post-treatment image dataset according to the dark target area is: the corresponding position of the dark target area in the pre-treatment image dataset in the region of interest in the post-treatment image dataset is recorded as an anchor region; then each dark target area has a corresponding anchor region in the region of interest; Take the position corresponding to the point with the maximum gray value in the dark target area on the region of interest as TaP; take the position corresponding to the point with the minimum gray value in the anchor region of the dark target area on the region of interest as AnP; record the direction from the point TaP to the point AnP as the retreat direction; detect a plurality of edge lines on the region of interest by using an edge detection operator, and divide the surface of the region of interest into a plurality of to-be-tested regions by using each edge line; Determine the tumor retreat channel of each to-be-tested region, specifically as follows: In each to-be-tested region, take the straight line connected by the point with the minimum gray value on the boundary of the to-be-tested region and the point with the minimum gray value in the anchor region as the hydration tendency line; take the straight line connected by the point with the maximum gray value on the boundary of the to-be-tested region and the point with the maximum gray value in the anchor region as the retreat tendency line; and take the region between the hydration tendency line and the retreat tendency line as the tumor retreat channel of the to-be-tested region; (The tumor retreat channel is the retreat direction between each to-be-tested region and the anchor region along the tumor center. During the treatment of the tumor, especially after the neoadjuvant chemotherapy (NAT), the tumor will shrink. Due to the significant reduction of tumor blood supply, cell death, interstitial fibrous tissue proliferation and inflammatory cell infiltration, the performance in the MRI imaging is affected. The gray value of the region is relatively discrete and irregular, and the tumor boundary is not clear. The existing deep learning model is difficult to distinguish the region from the surrounding tissue, and the recognition accuracy is low.) Judge whether the current to-be-tested region is a tumor retreat sub-image according to the tumor retreat channel, specifically as follows: Record each to-be-tested region intersecting with the tumor retreat channel of the current to-be-tested region as a feature region, and sequentially judge each feature region. If the gray value of the feature region is less than the gray value of a feature region before the feature region in the retreat direction, and the gray value of the feature region is less than the gray value of a feature region after the feature region in the retreat direction, record the current to-be-tested region as a tumor retreat sub-image.

[0047] Beneficial effects: According to the gray values of all the feature regions on the entire path to the tumor region, the change trend of the hydrogen proton content, blood supply and cell density in the key region of the fine tissue can be accurately monitored. The tumor retreat sub-image changed due to the NAT treatment can be accurately recognized from the MRI image, and the recognition accuracy of the deep learning model in the subsequent step is greatly improved.

[0048] Embodiment 3 This embodiment 3 is to replace the method of judging whether the current to-be-tested region is a tumor retreat sub-image according to the tumor retreat channel on the basis of embodiment 2, specifically as follows: Preferably, judge whether the current to-be-tested region is a tumor retreat sub-image according to the tumor retreat channel, specifically as follows: Each of the to-be-tested regions intersecting with the tumor retreat channel of the current to-be-tested region is recorded as a feature region, and a retreat gray difference value of each feature region in the tumor retreat channel is calculated, specifically: The feature region with the maximum average gray value in the tumor retreat channel is recorded as a peak feature region; the feature region with the minimum average gray value in the tumor retreat channel is recorded as a valley feature region; each of the feature regions in the tumor retreat channel is sequentially sorted according to the retreat direction, and the serial number of the peak feature region among the feature regions in the tumor retreat channel is recorded as UpI; the serial number of the valley feature region among the feature regions in the tumor retreat channel is recorded as LowI; The average value of the gray values of all the feature regions in the tumor retreat channel from the position of the current to-be-tested region to UpI is recorded as a front retreat gray; the average value of the gray values of all the feature regions in the tumor retreat channel from the position of the current to-be-tested region to LowI is recorded as a rear retreat gray; the difference between the front retreat gray and the rear retreat gray is recorded as a retreat gray difference value; For each of the feature regions in the tumor retreat channel, if there is a feature region whose retreat gray difference value is less than the retreat gray difference value of the feature region before it in the retreat direction, and the retreat gray difference value of the feature region is less than the retreat gray difference value of the feature region after it in the retreat direction, the current to-be-tested region is determined as a tumor retreat sub-image.

[0049] Among them, the key source code of C# language description of the specific implementation of the method for identifying the tumor retreat sub-image of the post-treatment image data set according to the dark target region is: public class HydrationRetreatAnalyzer {public List <rectangle>AnalyzeHydrationRetreat( Image<Gray, byte> retreatChannelImage, List <rectangle>testRegions, int retreatDirection = 0) / / Retreat direction (0 - horizontal to the right, 1 - vertical downward) / / 1. Identify feature regions List <featureregion>featureRegions = IdentifyFeatureRegions(retreatChannelImage, testRegions); / / 2. Sort feature regions by retreat direction SortFeatureRegions(featureRegions, retreatDirection); / / 3. Calculate retreat gray difference values for each feature region CalculateRetreatGrayDifferences(featureRegions); / / 4. Identify tumor retreat sub-images List <rectangle>hydrationSubImages = IdentifyHydrationSubImages(featureRegions); return hydrationSubImages;} / / 1. Identify feature regions private List <featureregion>IdentifyFeatureRegions( Image<Gray, byte> retreatChannelImage, List <rectangle>testRegions) {var featureRegions = new List <featureregion>(); foreach (var region in testRegions) / / Calculate the average gray value of the region var roi = retreatChannelImage.Copy(region); var mean = CvInvoke.Mean(roi); featureRegions.Add(new FeatureRegion {Region = region, AverageGrayValue = mean.V0});} return featureRegions;} / / 2. Sort feature regions by retreat direction private void SortFeatureRegions(List <featureregion>regions, int retreatDirection) {if (retreatDirection == 0) / / horizontal right {regions.Sort((a, b) =>a.Region.X.CompareTo(b.Region.X));} else / / vertical down {regions.Sort((a, b) =>a.Region.Y.CompareTo(b.Region.Y));} / / set sequence index for (int i = 0; i<regions.Count; i++) {regions[i].SequenceIndex = i; }} / / 3. calculate retreat gray difference for each feature region private void CalculateRetreatGrayDifferences(List <featureregion>featureRegions) {if (featureRegions.Count == 0) return; / / Find peak and valley feature regions var peakRegion = featureRegions.OrderByDescending(r =>r.AverageGrayValue).First(); var valleyRegion = featureRegions.OrderBy(r =>r.AverageGrayValue).First(); int peakIndex = peakRegion.SequenceIndex; int valleyIndex = valleyRegion.SequenceIndex; / / Calculate retreat gray difference value for each feature region foreach (var currentRegion in featureRegions) {int currentIndex = currentRegion.SequenceIndex; / / Calculate front retreat gray double frontRetreatGray = CalculateAverageGrayBetween( featureRegions, currentIndex, peakIndex); / / Calculate rear retreat gray double rearRetreatGray = CalculateAverageGrayBetween( featureRegions, currentIndex, valleyIndex); / / Calculate retreat gray difference value currentRegion.RetreatGrayDiff = frontRetreatGray - rearRetreatGray; }} / / Helper method: Calculate average gray value between two sequence indexes private double CalculateAverageGrayBetween( List <featureregion>regions, int startIndex, int endIndex) {if (regions.Count == 0) return 0; if (startIndex == endIndex) return regions[startIndex].AverageGrayValue; int step = startIndex <endIndex ? 1 : -1; int count = 0; double sum = 0; for (int i = startIndex; i != endIndex; i += step) {sum += regions[i].AverageGrayValue; count++;} / / Grayscale value including end position sum += regions[endIndex].AverageGrayValue; count++; return sum / count;} / / 4. Identify tumor retraction sub-images private List <rectangle>IdentifyHydrationSubImages(List <featureregion>featureRegions) { var result = new List <rectangle>(); for (int i = 1; i <featureRegions.Count - 1; i++) {var current = featureRegions[i]; var prev = featureRegions[i - 1]; var next = featureRegions[i + 1]; / / Judgment condition: The current difference value is less than the difference value of the adjacent regions before and after it. if (current.RetreatGrayDiff <prev.RetreatGrayDiff&& current.RetreatGrayDiff <next.RetreatGrayDiff) {result.Add(current.Region); }} return result;}}.

[0050] To evaluate the applicability of the model in clinical practice, with the consent and authorization of the breast cancer patients, MRI images of lesions before NAT treatment were randomly selected from 50 anonymous breast cancer patients to form a pre-treatment image dataset, and MRI images of lesions during the early stages of NAT treatment were selected to form a post-treatment image dataset.

[0051] The pre-treatment image dataset and post-treatment image dataset are multimodal MRI images collected from various breast cancer patients at two time points: before NAT (C0) and in the early treatment stage (C2), respectively, covering three sequences (T2WI, DWI, and T1+C). N4 bias field correction, three-dimensional isotropic resampling, grayscale normalization, and discretization were employed to unify imaging characteristics and reduce cross-device and cross-center interference. A dataset of pre-treatment images of 100 randomly selected breast cancer patients was used for data comparison in subsequent application example 1 and comparative example 1. Application Example 1: Input the pre-treatment image dataset and the reconstructed image dataset generated in Example 1 into the ResNeXt network model for prediction, generating a prediction result for each sample. Record the prediction result and corresponding probability value for each sample.

[0052] Comparative Example 1: The pre-treatment and post-treatment image datasets were input into the ResNeXt network model for prediction, generating prediction results for each sample. The prediction result and corresponding probability value for each sample were recorded.

[0053] Experimental evaluation: The application will apply Example 1 and Comparative Example 1 to predict the prediction effect of the reconstructed image data set and the post-treatment image data set by the pre-trained ResNeXt network model respectively, and compare it with the subjective visual assessment of the clinician. When using the original post-treatment image data set without processing to extract the breast cancer lesions, some of the tumor boundary pixels will be misjudged as non-tumor pixels, the tumor shrinkage phenomenon will be disturbed by the adjacent adjacent regions, resulting in blurred tumor boundary, resulting in the tumor and the surrounding tissue cannot be distinguished, so that the accuracy of some breast cancer lesion boundary prediction will be reduced. In Example 1 and Comparative Example 1, the bilateral statistical significance level is set to 0.05.

[0054] The reconstructed image data set can make the ResNeXt network model learn to extract rich feature information due to the correction of the boundary, and the boundary prediction effect of the breast cancer lesion is better.

[0055] The accuracy, sensitivity and specificity of Example 1 are 0.662 (0.612-0.876), 0.562 (0.476-0.787) and 0.756 (0.718-0.956) respectively; The accuracy, sensitivity and specificity of Comparative Example 1 are 0.742 (0.612-0.876), 0.692 (0.476-0.787) and 0.824 (0.718-0.956) respectively.

[0056] Note: The accuracy refers to the proportion of the number of samples correctly predicted by the classification model to the total number of samples; the higher the accuracy, the better the overall performance of the model.

[0057] Sensitivity, also known as recall rate or true rate, is the proportion of the number of samples correctly predicted as positive by the classification model to the total number of positive samples, and the higher the sensitivity, the stronger the recognition ability of the model on positive samples.

[0058] Specificity refers to the proportion of the number of samples correctly predicted as negative by the classification model to the total number of negative samples, and the higher the specificity, the stronger the recognition ability of the model on negative samples.

[0059] Therefore, it can be seen that in the same ResNeXt network model for the reconstructed image data set and the post-treatment image data set of the patient, the segmentation performance of the reconstructed image data set is better, and each performance indicator of the reconstructed image data set is better than that of the post-treatment image data set.

[0060] The disclosure provides a post-treatment image data set AT therapeutic effect prediction system structure diagram, and the artificial intelligence-based breast cancer NAT therapeutic effect prediction system of the embodiment includes a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor implements the steps in the artificial intelligence-based breast cancer NAT therapeutic effect prediction system embodiment described above when executing the computer program.

[0061] The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program in the following system units: An image data acquisition unit is configured to acquire MRI images of a breast cancer patient's lesion before NAT treatment to form a pre-treatment image data set, and to acquire MRI images of an early stage of NAT treatment to form a post-treatment image data set. A data registration and labeling unit is configured to label the corresponding lesion position as a region of interest after image registration of the pre-treatment image data set and the post-treatment image data set, respectively. A tumor shrinkage recognition unit is configured to locate the core region of the tumor in the region of interest of the pre-treatment image data set as a dark target region, and to recognize a tumor shrinkage sub-image of the post-treatment image data set according to the dark target region. A shrinkage boundary reconstruction unit is configured to reconstruct the boundary of the tumor shrinkage sub-image of the post-treatment image data set to obtain a reconstructed image data set. A therapeutic effect result prediction unit is configured to obtain a prediction result of the breast cancer NAT therapeutic effect based on the reconstructed image data set using a pre-trained deep learning model.

[0062] The artificial intelligence-based breast cancer NAT therapeutic effect prediction system can be run in a desktop computer, a notebook, a palm computer, and a cloud server, etc. The artificial intelligence-based breast cancer NAT therapeutic effect prediction system can be run in a system that can include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the example is only an example of the artificial intelligence-based breast cancer NAT therapeutic effect prediction system and does not constitute a limitation on the artificial intelligence-based breast cancer NAT therapeutic effect prediction system, which can include more or fewer components, or combine certain components, or different components, for example, the artificial intelligence-based breast cancer NAT therapeutic effect prediction system can also include an input and output device, a network access device, a bus, etc.

[0063] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The processor is a control center of the artificial intelligence-based breast cancer NAT efficacy prediction system running system, and connects various parts of the artificial intelligence-based breast cancer NAT efficacy prediction system running system through various interfaces and lines.

[0064] The memory can be used to store the computer program and / or modules, and the processor realizes various functions of the artificial intelligence-based breast cancer NAT efficacy prediction system by running or executing the computer program and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program required for a function (such as a sound playing function, an image playing function, etc.), etc.; and the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state memory devices.

[0065] Although the description of the present disclosure has been quite detailed and particularly described with respect to several described embodiments, it is not intended to be limited to any of these details or embodiments or any special embodiment, so as to effectively cover the intended scope of the present disclosure. In addition, the present disclosure is described above in embodiments that the inventors can foresee, and the purpose is to provide a useful description, and non-essential modifications to the present disclosure that have not yet been foreseen can still represent equivalent modifications of the present disclosure.< / rectangle> < / featureregion> < / rectangle> < / featureregion> < / featureregion> < / featureregion> < / featureregion> < / rectangle> < / featureregion> < / rectangle> < / featureregion> < / rectangle> < / rectangle> < / rectangle> < / rectangle> < / rectangle>

Claims

1. An artificial intelligence-based breast cancer NAT therapeutic effect prediction method, characterized by, The method comprises the following steps: S100, acquiring MRI images of a breast cancer patient before treatment as pre-treatment image data set, and acquiring MRI images of the breast cancer patient in an early stage of treatment as post-treatment image data set; S200, marking the positions of the lesions in the pre-treatment image data set and the post-treatment image data set as regions of interest after image registration; S300, positioning a tumor core region of the region of interest in the pre-treatment image data set as a dark target region, and identifying a tumor shrinkage sub-image of the post-treatment image data set according to the dark target region; S400, reconstructing the boundary of the tumor shrinkage sub-image of the post-treatment image data set to obtain a reconstructed image data set; S500, obtaining a prediction result of the breast cancer NAT treatment effect based on the reconstructed image data set and using a pre-trained deep learning model.

2. The breast cancer NAT treatment effect prediction method based on artificial intelligence according to claim 1, wherein in S300, the method for positioning the tumor core region of the region of interest in the pre-treatment image data set as the dark target region is: gray-scale processing the region of interest to obtain a gray-scale image, dividing the gray-scale image into sub-regions, and taking the sub-region with the minimum average gray-scale value of each pixel in each sub-region as the tumor core region, and taking the tumor core region as the dark target region.

3. The breast cancer NAT treatment effect prediction method based on artificial intelligence according to claim 1, wherein in S300, the method for identifying the tumor shrinkage sub-image of the post-treatment image data set according to the dark target region is: taking the corresponding position of the dark target region in the pre-treatment image data set in the region of interest in the post-treatment image data set as an anchor region; each dark target region corresponds to an anchor region in the post-treatment image data set; acquiring the maximum gray-scale value of each point in the dark target region as TaMax; acquiring the minimum gray-scale value of each point in the anchor region corresponding to the dark target region as AnMin; taking the absolute value of the difference between TaMax and AnMin as NAT gray-scale difference; detecting a plurality of edge lines in the region of interest in the post-treatment image data set using an edge detection operator, and dividing the surface of the region of interest into a plurality of to-be-tested regions using each edge line; taking the average gray-scale value of each pixel in the to-be-tested region as to-be-tested region gray-scale; sequentially judging the to-be-tested region gray-scale of each to-be-tested region, and marking the to-be-tested region corresponding to the to-be-tested region gray-scale as the tumor shrinkage sub-image if the absolute value of the difference between the to-be-tested region gray-scale and the average gray-scale value of all points on the anchor region is less than the NAT gray-scale difference.

4. The breast cancer NAT treatment effect prediction method based on artificial intelligence according to claim 3, wherein in S300, the method for identifying the tumor shrinkage sub-image of the post-treatment image data set according to the dark target region is: taking the corresponding position of the dark target region in the pre-treatment image data set in the region of interest in the post-treatment image data set as an anchor region; each dark target region corresponds to an anchor region in the region of interest; taking the position corresponding to the point with the maximum gray-scale value in the dark target region in the region of interest as TaP; taking the position corresponding to the point with the minimum gray-scale value in the anchor region corresponding to the dark target region in the region of interest as AnP; The direction from the point TaP to the point AnP is recorded as a retraction direction; a plurality of edge lines are detected in the region of interest by using an edge detection operator, and the surface of the region of interest is divided into a plurality of to-be-detected regions by the edge lines; whether the current to-be-detected region is a tumor retraction sub-image is determined according to the tumor retraction channel.

5. The artificial intelligence-based breast cancer NAT efficacy prediction method according to claim 4, wherein the tumor retraction channel of each to-be-detected region is determined, and specifically, in each to-be-detected region, a straight line connecting the point with the minimum gray value on the boundary of the to-be-detected region and the point with the minimum gray value in the anchor region is a hydration tendency line; a straight line connecting the point with the maximum gray value on the boundary of the to-be-detected region and the point with the maximum gray value in the anchor region is a retraction tendency line; and a region between the hydration tendency line and the retraction tendency line is the tumor retraction channel of the to-be-detected region.

6. The artificial intelligence-based breast cancer NAT efficacy prediction method according to claim 4, wherein whether the current to-be-detected region is a tumor retraction sub-image is determined according to the tumor retraction channel, and specifically, each to-be-detected region intersecting with the tumor retraction channel of the current to-be-detected region is recorded as a feature region, and each feature region is sequentially determined; if the gray value of a feature region is less than the gray value of a feature region before the retraction direction of the feature region, and the gray value of the feature region is less than the gray value of a feature region after the retraction direction of the feature region, the current to-be-detected region is recorded as a tumor retraction sub-image.

7. The artificial intelligence-based breast cancer NAT efficacy prediction method according to claim 4, wherein whether the current to-be-detected region is a tumor retraction sub-image is determined according to the tumor retraction channel, and specifically: each to-be-detected region intersecting with the tumor retraction channel of the current to-be-detected region is recorded as a feature region, and a retraction gray value difference of each feature region in the tumor retraction channel is calculated, and specifically: a feature region with the maximum average gray value in the tumor retraction channel is recorded as a peak feature region; a feature region with the minimum average gray value in the tumor retraction channel is recorded as a valley feature region; each feature region in the tumor retraction channel is sequentially sorted according to the retraction direction, the serial number of the peak feature region in each feature region in the tumor retraction channel is recorded as UpI, and the serial number of the valley feature region in each feature region in the tumor retraction channel is recorded as LowI; the average of the gray values of all feature regions from the position of the current to-be-detected region to UpI in the tumor retraction channel is recorded as a front retraction gray value; the average of the gray values of all feature regions from the position of the current to-be-detected region to LowI in the tumor retraction channel is recorded as a rear retraction gray value; and the difference between the front retraction gray value and the rear retraction gray value is recorded as a retraction gray value difference; for each feature region in the tumor retraction channel, if the retraction gray value difference of a feature region is less than the retraction gray value difference of a feature region before the retraction direction of the feature region, and the retraction gray value difference of the feature region is less than the retraction gray value difference of a feature region after the retraction direction of the feature region, the current to-be-detected region is determined as a tumor retraction sub-image.

8. The artificial intelligence-based breast cancer NAT efficacy prediction method according to claim 1, wherein in S400, the method for reconstructing the boundaries of the tumor regression sub-images of the post-treatment image dataset to obtain the reconstructed image dataset is as follows: taking the point with the minimum gray value on the boundary line of the tumor regression sub-image of the post-treatment image dataset as P1 and the point with the maximum gray value as P2, P1 and P2 divide the boundary line of the tumor regression sub-image into two curve segments, and the curve segment with the shorter length is taken as the fuzzy boundary line and the curve segment with the longer length is taken as the stable boundary line; taking the point with the minimum absolute value of the difference between the gray value of P1 and the gray value of each point on the boundary line of the anchor region as PA1 and the point with the minimum absolute value of the difference between the gray value of P2 and the gray value of each point on the boundary line of the anchor region as PA2, PA1 and PA2 divide the boundary line of the anchor region into two curve segments, and the curve segment with the shorter length is taken as the mapping boundary line; copying the mapping boundary line to obtain a replacement curve segment, scaling the replacement curve segment to the same size as the fuzzy boundary line to obtain a clear boundary line, and deleting the fuzzy boundary line of the tumor regression sub-image to leave the stable boundary line; moving the clear boundary line to the position where the two end points coincide with P1 and P2 respectively, and then constructing a new boundary line of the tumor regression sub-image from the clear boundary line and the stable boundary line; composing the reconstructed image dataset from the tumor regression sub-images with the new boundary lines obtained.

9. The artificial intelligence-based breast cancer NAT efficacy prediction method according to claim 1, wherein in S500, the method for obtaining the prediction result of the breast cancer NAT efficacy based on the reconstructed image dataset using the pre-trained deep learning model is as follows: using a ResNeXt network model to extract features from the reconstructed image dataset to obtain a feature vector; extracting the difference features between the pre-treatment image dataset and the reconstructed image dataset by comparison to obtain a corresponding difference comparison feature vector; fusing the difference comparison feature vector to obtain a fused feature vector; based on the fused feature vector, obtaining the classification prediction result of the artificial intelligence-based breast cancer NAT efficacy. The artificial intelligence-based breast cancer NAT efficacy prediction system comprises a processor, a memory, and a computer program stored in the memory and running on the processor, and the processor implements the steps in the artificial intelligence-based breast cancer NAT efficacy prediction method of claim 1 when executing the computer program.

10. An artificial intelligence-based breast cancer NAT efficacy prediction system, characterized by, ​

Images