Method, apparatus and device for predicting pcr based on dce-mri

Abstract

The application discloses a method, device and equipment for predicting pCR based on DCE-MRI, the method comprises the following steps: acquiring a DCE-MRI set and auxiliary information corresponding to the DCE-MRI set; inputting the DCE-MRI set and the auxiliary information into a trained pCR prediction model; determining image features corresponding to the DCE-MRI set through the pCR prediction model; determining target features based on the image features and the auxiliary information; and predicting a pCR category based on the target features. The application takes SER information as supplementary information of clinical information, combines the auxiliary information including the clinical information and the SER information with the DCE-MRI set, so that the pCR prediction model can learn the image features carried by the DCE-MRI set and the clinical features carried by the auxiliary information, and combines the image features and the clinical features to predict the pCR, thereby improving the accuracy of the pCR prediction, and further improving the accuracy of the NAC reaction evaluation.

Description

Technical Field

[0001] This application relates to the field of biomedical engineering technology, and in particular to a method, apparatus and device for predicting pCR based on DCE-MRI. Background Technology

[0002] Breast cancer is one of the most common and prevalent malignant tumors. Surgical resection is usually the preferred treatment for breast cancer; however, for patients with locally advanced disease, the extensive extent of the tumor or invasion of adjacent tissues makes breast-conserving surgery difficult to perform. Therefore, neoadjuvant chemotherapy (NAC) has become the standard of care for patients with locally advanced breast cancer, aiming to improve the resection rate and the likelihood of breast-conserving surgery by reducing the clinical stage of the tumor. On the other hand, pCR (pathological complete remission) – the absence of residual invasive cancer cells or carcinoma in situ in the breast – has been proven to be the best prognostic indicator after NAC treatment. Nevertheless, only about 30% of patients achieve pCR, and a small percentage of patients experience disease progression during NAC treatment. Furthermore, the assessment of pathological response status requires postoperative pathological examination, a process with a certain lag, which affects the accuracy and timeliness of pCR prediction.

[0003] Therefore, existing technologies still need to be improved and enhanced. Summary of the Invention

[0004] The technical problem to be solved by this application is to provide a method, apparatus and device for predicting pCR based on DCE-MRI, which addresses the shortcomings of the existing technology.

[0005] To address the aforementioned technical problems, the first aspect of this application provides a method for predicting pCR based on DCE-MRI, wherein the method specifically includes:

[0006] Obtain a DCE-MRI set and corresponding auxiliary information, wherein the auxiliary information includes SER information and clinical information, and the DCE-MRI in the DCE-MRI set carries breast cancer lesion areas;

[0007] The DCE-MRI set and the auxiliary information are input into a trained pCR prediction model. The pCR prediction model determines the image features corresponding to the DCE-MRI set, and the target features are determined based on the image features and the auxiliary information. The pCR category is then predicted based on the target features.

[0008] The pCR prediction method based on DCE-MRI is described above, wherein the DCE-MRI set includes a first DCE-MRI before NAC and at least one second DCE-MRI in NAC.

[0009] The pCR prediction method based on DCE-MRI, wherein the process of acquiring the SER information specifically includes:

[0010] Extract the region of interest (ROI) of the breast background from the first DCE-MRI and the region of interest (ROI) of the breast background from the second DCE-MRI.

[0011] Locate the target lesion region with the highest signal intensity on the first DCE-MRI and the target lesion region with the highest signal intensity on the second DCE-MRI.

[0012] Based on the breast background region of interest and target lesion region of the first DCE-MRI, and the breast background region of interest and target lesion region of the second DCE-MRI, the SER information corresponding to the DCE-MRI set is determined.

[0013] The pCR prediction method based on DCE-MRI, wherein the localization process of the target lesion region specifically includes:

[0014] Extract the first 3D mask of the region of interest in the breast background and the second 3D mask of the breast cancer lesion region;

[0015] A sliding region adapted to the region of interest of the breast background is constructed based on the first three-dimensional mask;

[0016] The slidable region is slid within the second three-dimensional mask, and the signal intensity of the slidable region at each sliding position is calculated to obtain the target lesion region.

[0017] The pCR prediction method based on DCE-MRI, wherein determining the image features corresponding to the DCE-MRI set through the pCR prediction model specifically includes:

[0018] Extract the radiomics features of the DCE-MRI set;

[0019] Extract slice features from the DCE-MRI set;

[0020] The image features are obtained by performing feature filtering on the radiomics features and the extracted slice features respectively.

[0021] The pCR prediction method based on DCE-MRI includes an image feature extraction module, a stitching module, and a classifier. The image feature extraction module is connected to the stitching module, and the stitching module is connected to the classifier. The image feature extraction module includes a radiomics feature extraction unit, a slice feature extraction unit, and a feature selection module. The radiomics feature extraction unit and the slice feature extraction unit are parallel and both are connected to the feature selection unit. The input items of the stitching module include image features determined by the image feature extraction module and auxiliary information.

[0022] The pCR prediction method based on DCE-MRI, wherein determining the target features based on the image features and the auxiliary information specifically includes:

[0023] Obtain the information feature vector corresponding to the auxiliary information;

[0024] The information feature vector is concatenated with the image features to obtain the target features.

[0025] A second aspect of this application provides a pCR prediction device based on DCE-MRI, wherein the pCR prediction device based on DCE-MRI specifically includes:

[0026] The acquisition module is used to acquire the DCE-MRI set and the auxiliary information corresponding to the DCE-MRI set, wherein the auxiliary information includes SER information and clinical information, and the DCE-MRI set carries the breast cancer lesion area;

[0027] The control module is used to input the DCE-MRI set and the auxiliary information into a trained pCR prediction model, determine the image features corresponding to the DCE-MRI set through the pCR prediction model, determine the target features based on the image features and the auxiliary information, and predict the pCR category based on the target features.

[0028] A third aspect of this application provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps in the pCR prediction method based on DCE-MRI as described above.

[0029] A fourth aspect of this application provides a terminal device, which includes: a processor and a memory;

[0030] The memory stores a computer-readable program that can be executed by the processor;

[0031] When the processor executes the computer-readable program, it implements the steps in the pCR prediction method based on DCE-MRI as described above.

[0032] Beneficial Effects: Compared with existing technologies, this application provides a method, apparatus, and device for predicting pCR based on DCE-MRI. The method includes acquiring a DCE-MRI set and corresponding auxiliary information, inputting the DCE-MRI set and the auxiliary information into a trained pCR prediction model, determining image features corresponding to the DCE-MRI set through the pCR prediction model, determining target features based on the image features and the auxiliary information, and predicting the pCR category based on the target features. This application uses SER information as supplementary information to clinical information and combines auxiliary information including clinical information and SER information with the DCE-MRI set. This allows the pCR prediction model to learn the image features carried by the DCE-MRI set and the clinical features carried by the auxiliary information, and to perform pCR prediction by combining image features and clinical features, thereby improving the accuracy of pCR prediction and thus improving the accuracy of NAC response assessment. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 A flowchart of a pCR prediction method based on DCE-MRI provided in the embodiments of this application.

[0035] Figure 2 This is an example diagram of the SER information retrieval process.

[0036] Figure 3 This is a flowchart illustrating the principle of a specific implementation of the pCR prediction method based on DCE-MRI provided in the embodiments of this application.

[0037] Figure 4 A schematic diagram of the structure of the pCR prediction device based on DCE-MRI provided in the embodiments of this application.

[0038] Figure 5 A schematic diagram of the structure of the terminal device provided in the embodiments of this application. Detailed Implementation

[0039] This application provides a method, apparatus, and device for predicting pCR based on DCE-MRI. To make the objectives, technical solutions, and effects of this application clearer and more explicit, the following detailed description is provided with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining this application and are not intended to limit this application.

[0040] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0041] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0042] It should be understood that the sequence number and size of each step in this embodiment do not imply the order of execution. The execution order of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application embodiment.

[0043] Studies have shown that breast cancer is one of the most common and prevalent malignant tumors worldwide. Surgical resection is usually the preferred treatment for breast cancer; however, for patients with locally advanced disease, the extensive extent of the tumor or invasion of adjacent tissues makes breast-conserving surgery difficult to perform. Therefore, neoadjuvant chemotherapy (NAC) has become the standard of care for patients with locally advanced breast cancer, aiming to improve the resection rate and the likelihood of breast-conserving surgery by reducing the clinical stage of the tumor. On the other hand, pCR (pathological complete remission) – the absence of residual invasive cancer cells or carcinoma in situ in the breast – has been proven to be the best prognostic indicator after NAC treatment. Nevertheless, only about 30% of patients achieve pCR, and a small percentage of patients experience disease progression during NAC treatment. Furthermore, the assessment of pathological response status requires postoperative pathological examination, a process that is somewhat delayed.

[0044] In recent years, some studies have shown that breast imaging can predict pCR (progressive cerebral complete remission), while DCE-MRI can be used to assess tumor response to NAC (negative contrast agent injection), with better results than imaging methods such as mammography and ultrasound. DCE-MRI is a non-invasive comprehensive examination that provides quantitative information on tissue vascular permeability and hemodynamics by continuously monitoring the distribution and washout process of the contrast agent in the tissue after injection. It can display changes in tumor microcirculation and angiogenesis, thus helping to predict the final outcome after breast cancer NAC.

[0045] Currently, radiomics is commonly used to assess NAC response based on DCE-MRI. This involves extracting massive quantitative features from precisely segmented medical images to reveal the deep connections between these features and underlying pathophysiological processes, thus significantly reflecting the tumor's microscopic structure. However, due to the high heterogeneity and complexity of the internal structure of malignant tumors, the kinetic and textural characteristics of drug metabolism within the tumor undergo significant changes during treatment, leading to low accuracy in assessing NAC response using radiomics.

[0046] To address the aforementioned issues, this application embodiment acquires a DCE-MRI set and corresponding auxiliary information. The DCE-MRI set and the auxiliary information are then input into a trained pCR prediction model. The pCR prediction model determines the image features corresponding to the DCE-MRI set, and based on these image features and the auxiliary information, it determines target features and predicts the pCR category based on the target features. This application uses SER information as supplementary information to clinical information and combines auxiliary information including clinical information and SER information with the DCE-MRI set. This allows the pCR prediction model to learn the image features carried by the DCE-MRI set and the clinical features carried by the auxiliary information. By combining image features and clinical features to perform pCR prediction, the accuracy of pCR prediction is improved, thereby enhancing the accuracy of NAC response assessment.

[0047] The application content will be further explained below with reference to the accompanying drawings and the description of the embodiments.

[0048] This embodiment provides a method for predicting pCR based on DCE-MRI, such as Figure 1 As shown, the method includes:

[0049] S10. Obtain the DCE-MRI set and the auxiliary information corresponding to the DCE-MRI set.

[0050] Specifically, the DCE-MRI (Dynamic Enhancement Resonance Imaging) images in the DCE-MRI set are obtained through breast MRI examinations of breast cancer patients. Each DCE-MRI image in the set carries a breast cancer lesion region. Ancillary information is used to describe the clinical characteristics of the breast cancer patients corresponding to the DCE-MRI set. This ancillary information includes SER (Signal Enhancement Ratio, the ratio of signal intensity of the breast cancer lesion to the signal intensity of the breast background parenchyma) information and clinical information. SER information is a quantitative indicator, representing the ratio of the signal intensity of the selected tumor region to the signal intensity of the circular region of interest in the breast background. SER information reflects the absorption and accumulation of contrast agent in the tumor region. Clinical information may include one or more of the following: the tumor proliferation index Ki67, progesterone receptor (PR), estrogen receptor (ER), and human epidermal growth factor receptor 2 (HER2). Of course, in practical applications, clinical information can also include patient information, which may include age at first onset, menstrual status, fertility status, clinical stage, NAC treatment regimen, chemotherapy cycle, and immunohistochemical results.

[0051] The DCE-MRI set can be DCE-MRI before NAC, DCE-MRI during NAC, or both. In the embodiments of this application, the DCE-MRI set includes a set of DCE-MRI before NAC and at least one DCE-MRI during NAC. That is, the DCE-MRI set includes a first DCE-MRI before NAC and at least one second DCE-MRI during NAC. For example, the DCE-MRI set includes a first DCE-MRI within two weeks before NAC and a second DCE-MRI in the early NAC period, or the DCE-MRI set includes a first DCE-MRI within two weeks before NAC, a second DCE-MRI in the early NAC period, and a second DCE-MRI in the middle NAC period, etc. This application embodiment acquires a first DCE-MRI before NAC and at least one second DCE-MRI during NAC, and then uses the first DCE-MRI before NAC and at least one second DCE-MRI during NAC to acquire image features. This allows the image features to learn the dynamic characteristics and texture features of drug metabolism within the tumor that change significantly during treatment, thereby improving the accuracy of pCR prediction.

[0052] Neoadjuvant chemotherapy (NAC) for breast cancer patients is classified using the pCR (prognostic complication) method. pCR is defined as the absence of residual invasive cancer cells or carcinoma in situ in the breast. Predicting pCR can help predict the prognostic outcome of NAC. The pCR category is based on prognostic indicators for NAC, including pCR and non-pCR. Non-pCR indicates a prognostic grade of 1-4, while pCR indicates a prognostic grade of 5. The prognostic criteria for NAC are graded according to the Miller-Payne (MP) scale. Grade 1 indicates no change in invasive cancer cells or only a few cancer cells, with no overall reduction in the number of cancer cells; Grade 2 indicates a slight reduction in invasive cancer cells, but the total number remains high, with a reduction of no more than 30%; Grade 3 indicates a reduction in invasive cancer cells between 30% and 90%; Grade 4 indicates a significant reduction in invasive cancer cells exceeding 90%, with only scattered small clusters of cancer cells or single cancer cells remaining; Grade 5 indicates no invasive cancer cells left in the original tumor bed, but ductal carcinoma in situ may still be present.

[0053] It should be noted that since the first and second DCE-MRI images were acquired at different times, their resolutions may differ, leading to different voxel spacings. For example, the voxel spacing of the first DCE-MRI might be [0.7, 0.7, 2.91], while that of the second DCE-MRI might be [0.7, 0.7, 2.93]. Different voxel spacings affect the appearance of the tumor in the image, thus impacting the calculation of its related features. To ensure that the features extracted from all images are consistent in spatial scale after acquiring the first and second DCE-MRI images, the comparability between features is guaranteed, thereby enhancing the generalization ability of the subsequently established machine learning classification model.

[0054] Furthermore, the DCE-MRI set includes a first DCE-MRI image before the NAC and at least one second DCE-MRI image within the NAC. Correspondingly, the SER information includes the SER information of the first DCE-MRI and the SER information of the second DCE-MRI. Therefore, when acquiring SER information, the SER information of the first and second DCE-MRI images is acquired sequentially. In addition, since SER information is used to represent the signal intensity ratio between the breast cancer lesion and the breast background parenchyma, the signal intensity of the breast background parenchyma (i.e., the region of interest in the breast background) and the maximum signal intensity of the breast cancer lesion are extracted when acquiring SER information, and then the SER information is calculated based on the two acquired signal intensities. Based on this, the process of acquiring SER information specifically includes:

[0055] Extract the region of interest (ROI) of the breast background from the first DCE-MRI and the region of interest (ROI) of the breast background from the second DCE-MRI.

[0056] Locate the target lesion region with the highest signal intensity on the first DCE-MRI and the target lesion region with the highest signal intensity on the second DCE-MRI.

[0057] Based on the breast background region of interest and target lesion region of the first DCE-MRI, and the breast background region of interest and target lesion region of the second DCE-MRI, the SER information corresponding to the DCE-MRI set is determined.

[0058] Specifically, the regions of interest (ROIs) for the breast background in both the first and second DCE-MRI scans are circular. These ROIs can be determined using deep learning, traditional edge recognition methods, or manual annotation. In a typical implementation, the ROIs for the breast background and the breast cancer lesion region are extracted using the Nibabel library. Specifically, after acquiring the first and second DCE-MRI scans, the Nibabel library is used to load the first and second DCE-MRI scans, and the ROIs for the breast background and the breast cancer lesion region are extracted from both scans. Then, the target lesion region with the highest signal intensity is selected within the breast cancer lesion region. Finally, SER information is calculated based on the signal intensity of the ROIs for the breast background and the target lesion region. The SER information for the first DCE-MRI is calculated based on the ROIs for the breast background and the target lesion region, and the SER information for the second DCE-MRI is calculated based on the ROIs for the breast background and the target lesion region. This application quantifies the signal enhancement degree of breast tumors by calculating SER information, which reflects the ratio of signal intensity in the region of interest in the breast background to the maximum signal intensity in the breast cancer lesion region. In this way, the pCR prediction model can learn the signal enhancement degree of breast tumors through SER information, and thus more accurately predict pCR.

[0059] It should be noted that when locating the target lesion area with the strongest signal intensity, it can be located through manual annotation, deep learning, or by sliding the region of interest (ROI) of the breast background. In one implementation of this application, the target lesion area is located by sliding the ROI of the breast background. Accordingly, the localization process of the target lesion area specifically includes:

[0060] Extract the first 3D mask of the region of interest in the breast background and the second 3D mask of the breast cancer lesion region;

[0061] A sliding region adapted to the region of interest of the breast background is constructed based on the first three-dimensional mask;

[0062] The slidable region is slid within the second three-dimensional mask, and the signal intensity of the slidable region at each sliding position is calculated to obtain the target lesion region.

[0063] Specifically, such as Figure 2As shown, the slidable region is a three-dimensional region, and its shape and volume are consistent with those of the first three-dimensional mask. That is, when constructing the slidable region, the shape and volume of the breast background region of interest are first obtained, and then a slidable three-dimensional matrix is ​​constructed based on the obtained shape and volume to obtain a slidable region adapted to the breast background region of interest. The shape and volume of the breast background region of interest can be determined by calculating the maximum and minimum values ​​of the first three-dimensional mask in the x, y, and z dimensions. This embodiment constructs a slidable region adapted to the breast background region of interest, and then determines the target lesion region by sliding the slidable region. This not only allows for rapid determination of the target lesion region but also ensures that the shape and volume of the target lesion region are consistent with those of the breast background region of interest, thereby improving the accuracy of the calculated SER information.

[0064] S20. Input the DCE-MRI set and the auxiliary information into the trained pCR prediction model, determine the image features corresponding to the DCE-MRI set through the pCR prediction model, determine the target features based on the image features and the auxiliary information, and predict the pCR category based on the target features.

[0065] Specifically, the pCR prediction model is a trained tool for predicting complete pathological response (NCR) to neoadjuvant chemotherapy in breast cancer. It predicts the pCR category during neoadjuvant chemotherapy based on the DCE-MRI dataset and auxiliary information, thereby predicting NCR. The pCR category includes both non-pCR and pCR, and the pCR category can predict the NAC grade, and thus the prognostic outcome of NAC. The image features are extracted from the DCE-MRI dataset by the pCR prediction model, while the target features are obtained based on both image features and auxiliary information. These target features carry the feature information included in both the image features and the auxiliary information, enriching the feature information that the pCR prediction model can learn and improving its predictive accuracy.

[0066] For example, the pCR prediction model may include an image feature extraction module, a stitching module, and a classifier. The image feature extraction module is used to extract image features corresponding to the DCE-MRI set. The stitching module is used to determine target features based on the image features and the auxiliary information. The classifier is used to predict the pCR category based on the target features. That is, after inputting the DCE-MRI set and the auxiliary information into the trained pCR prediction model, the image feature extraction module extracts features from the DCE-MRI set to obtain image features. The image features and auxiliary information are then input into the stitching module, which stitches them together to obtain the target features. The target features are then input into the classifier, which outputs the pCR category.

[0067] It should be noted that since the DCE-MRI set can include two or more DCE-MRI images, in this embodiment, each DCE-MRI image in the set is input into the pCR prediction model. The image feature extraction module sequentially extracts features from each DCE-MRI image to obtain the corresponding image features. Then, the image features of each DCE-MRI image are stitched together to obtain the image features. Alternatively, the image feature set of each DCE-MRI image can be used as the image features. Of course, in practical applications, before inputting the DCE-MRI set into the pCR prediction model, it can be stitched together according to the channel direction to obtain a stitched DCE-MRI image. Then, the stitched DCE-MRI image is input into the pCR prediction model, and the image feature extraction module extracts features from the stitched DCE-MRI image to obtain the image features.

[0068] In one implementation, the image features include radiomics features and slice features; that is, the pCR prediction model extracts radiomics features and slice features from the DCE-MRI. Accordingly, determining the image features corresponding to the DCE-MRI set through the pCR prediction model specifically includes:

[0069] Extract the radiomics features of the DCE-MRI set;

[0070] Extract slice features from the DCE-MRI set;

[0071] The image features are obtained by performing feature filtering on the radiomics features and the extracted slice features respectively.

[0072] Specifically, the image feature extraction module is used to extract radiomics features and slice features, and to filter the extracted radiomics features and slice features. Based on this, such as Figure 3As shown, the image feature extraction module may include a radiomics feature extraction unit, a slice feature extraction unit, and a feature filtering module. The radiomics feature extraction unit and the slice feature extraction unit run in parallel and are both connected to the feature filtering unit. The radiomics feature extraction unit is used to extract radiomics features, the slice feature extraction unit is used to extract slice features, and the feature filtering unit is used to filter the radiomics features and slice features. That is, as... Figure 3 As shown, the pCR prediction model includes an image feature extraction module, a stitching module, and a classifier. The image feature extraction module is connected to the stitching module, and the stitching module is connected to the classifier. The image feature extraction module includes a radiomics feature extraction unit, a slice feature extraction unit, and a feature selection module. The radiomics feature extraction unit and the slice feature extraction unit run in parallel and are both connected to the feature selection unit. The input items of the stitching module include image features determined by the image feature extraction module and auxiliary information. This embodiment of the application, by extracting radiomics features and slice features from DCE-MRI, can learn more feature information, thereby improving the prediction accuracy of the pCR prediction model.

[0073] Radiomics features can include intensity, shape, texture, and transformation features. These features can be extracted using radiomics methods, such as Python-based radiomics toolkits that can automatically extract them. However, radiomics features extracted through these methods often contain a large number of redundant features, which can lead to overfitting and the "curse of dimensionality." Therefore, after extraction, the radiomics features are filtered to retain only those most relevant to pCR prediction.

[0074] In one implementation, a coarse-to-fine feature selection method can be used to screen radiomics features. Specifically, firstly, the ANOVA F-test feature selection method (f_class if) from the scikit-learn toolkit can be used to perform coarse screening of radiomics features to obtain coarsely screened radiomics features (e.g., retaining the 10 most statistically significant radiomics features). The ANOVA F-test feature selection method evaluates the importance of features by calculating the F-statistic between each radiomics feature and pCR and non-class features for coarse screening. Secondly, the Least Absolute Shrinkage and Selection Operator (LASSO) is introduced to perform fine screening on the coarsely screened radiomics features to obtain the screened radiomics features corresponding to DCE-MRI.

[0075] It should be noted that the DCE-MRI set includes a first DCE-MRI and at least one second DCE-MRI. Therefore, during radiomics feature extraction, radiomics features are extracted and filtered from both the first and second DCE-MRIs separately. The number of features corresponding to the radiomics features of the first DCE-MRI can be the same as or different from the number of features corresponding to the second DCE-MRI. In this embodiment, the number of features corresponding to the radiomics features of the first DCE-MRI is different from that of the second DCE-MRI. This retains the radiomics features most relevant to pCR prediction while minimizing the number of radiomics features, thus better avoiding overfitting and the "curse of dimensionality." For example, the first DCE-MRI data retains 5 radiomics features, while the second DCE-MRI data retains 4 radiomics features.

[0076] The slice features are obtained through visual language model learning on DCE-MRI. The visual language model extracts sub-slice features from each slice in the DCE-MRI, and then fuses all sub-slice features to obtain the slice features. Specifically, when the DCE-MRI set includes a first DCE-MRI and at least one second DCE-MRI, the slice features consist of all sub-slice features from the first DCE-MRI and all sub-slice features from each second DCE-MRI. In one implementation, the visual language model can employ an image encoder based on the BiomedCLIP model. The BiomedCLIP model's image encoder extracts sub-slice features from each slice of the first and second DCE-MRI, and then fuses these sub-slice features to obtain the slice features. Furthermore, before feature extraction using the BiomedCLIP model's image encoder, the DCE-MRI can be cropped to focus on the breast cancer lesion area and reduce interference from redundant information. ROI cropping can involve equidistant expansion from the ROI center to obtain the ROI region, which is then used as the DCE-MRI slice feature for extraction. For example, a 224×224 pixel ROI region can be obtained by equidistant expansion from the ROI center.

[0077] In one implementation, determining the target features based on the image features and the auxiliary information specifically includes:

[0078] Obtain the information feature vector corresponding to the auxiliary information;

[0079] The information feature vector is concatenated with the image features to obtain the target features.

[0080] Specifically, the auxiliary information includes SER information and clinical information. Therefore, when determining the target feature based on the auxiliary information and image features, the SER information and clinical information can be concatenated into an information feature vector. Alternatively, the clinical information can be first filtered to remove redundant information, and then the filtered clinical information can be concatenated with the SER information to obtain the information feature vector. Another approach is to first extract clinical features from the clinical information using a text feature extraction module, and then combine the clinical features with the SER information to obtain the information feature vector. The obtained information feature vector is then concatenated with image features to obtain the target feature. Of course, in practical applications, the SER information, clinical information, and radiomics features and slice features from the image features can also be directly concatenated to obtain the target feature.

[0081] In one implementation, the predicted pCR category is obtained by classifying the target features. This classification process is performed using a classifier, which can be an LR (Logistic Regression) classifier. An LR classifier transforms a linear combination of one or more independent variables (or features, explanatory variables) into a probability prediction by applying a sigmoid function. Given independent variables, the goal is to predict the probability of the binary response variable to predict the pCR category.

[0082] To further illustrate the pCR prediction method based on DCE-MRI provided in this application, comparative experiments were conducted. In the comparative experiments, firstly, without incorporating clinical features, three feature extraction methods—radiomics, BiomedCLIP, and a combination of radiomics and BiomedCLIP—were used to introduce feature extractors (SERs) and construct prediction models, which were then compared with models without SERs. Secondly, with clinical features incorporated, three feature extraction methods—radiomics, BiomedCLIP, and a combination of radiomics and BiomedCLIP—were used to introduce SERs and construct prediction models. Similarly, the prediction models constructed using this method were compared with the corresponding models without SERs. This experimental design aims to explore whether SERs can contribute to improving the accuracy of pCR prediction independently of clinical features.

[0083] The final results are as follows: When analyzing the changes in model performance under different feature combinations across multiple NAC periods, it can be observed that simply adding SER did not significantly improve the performance of the trained models on the test set. However, when SER was combined with clinical features, the performance of each trained model on the test set improved. This confirms that introducing SER combined with clinical features into the model is effective and can help the model better capture the treatment response of breast cancer patients to NAC.

[0084] In summary, when using radiomics methods, the AUCs of the predictive models built based on pre-NAC, early-NAC, and clinical information were 0.641, 0.603, and 0.513 on the validation set, and 0.506, 0.677, and 0.628 on the test set, respectively. The predictive model combining multiple time periods and clinical features had an AUC of 0.633 on the validation set and 0.665 on the test set. When the BiomedCLIP method was combined with radiomics, the AUC of the model improved to 0.830 on the validation set and 0.773 on the test set. Finally, introducing SER further improved the AUC of the model to 0.842 on the validation set and 0.781 on the test set.

[0085] In summary, this embodiment provides a pCR prediction method based on the aforementioned DCE-MRI. This method includes acquiring a DCE-MRI set and corresponding auxiliary information; inputting the DCE-MRI set and the auxiliary information into a trained pCR prediction model; determining image features corresponding to the DCE-MRI set through the pCR prediction model; determining target features based on the image features and the auxiliary information; and predicting the pCR category based on the target features. This application first introduces SER as an auxiliary feature, combining it with clinical information. Secondly, it extracts effective image depth features through BiomedCLIP and combines the features extracted by BiomedCLIP with radiomics features as image features. Finally, it combines the auxiliary information and image features to predict the pCR category, enabling the pCR prediction model to learn diverse feature information and better predict pCR during breast cancer NAC.

[0086] Based on the above-described pCR prediction method based on DCE-MRI, this embodiment provides a pCR prediction device based on DCE-MRI, such as... Figure 4 As shown, the pCR prediction device based on DCE-MRI specifically includes:

[0087] The acquisition module 100 is used to acquire the DCE-MRI set and the auxiliary information corresponding to the DCE-MRI set, wherein the auxiliary information includes SER information and clinical information, and the DCE-MRI set carries the breast cancer lesion area;

[0088] The control module 200 is used to input the DCE-MRI set and the auxiliary information into a trained pCR prediction model, determine the image features corresponding to the DCE-MRI set through the pCR prediction model, determine the target features based on the image features and the auxiliary information, and predict the pCR category based on the target features.

[0089] Based on the above-described pCR prediction method based on DCE-MRI, this embodiment provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps in the pCR prediction method based on DCE-MRI as described in the above embodiment.

[0090] Based on the above-described pCR prediction method based on DCE-MRI, this application also provides a terminal device, such as... Figure 5 As shown, it includes at least one processor 20; a display screen 21; and a memory 22, and may also include a communication interface 23 and a bus 24. The processor 20, display screen 21, memory 22, and communication interface 23 can communicate with each other via the bus 24. The display screen 21 is configured to display a preset user guide interface in the initial setup mode. The communication interface 23 can transmit information. The processor 20 can invoke logical instructions in the memory 22 to execute the methods described in the above embodiments.

[0091] Furthermore, the logical instructions in the aforementioned memory 22 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0092] The memory 22, as a computer-readable storage medium, can be configured to store software programs, computer-executable programs, such as program instructions or modules corresponding to the methods in the embodiments of this disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions, or modules stored in the memory 22, thereby implementing the methods in the above embodiments.

[0093] The memory 22 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 22 may include high-speed random access memory (RAM) and non-volatile memory. Examples include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, as well as transient storage media.

[0094] Furthermore, the specific process of loading and executing multiple instruction processors in the aforementioned storage medium and terminal device has been described in detail in the above method, and will not be repeated here.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for predicting pCR based on DCE-MRI, characterized in that, The aforementioned pCR prediction method based on DCE-MRI specifically includes: Obtain a DCE-MRI set and corresponding auxiliary information, wherein the auxiliary information includes SER information and clinical information, and the DCE-MRI in the DCE-MRI set carries breast cancer lesion areas; The DCE-MRI dataset and the auxiliary information are input into a trained pCR prediction model. The pCR prediction model extracts radiomics features and slice features from the DCE-MRI dataset. Feature filtering is performed on the radiomics features and slice features to obtain image features. Target features are determined based on the image features and the auxiliary information, and the pCR category is predicted based on the target features. The pCR prediction model includes an image feature extraction module, a stitching module, and a classifier. The image feature extraction module is connected to the stitching module, and the stitching module is connected to the classifier. The image feature extraction module includes a radiomics feature extraction unit, a slice feature extraction unit, and a feature filtering unit. The radiomics feature extraction unit and the slice feature extraction unit run in parallel and are both connected to the feature filtering unit. The input items of the stitching module include the image features determined by the image feature extraction module and the auxiliary information. The process of obtaining the SER information specifically includes: Extract the breast background region of interest from the first DCE-MRI and the second DCE-MRI, where the first DCE-MRI is the area before the NAC and the second DCE-MRI is the area within the NAC. Locate the target lesion region with the highest signal intensity on the first DCE-MRI and the target lesion region with the highest signal intensity on the second DCE-MRI. Based on the breast background region of interest and target lesion region of the first DCE-MRI, and the breast background region of interest and target lesion region of the second DCE-MRI, the SER information corresponding to the DCE-MRI set is determined. The process of locating the target lesion area specifically includes: Extract the first 3D mask of the region of interest in the breast background and the second 3D mask of the breast cancer lesion region; A sliding region adapted to the region of interest of the breast background is constructed based on the first three-dimensional mask; The slidable region is slid within the second three-dimensional mask, and the signal intensity of the slidable region at each sliding position is calculated to obtain the target lesion region.

2. The method for predicting pCR based on DCE-MRI according to claim 1, characterized in that, The DCE-MRI set includes a first DCE-MRI prior to the NAC and at least one second DCE-MRI within the NAC.

3. The method for predicting pCR based on DCE-MRI according to claim 1, characterized in that, The determination of target features based on the image features and the auxiliary information specifically includes: Obtain the information feature vector corresponding to the auxiliary information; The information feature vector is concatenated with the image features to obtain the target features.

4. A device for predicting pCR based on DCE-MRI, characterized in that, The aforementioned pCR prediction device based on DCE-MRI specifically includes: The acquisition module is used to acquire the DCE-MRI set and the auxiliary information corresponding to the DCE-MRI set, wherein the auxiliary information includes SER information and clinical information, and the DCE-MRI set carries the breast cancer lesion area; A control module is used to input the DCE-MRI set and the auxiliary information into a trained pCR prediction model, extract radiomics features and slice features from the DCE-MRI set through the pCR prediction model, perform feature filtering on the radiomics features and slice features to obtain image features, determine target features based on the image features and the auxiliary information, and predict the pCR category based on the target features. The pCR prediction model includes an image feature extraction module, a stitching module, and a classifier. The image feature extraction module is connected to the stitching module, and the stitching module is connected to the classifier. The image feature extraction module includes a radiomics feature extraction unit, a slice feature extraction unit, and a feature filtering unit. The radiomics feature extraction unit and the slice feature extraction unit run in parallel and are both connected to the feature filtering unit. The input items of the stitching module include the image features determined by the image feature extraction module and the auxiliary information. The process of obtaining the SER information specifically includes: Extract the breast background region of interest from the first DCE-MRI and the second DCE-MRI, where the first DCE-MRI is the area before the NAC and the second DCE-MRI is the area within the NAC. Locate the target lesion region with the highest signal intensity on the first DCE-MRI and the target lesion region with the highest signal intensity on the second DCE-MRI. Based on the breast background region of interest and target lesion region of the first DCE-MRI, and the breast background region of interest and target lesion region of the second DCE-MRI, the SER information corresponding to the DCE-MRI set is determined. The process of locating the target lesion area specifically includes: Extract the first 3D mask of the region of interest in the breast background and the second 3D mask of the breast cancer lesion region; A sliding region adapted to the region of interest of the breast background is constructed based on the first three-dimensional mask; The slidable region is slid within the second three-dimensional mask, and the signal intensity of the slidable region at each sliding position is calculated to obtain the target lesion region.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the steps in the pCR prediction method based on DCE-MRI as described in any one of claims 1-3.

6. A terminal device, characterized in that, include: Processor and memory; The memory stores a computer-readable program that can be executed by the processor; When the processor executes the computer-readable program, it implements the steps in the pCR prediction method based on DCE-MRI as described in any one of claims 1-3.

Images