Prediction method for complete pathology remission in breast cancer adjuvant therapy and electronic equipment

Abstract

The invention provides a breast cancer adjuvant therapy pathology complete remission prediction method and electronic device.The breast cancer adjuvant therapy pathology complete remission prediction method comprises the steps that at the first stage, based on multi-time-point and multi-parameter MRI images of a patient before and after adjuvant therapy, the breast cancer adjuvant therapy pathology complete remission prediction result is obtained; training an image feature encoder with a tumor dynamic change identification capability through a dynamic supervision pre-training mode; and a second stage of constructing a multi-modal deep fusion network based on the image feature encoder to integrate the image features, radiomics features and clinical pathological features of the patient, and outputting a prediction result of complete pathology remission, the radiomics features being extracted from the MRI image. And the purposes of high precision and strong generalization capability are achieved.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing technology, and in particular relates to a method and electronic device for predicting pathological complete remission in adjuvant therapy for breast cancer. Background Technology

[0002] Breast cancer is a leading cause of cancer-related death among women worldwide. Neoadjuvant therapy (NAT) is a core treatment strategy for locally advanced breast cancer, which can reduce tumor size, decrease the risk of metastasis, increase the rate of breast-conserving surgery, and provide a basis for assessing tumor drug response. Pathological complete response (pCR) is a key indicator for measuring the efficacy of NAT and the long-term prognosis of patients. Current pCR prediction methods have significant limitations: clinicopathological features (such as hormone receptors and HER2 status) can only reflect partial information and cannot cover the complexity of the tumor microenvironment; medical imaging (such as DCE-MRI) can provide morphological, blood perfusion, and permeability information, but the accuracy and timeliness of traditional subjective assessments or RECIST criteria are insufficient; existing AI models mostly rely on single-modality or single-time-point data, radiomics "manual" feature extraction has limitations, deep learning models do not fully utilize the dynamic changes in tumors during NAT, and are mostly trained on single centers with small samples, resulting in weak generalization ability. They cannot meet the clinical need for accurate and universal prediction tools, leading to some patients not receiving personalized treatment or even undergoing unnecessary surgery. Summary of the Invention

[0003] In view of the problems existing in the prior art, the present invention provides a method and electronic device for predicting pathological complete remission in adjuvant therapy for breast cancer, which at least partially solves the problems of low prediction accuracy and weak generalization ability in the prior art.

[0004] In a first aspect, embodiments of this disclosure provide a method for predicting pathological complete response in adjuvant therapy for breast cancer, including: In the first stage, based on the patient's multi-time-point, multi-parameter MRI images before and after adjuvant therapy, an image feature encoder capable of recognizing dynamic changes in tumors was trained through dynamic supervised pre-training. In the second stage, a multimodal deep fusion network is constructed based on the image feature encoder to integrate image features, radiomics features, and the patient's clinicopathological features, and outputs a prediction result of complete pathological remission. The radiomics features are extracted from MRI images.

[0005] Optionally, the dynamic supervised pre-training in the first stage includes: A dual-tower encoder-decoder network model was used to process MRI images before and after treatment, respectively. The dual-tower encoder-decoder is trained by jointly optimizing three loss functions, which include: Reconstruction loss is used to constrain the reconstruction accuracy of the encoder-decoder of the input image; Contrast loss is used to differentiate patients with pathological complete remission from those with non-pathological complete remission based on differences in characteristics before and after treatment. An auxiliary classification loss is used to associate the features of pre-treatment, post-treatment, and the difference between the two with the pathological complete remission label.

[0006] Optionally, the calculation process for the auxiliary classification loss includes: The pre-treatment features, post-treatment features, and difference features are spliced ​​together; The concatenated features are input into the convolutional attention module for weighting; The weighted features are input into a linear classifier, and the difference from the true pathological complete remission label is calculated using the binary cross-entropy loss function.

[0007] Optionally, the formula for calculating the reconstruction loss is: , in, Indicates MRI modality, Indicates dynamic contrast-enhanced MRI, This indicates diffusion-weighted imaging. and These represent the original MRI images before and after treatment, respectively. and These represent the pre-treatment and post-treatment features extracted by the encoder, respectively. This indicates that it corresponds to a mode. The decoder; The formula for calculating the contrast loss is: , in For feature distance, For marginal hyperparameters, Label for complete pathological remission; The formula for calculating the auxiliary classification loss is as follows: , in, For difference features, , This represents the convolutional attention module. Indicates a linear layer. This represents the activation function. This represents the binary cross-entropy loss function.

[0008] Optionally, the multimodal deep fusion network in the second stage includes: The frequency domain transformation fusion module is used to adaptively weight and mine frequency domain information from image features from different sequences to obtain fused image features. The cross-modal attention dynamic fusion module is used to perform attention weighting on radiomics features based on clinical features and dynamically fuse clinical and radiomics features to obtain fused tabular features. The bidirectional cross-attention fusion module is used to perform bidirectional interactive alignment between the fused image features and the fused table features to obtain full-modal integrated features.

[0009] Optionally, the frequency domain transform fusion module includes: Weights are generated through a gating mechanism to adaptively weight image features from DCE-MRI and DWI modalities; The weighted features are subjected to Fast Fourier Transform to extract frequency domain information; The frequency domain information is output after performing an inverse Fourier transform.

[0010] Optionally, the cross-modal attention dynamic fusion module includes: The clinical feature vector is used as the query, and the radiomics feature vector is used as the key and value, respectively, to calculate the attention weight; The radiomics features are weighted according to the attention weights to obtain attention-weighted radiomics features; Clinical feature vectors are fused with attention-weighted radiomics features using learnable dynamic weights.

[0011] Optionally, the second stage includes, The multimodal deep fusion network is optimized using a combined loss function, which includes Focal Loss and cross-entropy loss with class weights, thereby achieving complete pathological relief and a balance between positive and negative samples.

[0012] Optionally, the second stage includes: Key predictive regions in MRI images are visualized using gradient-weighted class activation maps; Based on the visualization results, SHAP analysis is used to quantify the contribution of each input feature to the final prediction result.

[0013] Secondly, embodiments of this disclosure also provide an electronic device, the electronic device comprising: at least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform any of the methods for predicting pathological complete remission in adjuvant therapy for breast cancer as described in the first aspect.

[0014] The method for predicting pathological complete remission in adjuvant therapy for breast cancer provided by this invention integrates longitudinal dynamic MRI images, quantitative radiomics features, and clinicopathological features before and after NAT treatment based on two-stage multimodal pCR prediction, and constructs a multimodal deep fusion network with both high prediction accuracy and strong generalization ability, thereby achieving the goal of high accuracy and strong generalization ability. Attached Figure Description

[0015] The above and other objects, features and advantages of this disclosure will become more apparent from the accompanying drawings, in which like reference numerals generally denote like parts.

[0016] Figure 1 A flowchart of a method for predicting pathological complete response in adjuvant therapy for breast cancer provided in this embodiment of the disclosure; Figure 2a to Figure 2h A schematic diagram of the training set and three test set ROC curves and confusion matrix of the TSMGP model for predicting pCR in breast cancer according to embodiments of this disclosure; Figure 3 This disclosure provides Grad-CAM visualization heatmaps of two patients' TSMGP models for embodiments; Figure 4 A schematic block diagram of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0017] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0018] It should be understood that the following specific examples illustrate the implementation of this disclosure, and those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. This disclosure can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0019] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0020] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this disclosure. The illustrations only show the components related to this disclosure and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0021] Furthermore, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.

[0022] DCE-MRI (Dynamic Contrast-Enhanced Magnetic Resonance Imaging) refers to dynamic contrast-enhanced magnetic resonance imaging. DWI (Diffusion-Weighted Imaging) refers to diffusion-weighted imaging. Focal loss refers to the loss function used to solve the class imbalance problem. NAT refers to neoadjuvant therapy; pCR refers to complete pathological remission; The dual-tower encoder-decoder network model uses two encoders with identical structures but independent parameters (i.e., "dual towers") to process medical images before and after treatment, respectively, and reconstructs the images through the decoder. At the same time, multiple supervision signals are introduced to ensure that the encoder can extract dynamic features that are highly correlated with the therapeutic effect (pCR) and have discriminative power.

[0023] This embodiment integrates a two-stage multimodal deep learning model that combines longitudinal dynamic imaging, radiomics, and clinicopathological features for non-invasive and accurate prediction of pathological complete response (pCR) after neoadjuvant therapy (NAT) for breast cancer, providing technical support for individualized clinical treatment decisions.

[0024] Addressing the core pain points of existing methods for predicting pathological complete response (pCR) in neoadjuvant breast cancer therapy—namely, their limited information dimensions, neglect of dynamic tumor changes, and weak model generalization—this embodiment proposes a Two-Stage Multi-modal Prediction Model (TSMGP). Through a technical approach of "dynamic pre-training to enhance feature extraction + deep multi-modal fusion to strengthen information interaction + multi-center validation to ensure generalization ability," it constructs a solution covering all dimensions of "imaging-radiomics-clinical," balancing dynamic changes with accurate prediction. The details are as follows: like Figure 1 As shown in the figure, this embodiment discloses a method for predicting pathological complete response in adjuvant therapy for breast cancer, including: In the first stage, based on the patient's multi-time-point, multi-parameter MRI images before and after adjuvant therapy, an image feature encoder capable of recognizing dynamic changes in tumors was trained through dynamic supervised pre-training. In the second stage, a multimodal deep fusion network is constructed based on the image feature encoder to integrate image features, radiomics features, and the patient's clinicopathological features, and outputs a prediction result of complete pathological remission. The radiomics features are extracted from MRI images.

[0025] Optionally, the dynamic supervised pre-training in the first stage includes: A dual-tower encoder-decoder network model was used to process MRI images before and after treatment, respectively. The dual-tower encoder-decoder is trained by jointly optimizing three loss functions, which include: Reconstruction loss is used to constrain the reconstruction accuracy of the encoder-decoder of the input image; Contrast loss is used to differentiate patients with pathological complete remission from those with non-pathological complete remission based on differences in characteristics before and after treatment. An auxiliary classification loss is used to associate the features of pre-treatment, post-treatment, and the difference between the two with the pathological complete remission label.

[0026] Optionally, the calculation process for the auxiliary classification loss includes: The pre-treatment features, post-treatment features, and difference features are spliced ​​together; The concatenated features are input into the convolutional attention module for weighting; The weighted features are input into a linear classifier, and the difference from the true pathological complete remission label is calculated using the binary cross-entropy loss function.

[0027] Specifically, the first stage: Dynamic supervised pre-training – overcoming the pain point of “ignoring dynamic changes in tumors.” Addressing the shortcomings of traditional models that rely solely on static pre-treatment data and cannot capture tumor biological changes during neoadjuvant therapy (NAT), this stage uses multi-timepoint, multi-parameter MRI images (dynamic enhancement MRI (DCE-MRI) and diffusion-weighted imaging (DWI)) from before and after NAT treatment as the core input. Through a “dual-tower encoder-decoder architecture + joint supervision of three loss functions,” an image feature encoder with “dynamic change recognition capabilities” is trained. The three loss functions constrain the model learning from three dimensions: “image detail reconstruction,” “distinguishing differences before and after treatment,” and “pCR label association.” Reconstruction loss: Encoder-decoder reconstruction before treatment constrained by mean squared error (L2 norm). Post-treatment The reconstruction accuracy of MRI images ensures that the model can capture basic features such as tumor morphology and blood flow. The formula for calculating reconstruction loss is as follows: F represents the encoder output feature, and Dec represents the decoder. Contrast loss: Based on Euclidean distance to measure the difference in characteristics before and after treatment, it prompts pCR patients Increased characteristic distance, non-pCR patients The reduced feature distance strengthens the correlation between "dynamic changes and therapeutic efficacy." The formula for calculating the contrast loss is as follows: For feature distance, These are marginal hyperparameters; Auxiliary classification loss: splicing pre-treatment, post-treatment, and difference features After being weighted by the CBAM attention module, pCR labels are associated using binary cross-entropy (BCE) to pre-implant efficacy discrimination logic. The formula for calculating the auxiliary classification loss is as follows: .

[0028] Through collaborative optimization of three loss functions (total loss) This process ultimately yields a pre-trained encoder capable of accurately extracting "tumor dynamic change features," laying a high-quality feature foundation for subsequent predictions.

[0029] Optionally, the multimodal deep fusion network in the second stage includes: The frequency domain transformation fusion module is used to adaptively weight and mine frequency domain information from image features from different sequences to obtain fused image features. The cross-modal attention dynamic fusion module is used to perform attention weighting on radiomics features based on clinical features and dynamically fuse clinical and radiomics features to obtain fused tabular features. The bidirectional cross-attention fusion module is used to perform bidirectional interactive alignment between the fused image features and the fused table features to obtain full-modal integrated features.

[0030] Optionally, the frequency domain transform fusion module includes: Weights are generated through a gating mechanism to adaptively weight image features from DCE-MRI and DWI modalities; The weighted features are subjected to Fast Fourier Transform to extract frequency domain information; The frequency domain information is output after performing an inverse Fourier transform.

[0031] Optionally, the cross-modal attention dynamic fusion module includes: The clinical feature vector is used as the query, and the radiomics feature vector is used as the key and value, respectively, to calculate the attention weight; The radiomics features are weighted according to the attention weights to obtain attention-weighted radiomics features; Clinical feature vectors are fused with attention-weighted radiomics features using learnable dynamic weights.

[0032] Optionally, the second stage includes, The multimodal deep fusion network is optimized using a combined loss function, which includes Focal Loss and cross-entropy loss with class weights, thereby achieving complete pathological relief and a balance between positive and negative samples.

[0033] Optionally, the second stage includes: Key predictive regions in MRI images are visualized using gradient-weighted class activation maps; Based on the visualization results, SHAP analysis is used to quantify the contribution of each input feature to the final prediction result.

[0034] Phase Two: Multimodal Deep Fusion – Addressing the Problem of “Single Information Dimension”. To overcome the limitations of the one-sidedness of single-modal information, this phase loads the pre-trained encoder weights from Phase One to construct a three-modal fusion architecture of “imaging-radiomics-clinical”. Through three dedicated fusion modules, deep information interaction is achieved, fully leveraging the complementary value of each modality: Input feature processing: Imaging features: Core features were extracted from pre-trained encoders of DCE-MRI and DWI images before treatment; Radiomics features: 1132 quantized features (including 18 first-order statistical features, 14 three-dimensional shape features, 68 texture features, and 1032 filter-derived features) were extracted from MRI images and mapped to a 256-dimensional unified vector through a 3-layer MLP (layer normalization + ReLU + Dropout). ; Clinical characteristics: 21 clinicopathological indicators, including patient age, HER2 status, and Ki-67 index, were included and mapped to a 256-dimensional vector using a homostructured MLP. .

[0035] Layered integration module design: SPT (Frequency Domain Transform Fusion Module): First, through a gating mechanism... ,in, For the gated weight matrix, Features of DCE-MRI Features of DWI The gating bias term; adaptive weighted bimodal image features, then subjected to Fast Fourier Transform (F) and inverse transform. Mining global information in the frequency domain to compensate for the limitations of spatial domain features and outputting fused image features. ; MADN (Crossmodal Attention Dynamic Fusion Module): Query based on clinical features Radiomics features are key values Radiomics features strongly correlated with clinical information are screened through attention mechanisms, and then dynamically weighted and fused. Strengthen the link between clinical and radiomics. Dynamic weights for clinical features. These are clinicopathological features. Dynamic weights for radiomics features These are attention-filtered radiomics features; BCAM (Bidirectional Cross-Attention Fusion Module): Achieves image features through multi-head self-attention. Features of the integration of "clinical-radiomics" bidirectional interactive alignment ,in Image features enhanced by tabular data. , , , , and Both are weight matrices. To query the table characteristics of the source, d represents the image features of the key value source, and d represents the dimension of the key vector, ensuring that no modal information is omitted.

[0036] Prediction optimization: The fused comprehensive features are input into a fully connected classifier, which employs Focal Loss (to address imbalanced pCR samples). The pCR prediction result is output by jointly optimizing the cross-entropy loss (which reinforces minority class learning, with class weights [1.0, 2.0]) and the cross-entropy loss (which reinforces minority class learning).

[0037] The TSMGP model implemented in this study predicts pCR in breast cancer, such as Figures 2a to 2h As shown, where Figure 2a Figure 2e shows the ROC curve for the training set, and Figure 2e shows the confusion matrix for the training set. Figures 2b to 2d The ROC curves are for the three test sets. Figures 2f to 2h This is the confusion matrix for the three test sets.

[0038] Multi-center validation optimization: Training and validation were conducted based on data from 1292 patients across 5 medical centers to ensure the model's generalization ability.

[0039] This technology achieves accurate pCR prediction (primary cohort AUC 0.851, external validation AUC 0.787-0.804) and strong interpretability (Grad-CAM locates key tumor regions, SHAP quantifies feature contribution). It can assist physicians in developing individualized treatment plans, reducing overtreatment and ineffective surgeries, and improving the efficiency of breast cancer diagnosis and treatment. The TSMGP model Grad-CAM visualization heatmaps of two patients are shown below. Figure 3 As shown.

[0040] This embodiment uses data from 1292 breast cancer patients across five medical centers (primary cohort of 674 patients: 368 from Sir Run Run Shaw Hospital affiliated with Zhejiang University and 306 from the Affiliated Hospital of Zunyi Medical University; external validation cohort of 618 patients: 200 from Cancer Hospital of Chinese Academy of Medical Sciences, 237 from Shandong Cancer Hospital, and 181 from the First Affiliated Hospital of Xi'an Jiaotong University School of Medicine) to verify the effectiveness of the two-stage multimodal breast cancer neoadjuvant therapy pathological complete response (pCR) prediction model (TSMGP) based on dynamic imaging features. All patients were pathologically diagnosed with invasive breast cancer by biopsy and had multiparametric MRI images (DCE-MRI, DWI) before and after neoadjuvant therapy (NAT), complete clinicopathological data, and postoperative pCR results (pCR = ypT0 / is, non-pCR = invasive tumor present in surgical specimen). Bilateral breast cancer, poor MRI quality, and missing clinical information were excluded.

[0041] The following describes the TSMGP model application process using one HER2+ subtype patient from the main cohort as an example. Steps 1-9 were repeated for the remaining 1291 patients. The performance results of the entire cohort are summarized in Tables 1 and 2.

[0042] Table 1. Predictive performance statistics of the TSMGP model in 1292 patients.

[0043] Table 2. Statistical table of prediction results for 9 hyperparameter-selected samples.

[0044] I. Basic Sample Information: Basic information: Female, 52 years old, ECOG score of 1, clinical stage T2N0M0 (AJCC8 version). Pathological features: ER negative (<1% positive cells), PR negative (<1% positive cells), HER2 positive (IHC 3+), Ki-67 index 32%, molecular subtype is HER2+; Treatment and pathological results: The NAT regimen was "paclitaxel + carboplatin + trastuzumab" (6 cycles). 3.0T MRI was performed before and after treatment, and postoperative pathological evaluation showed pCR.

[0045] II. Steps for applying the TSMGP model: Step 1, Data Collection: MRI images: A 3.0T MRI scanner (Siemens Prisma) was used to acquire DCE-MRI (fat-suppressed T1WI acquired every 60 seconds after intravenous injection of Gd contrast agent, for a total of 7 phases) and DWI (b value 0, 1000s / mm²). Radiomics features: Based on MRI images and ROIs, 1132 quantitative features (including tumor morphology, texture, intensity distribution, etc.) were extracted using the PyRadiomics toolkit. Clinical characteristics: 21 clinical and pathological indicators were collected, including age, HER2 status, Ki-67 index, and T stage.

[0046] Step 2, Image Preprocessing: ROI annotation: Two radiologists with more than 5 years of experience manually annotated the tumor region of interest (ROI) on DCE-MRI peak enhancement and DWI images using ITK-SNAP software, avoiding normal tissue. When the difference is >10%, it is arbitrated by a third chief physician. Image registration: Using ANTS software, the pre-treatment MRI was used as a "floating image" and the post-treatment MRI was used as a "reference image" for non-linear three-dimensional registration, and the corresponding ROI after treatment was obtained by mapping. Standardization: The image containing the ROI was adjusted to 256×256 pixels, cropped and scaled to 224×224 pixels, and the data was enhanced by random flipping and ±15° rotation. Finally, mean-standard deviation normalization was performed (mean=0.5, standard deviation=0.5).

[0047] Step 3, First Stage – Dynamic Supervised Pre-training: Input data: Pre- and post-treatment paired MRI (DCE-MRI+DWI) of this patient and 673 other patients in the main cohort; Training process: A dual-tower encoder-decoder architecture was used, combining reconstruction loss (to ensure image reconstruction accuracy), contrast loss (to distinguish the differences in features before and after treatment in pCR / non-pCR patients), and auxiliary classification loss (to associate pCR labels) to train the model. Five-fold hierarchical cross-validation was used (hierarchical according to pCR labels). The optimizer was AdamW (initial learning rate 1e-4), the batch size was 16, and the early cessation patience value was 8. Output: A pre-trained encoder is obtained, which can extract dynamic features related to pCR in the patient's MRI before and after treatment (such as a 38% decrease in peak blood perfusion on DCE-MRI after treatment, which is consistent with the biological characteristics of pCR).

[0048] Step 4, Second Stage – Multimodal Feature Extraction: Image features: The pre-treatment MRI was input into a pre-trained encoder to extract 256-dimensional image feature vectors; Radiomics features: 1132 radiomics features were processed using a 3-layer MLP (including layer normalization, ReLU activation, and Dropout) to output a 256-dimensional vector; Clinical features: 21 clinical features were processed using a homogeneous MLP to output a 256-dimensional vector.

[0049] Step 5: Multimodal fusion: Features are integrated through 3 core modules: SPT module: Adaptive weighted DCE-MRI and DWI features, supplemented with frequency domain information through Fourier transform, output fused image features; MADN module: Using clinical features as queries and radiomics features as keys, it calculates attention weights and dynamically fuses the two types of features. BCAM module: Enables bidirectional interaction between image features and "clinical + radiomics" features to obtain comprehensive features across all modalities.

[0050] Step 6, pCR prediction: The comprehensive features were input into a fully connected classifier (including layer normalization, ReLU activation, and Dropout), and optimized using Focal Loss (to address sample imbalance) + cross-entropy loss. The output showed that the predicted probability of pCR for this patient was 0.91 (threshold 0.5, judged as pCR), which was consistent with the postoperative pathological results.

[0051] Step 7: Model interpretability verification: Grad-CAM visualization: The heat map focuses on the core tumor area (red accounts for 70% of the ROI) and the surrounding microvascular area, which is consistent with the biological logic of tumor necrosis in pCR patients; SHAP analysis: The top 3 key features were HER2 status (positive promotes pCR), Ki-67 index (high expression promotes pCR), and DCE-MRI maximum enhancement slope (decreased after treatment promotes pCR), which is consistent with clinical understanding.

[0052] Step 8, Hyperparameter Selection: Using an additional 9 patients from the main cohort with "manually annotated image features + clear pCR results" (5 pCR and 4 non-pCR, all of which were HER2+ subtypes), the optimal hyperparameters were selected as follows: encoder residual block level 3, SPT module Fourier transform dimension 256×256, MADN module attention head number 8, and classifier Dropout probability 0.3 (the results of the hyperparameter selection are shown in Table 2).

[0053] Step 9, External Validation: The patient data was input into the TSMGP model trained on the external validation cohort VC2 (Shandong Cancer Hospital, 237 cases, pCR rate 37.1%). Steps 2-6 were repeated. The pCR prediction probability was 0.88, and the patient was still identified as pCR, which was consistent with the pathological results, thus verifying the model's generalization ability.

[0054] The electronic device disclosed in this embodiment includes a memory and a processor. The memory is used to store non-transitory computer-readable instructions. Specifically, the memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc.

[0055] The processor may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions. In one embodiment of this disclosure, the processor is used to execute computer-readable instructions stored in the memory, causing the electronic device to perform all or part of the steps of the breast cancer adjuvant therapy pathological complete remission prediction method of the foregoing embodiments of this disclosure.

[0056] Those skilled in the art will understand that, in order to solve the technical problem of how to achieve a good user experience, this embodiment may also include well-known structures such as communication buses and interfaces, and these well-known structures should also be included within the protection scope of this disclosure.

[0057] like Figure 4This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. It illustrates a structural schematic diagram suitable for implementing the electronic device in the embodiment of the present disclosure. Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0058] like Figure 4 As shown, an electronic device may include a processing unit (such as a central processing unit, graphics processing unit, etc.) that can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) or a program loaded from a storage device into random access memory (RAM). The RAM also stores various programs and data required for the operation of the electronic device. The processing unit, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0059] Typically, the following devices can be connected to the I / O interface: input devices, such as sensors or visual information acquisition devices; output devices, such as displays; storage devices, such as magnetic tapes or hard drives; and communication devices. Communication devices allow electronic devices to exchange data wirelessly or via wired communication with other devices, such as edge computing devices. Although Figure 4 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0060] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, all or part of the steps of the breast cancer adjuvant therapy pathological complete response prediction method of embodiments of this disclosure are performed.

[0061] For a detailed description of this embodiment, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.

[0062] A computer-readable storage medium according to embodiments of the present disclosure stores non-transitory computer-readable instructions. When these non-transitory computer-readable instructions are executed by a processor, all or part of the steps of the breast cancer adjuvant therapy pathological complete response prediction methods of the foregoing embodiments of the present disclosure are performed.

[0063] The aforementioned computer-readable storage media include, but are not limited to: optical storage media (e.g., CD-ROM and DVD), magneto-optical storage media (e.g., MO), magnetic storage media (e.g., magnetic tape or portable hard drive), media with built-in rewritable non-volatile memory (e.g., memory card), and media with built-in ROM (e.g., ROM cartridge).

[0064] For a detailed description of this embodiment, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.

[0065] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0066] In this disclosure, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The block diagrams of devices, apparatuses, devices, and systems involved in this disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as "comprising," "including," "having," etc., are open-ended terms meaning "including but not limited to," and are used interchangeably with them. The terms "or" and "and" as used herein refer to the terms "and / or," and are used interchangeably with them unless the context clearly indicates otherwise. The term "such as" as used herein refers to the phrase "such as but not limited to," and is used interchangeably with it.

[0067] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0068] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.

[0069] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0070] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0071] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A method for predicting pathological complete response in adjuvant therapy for breast cancer, characterized in that, include: In the first stage, based on the patient's multi-time-point, multi-parameter MRI images before and after adjuvant therapy, an image feature encoder capable of recognizing dynamic changes in tumors was trained through dynamic supervised pre-training. In the second stage, a multimodal deep fusion network is constructed based on the image feature encoder to integrate image features, radiomics features, and the patient's clinicopathological features, and outputs a prediction result of complete pathological remission. The radiomics features are extracted from MRI images.

2. The method for predicting pathological complete remission in adjuvant therapy for breast cancer according to claim 1, characterized in that, The dynamic supervised pre-training in the first stage includes: A dual-tower encoder-decoder network model was used to process MRI images before and after treatment, respectively. The dual-tower encoder-decoder is trained by jointly optimizing three loss functions, which include: Reconstruction loss is used to constrain the reconstruction accuracy of the encoder-decoder of the input image; Contrast loss is used to differentiate patients with pathological complete remission from those with non-pathological complete remission based on differences in characteristics before and after treatment. An auxiliary classification loss is used to associate the features of pre-treatment, post-treatment, and the difference between the two with the pathological complete remission label.

3. The method for predicting pathological complete remission in adjuvant therapy for breast cancer according to claim 2, characterized in that, The calculation process for the auxiliary classification loss includes: The pre-treatment features, post-treatment features, and difference features are spliced ​​together; The concatenated features are input into the convolutional attention module for weighting; The weighted features are input into a linear classifier, and the difference from the true pathological complete remission label is calculated using the binary cross-entropy loss function.

4. The method for predicting pathological complete remission in adjuvant therapy for breast cancer according to claim 3, characterized in that, The formula for calculating the reconstruction loss is: ， in, Indicates MRI modality, Indicates dynamic contrast-enhanced MRI, This indicates diffusion-weighted imaging. and These represent the original MRI images before and after treatment, respectively. and These represent the pre-treatment and post-treatment features extracted by the encoder, respectively. This indicates that it corresponds to a mode. The decoder; The formula for calculating the contrast loss is: ， in For feature distance, For marginal hyperparameters, Label for complete pathological remission; The formula for calculating the auxiliary classification loss is as follows: ， in, For difference features, , This represents the convolutional attention module. Indicates a linear layer. This represents the activation function. This represents the binary cross-entropy loss function.

5. The method for predicting pathological complete remission in adjuvant therapy for breast cancer according to claim 1, characterized in that, The multimodal deep fusion network in the second stage includes: The frequency domain transformation fusion module is used to adaptively weight and mine frequency domain information from image features from different sequences to obtain fused image features. The cross-modal attention dynamic fusion module is used to perform attention weighting on radiomics features based on clinical features and dynamically fuse clinical and radiomics features to obtain fused tabular features. The bidirectional cross-attention fusion module is used to perform bidirectional interactive alignment between the fused image features and the fused table features to obtain full-modal integrated features.

6. The method for predicting pathological complete remission in adjuvant therapy for breast cancer according to claim 5, characterized in that, The frequency domain transformation fusion module includes: Weights are generated through a gating mechanism to adaptively weight image features from DCE-MRI and DWI modalities; The weighted features are subjected to Fast Fourier Transform to extract frequency domain information; The frequency domain information is output after performing an inverse Fourier transform.

7. The method for predicting pathological complete remission in adjuvant therapy for breast cancer according to claim 5, characterized in that, The cross-modal attention dynamic fusion module includes: The clinical feature vector is used as the query, and the radiomics feature vector is used as the key and value, respectively, to calculate the attention weight; The radiomics features are weighted according to the attention weights to obtain attention-weighted radiomics features; Clinical feature vectors are fused with attention-weighted radiomics features using learnable dynamic weights.

8. The method for predicting pathological complete remission in adjuvant therapy for breast cancer according to claim 1, characterized in that, The second phase includes, The multimodal deep fusion network is optimized using a combined loss function, which includes FocalLoss and cross-entropy loss with class weights, thereby achieving complete pathological remission and a balance between positive and negative samples.

9. The method for predicting pathological complete remission in adjuvant therapy for breast cancer according to claim 1, characterized in that, The second stage includes: Key predictive regions in MRI images are visualized using gradient-weighted class activation maps; Based on the visualization results, SHAP analysis is used to quantify the contribution of each input feature to the final prediction result.

10. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method for predicting pathological complete remission in adjuvant therapy for breast cancer as described in any one of claims 1-9.

Images