A Non-invasive Artificial Intelligence-Based Diagnostic System for Neoadjuvant Chemotherapy Efficacy in Breast Cancer

By combining a non-invasive artificial intelligence system with a bidirectional cross-attention mechanism and a multi-scale deep supervision objective function, the problem of multimodal data fusion and the disconnect between clinical practice in the diagnosis of neoadjuvant chemotherapy efficacy in breast cancer has been solved, achieving highly accurate and interpretable diagnostic results.

CN117883062BActive Publication Date: 2026-11-10SHANDONG NORMAL UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311515118.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-11-14
Publication Date
2026-11-10
Estimated Expiration
2043-11-14

AI Technical Summary

Technical Problem

Most existing diagnostic systems for neoadjuvant chemotherapy in breast cancer use single-modality data, neglecting the interaction between different data modalities. They also employ a single feature matrix fusion method, incur high computational costs, and are detached from clinical practice, resulting in low diagnostic accuracy.

Method used

A non-invasive artificial intelligence-based system is adopted to acquire clinical information and MRI image features through a feature extraction module, perform feature fusion using a bidirectional cross-attention mechanism, and conduct co-optimization of the objective function through multi-scale deep supervision, combined with the Grad-Cam algorithm for quantitative and interpretable analysis.

Benefits of technology

It improves the accuracy and interpretability of diagnosis of neoadjuvant chemotherapy efficacy in breast cancer, reduces computational overhead, enhances the system's fit with clinical practice, and reduces the misdiagnosis rate.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117883062B_ABST
    Figure CN117883062B_ABST
Patent Text Reader

Abstract

This invention discloses a non-invasive artificial intelligence-based diagnostic system for neoadjuvant chemotherapy efficacy in breast cancer, relating to the fields of postoperative efficacy diagnosis and medical AI-assisted technology. It includes a feature extraction module configured to acquire clinical information and MRI images, and extract clinical information features and MRI image features respectively; a feature embedding and fusion module configured to fuse clinical information features and MRI image features using a bidirectional cross-attention mechanism to obtain fused features; an objective function co-optimization module configured to co-optimize the fused features based on a multi-scale deep-supervised objective function to obtain breast cancer lesion information; and a result quantitative analysis and interpretability study module configured to perform quantitative and interpretable diagnostic result analysis on the breast cancer lesion information. This invention enables accurate diagnosis of non-invasive postoperative efficacy, thereby reducing patient suffering and alleviating the burden on doctors.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the fields of postoperative efficacy diagnosis and medical artificial intelligence-assisted technology for breast cancer, and particularly to a non-invasive artificial intelligence-based system for diagnosing the efficacy of neoadjuvant chemotherapy for breast cancer. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] In clinical practice, neoadjuvant chemotherapy is administered before surgery to reduce the stage of locally advanced breast cancer, thereby increasing the feasibility of surgery. Furthermore, the complete pathological response to neoadjuvant chemotherapy is a crucial indicator for assessing patient prognosis, typically determined by pathologists through invasive biopsy. Therefore, there is an urgent need to develop computer-aided diagnostic systems that integrate medical imaging (e.g., mammograms, ultrasound, and MRI) and clinical datasets to assist radiologists in preoperative diagnosis and alleviate patient suffering.

[0004] Current artificial intelligence methods and systems used for diagnosing the efficacy of neoadjuvant chemotherapy in breast cancer mostly employ data-driven approaches, with system input consisting of single-modality patient data. Typically, there is limited interaction between different patient modalities, and systems are often built specifically for one modality. A few systems consider the interactions between patient modalities, but their feature matrix fusion methods are relatively simplistic and fail to truly uncover the interactions and complementary information between modalities.

[0005] The inventors discovered that current artificial intelligence methods and systems used for diagnosing the efficacy of neoadjuvant chemotherapy in breast cancer suffer from three main problems: First, patient data is often used in a single modality, neglecting the interactions between different data modalities, making the system's results unconvincing to physicians and thus unsuitable for clinical application. Second, the feature matrix fusion method is simplistic; simple matrix stacking and addition fail to extract the relationships between different features and incur significant computational costs, leading to decreased diagnostic accuracy. Finally, the methods are detached from clinical practice, contradicting the diagnostic process of radiologists, failing to integrate multiple tasks, and ignoring the supporting role of clinical knowledge, thus casting doubt on the reliability of the diagnostic results. These problems result in low accuracy of diagnostic results for the efficacy of neoadjuvant chemotherapy in breast cancer, significantly deviating from actual clinical practice. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the purpose of this invention is to provide a non-invasive artificial intelligence-based diagnostic system for neoadjuvant chemotherapy in breast cancer, which can accurately diagnose the efficacy of non-invasive postoperative treatment, thereby reducing patient suffering and alleviating the burden on doctors.

[0007] To achieve the above objectives, the present invention is implemented through the following technical solution:

[0008] The first aspect of this invention provides a non-invasive artificial intelligence-based diagnostic system for neoadjuvant chemotherapy efficacy in breast cancer, comprising:

[0009] The feature extraction module is configured to acquire clinical information and MRI images, and extract clinical information features and MRI image features respectively;

[0010] The feature embedding and fusion module is configured to fuse clinical information features and MRI image features using a bidirectional cross-attention mechanism to obtain fused features;

[0011] The objective function collaborative optimization module is configured to collaboratively optimize the fused features based on a multi-scale deep supervision objective function to obtain breast cancer lesion information.

[0012] The quantitative analysis results, within the interpretability study module, are configured to perform quantitative and interpretable diagnostic result analysis of breast cancer lesion information.

[0013] Further, the specific steps for extracting clinical information features and MRI image features are as follows:

[0014] Clinical information and MRI images are mapped to clinical information feature matrices and MRI image feature matrices, respectively.

[0015] The clinical information feature matrix is ​​fed into a multi-head self-attention model to extract the clinical knowledge feature matrix.

[0016] Furthermore, the specific steps for mapping clinical information and MRI images into clinical information feature matrices and MRI image feature matrices, respectively, are as follows:

[0017] Unstructured clinical information is transformed into a clinical information feature matrix using linear mapping operations;

[0018] The convolution operation in deep learning is used to map structured image information into nuclear magnetic resonance image feature matrices.

[0019] Furthermore, the clinical information feature matrix has the same size as the MRI image feature matrix.

[0020] Furthermore, the clinical knowledge feature matrix includes clinical information and case report biomarker information. The clinical knowledge feature matrix serves as a prompting information in the clinical knowledge prompting mechanism to assist in the learning of MRI image information.

[0021] Furthermore, the feature embedding and fusion module includes a feature embedding module that uses a bidirectional cross-self-attention mechanism to embed the clinical knowledge feature matrix into the MRI image feature matrix and shares the Query and Key between the two feature matrices.

[0022] Furthermore, the feature embedding and fusion module also includes a feature matrix fusion module, which is used to achieve matrix fusion based on a shared parameter matrix. The specific steps are as follows:

[0023] Calculate the spatial value matrix and the channel value matrix;

[0024] The shared parameter matrix is ​​calculated based on the shared Query and Key. and

[0025] Shared parameter matrix and Perform cross-attention operations on the spatial value matrix and the channel value matrix respectively to obtain a unified feature matrix;

[0026] The unified feature matrix is ​​added to the original matrix to obtain the fused feature matrix.

[0027] Furthermore, the objective function of multi-scale deep supervision is used to collaboratively optimize the non-invasive diagnostic tasks of breast cancer localization and neoadjuvant chemotherapy efficacy, thereby achieving information sharing and complementarity among breast cancer-related tasks.

[0028] Furthermore, the specific steps for collaboratively optimizing the fused features based on a multi-scale deep-supervised objective function are as follows:

[0029] Calculate the main output of the feature encoder and the deep supervision loss function;

[0030] The tumor localization loss function is obtained by adding the main output of the feature encoder to the deep supervision loss function;

[0031] The balanced cross-entropy loss function is calculated, and the tumor localization loss function and the balanced cross-entropy loss function are combined to obtain the final objective function. The weights of each part of the final objective function are adjusted by hyperparameters.

[0032] Furthermore, the specific steps for quantitative and interpretable diagnostic analysis of breast cancer lesion information are as follows:

[0033] Quantitative and parametric analysis of breast cancer lesion information;

[0034] The Grad-Cam algorithm was used to perform interpretability analysis on breast cancer lesion information.

[0035] The above one or more technical solutions have the following beneficial effects:

[0036] This invention discloses a non-invasive artificial intelligence-based diagnostic system for the efficacy of neoadjuvant chemotherapy in breast cancer. First, the system follows clinical practice procedures, comprehensively considering data from all patient modalities, including MRI images from different time periods, pathological report biomarkers, and clinical follow-up records. A clinical knowledge prompting mechanism addresses the "semantic gap" and decreased diagnostic accuracy caused by the fusion of structured and unstructured data. Second, a parameter-sharing parallel cross-attention module is constructed to achieve efficient fusion of multimodal feature matrices, solving the problem of excessive computational overhead. Third, a multi-scale, deeply supervised objective function is used to collaboratively optimize the non-invasive diagnostic tasks of breast cancer localization and neoadjuvant chemotherapy efficacy, improving the system's diagnostic accuracy and reducing the misdiagnosis rate. Finally, interpretability and quantitative analyses are performed to ensure the system closely aligns with clinical practice and reduces obstacles to implementation.

[0037] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0038] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0039] Figure 1 This is a structural diagram of the breast cancer neoadjuvant chemotherapy efficacy diagnosis system based on non-invasive artificial intelligence, according to an embodiment of the present invention.

[0040] Figure 2 This is a flowchart of the diagnostic process for neoadjuvant chemotherapy efficacy in breast cancer based on non-invasive artificial intelligence, according to an embodiment of the present invention.

[0041] Figure 3 This is a schematic diagram of the feature matrix fusion module based on the clinical knowledge prompting mechanism in an embodiment of the present invention;

[0042] Figure 4 This is a schematic diagram illustrating the interpretability analysis results of breast cancer lesion information using the Grad-cam algorithm in an embodiment of the present invention. Detailed Implementation

[0043] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0044] It should be noted that the embodiments of the present invention involve data such as clinical information and MRI images. When the above embodiments of the present invention are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0045] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0046] Example:

[0047] This invention provides a non-invasive artificial intelligence-based diagnostic system for the efficacy of neoadjuvant chemotherapy in breast cancer, such as... Figure 1 As shown, the method includes a feature extraction module for structured and unstructured features, a feature embedding module based on clinical knowledge prompts, a parameter-sharing parallel cross-attention feature matrix fusion module, a multi-scale deep-supervised objective function collaborative optimization module, and a result quantitative analysis and interpretability research module. It should be noted that the method for diagnosing the efficacy of neoadjuvant chemotherapy in this embodiment is not only applicable to breast cancer but also to the diagnosis of the efficacy of neoadjuvant chemotherapy in other cancers. The data in this embodiment uses breast cancer as an example.

[0048] First, such as Figure 2 As shown, the data used in this system consists of MRI images, pathological report biomarkers, and clinical follow-up records of breast cancer patients who have undergone neoadjuvant chemotherapy at different stages. A "clinical knowledge" prompting mechanism is employed to embed unstructured clinical information into a structured image feature matrix, enhancing multimodal information association and bridging the "semantic gap." Secondly, a parameter-sharing parallel cross-attention module is constructed to achieve matrix fusion of multimodal data, preserving the shape, volume, and texture features of the tumor while reducing the computational burden on the system. Thirdly, a "multi-scale deep supervision" objective function is designed to collaboratively optimize the non-invasive diagnostic tasks of breast cancer localization and neoadjuvant chemotherapy efficacy, improving the system's diagnostic accuracy. Finally, quantitative and visualization analyses are used to verify the system's generalization ability, while simultaneously boosting physicians' confidence in its clinical application.

[0049] Specifically, it includes the following:

[0050] The feature extraction module is configured to acquire clinical information and MRI images, and extract clinical information features and MRI image features respectively. The specific steps are as follows:

[0051] (1) Map clinical information and MRI images to clinical information feature matrices and MRI image feature matrices, respectively; the clinical information feature matrices and MRI image feature matrices have the same size. Specifically, unstructured clinical information is transformed into a clinical information feature matrix using linear mapping operations; structured image information is mapped into an MRI image feature matrix using convolution operations in deep learning.

[0052] (2) The clinical information feature matrix is ​​fed into a multi-head self-attention model to extract the clinical knowledge feature matrix. The clinical knowledge feature matrix contains clinical information and case report biomarker information. The clinical knowledge feature matrix serves as a prompting information in the clinical knowledge prompting mechanism to assist in the learning of MRI image information.

[0053] The feature embedding and fusion module is configured to fuse clinical information features and MRI image features using a bidirectional cross-attention mechanism to obtain fused features. The feature embedding and fusion module includes a feature embedding module and a feature matrix fusion module. By constructing a parameter-shared parallel cross-attention module, it fuses the patient's radiomics, clinicalomics, and case-related features to uncover the clinical heterogeneity of breast cancer and eliminate the "semantic gap" problem caused by different data sources.

[0054] In this embodiment, the bidirectional cross-attention mechanism includes the embedding of clinical knowledge prompting mechanism and the fusion process of cross-attention module.

[0055] The feature embedding module is used to embed the clinical knowledge feature matrix into the MRI image feature matrix using a bidirectional cross-attention mechanism, and to share the Query and Key between the two feature matrices.

[0056] In one specific implementation, a clinical knowledge prompting mechanism is used to embed unstructured clinical information into a structured image feature matrix, and the effect is verified in a real dataset, as shown in Table 1.

[0057] Table 1. Validation table of clinical knowledge prompting mechanism on real datasets

[0058] Accuracy Precision Recall F1 score AUC No "hint" 0.693 0.511 0.548 0.493 0.653 Feature vector addition 0.718 0.545 0.598 0.547 0.687 Feature vector stacking 0.801 0.790 0.585 0.621 0.740 "clinical knowledge" hint 0.955 0.865 1.000 0.923 0.970

[0059] The clinical information includes clinical biomarker information from physician follow-ups and pathology report information after patient biopsies. In this embodiment, the clinical information and case report biomarkers are represented as unstructured feature matrices, while MRI images are represented as structured feature matrices. The clinical knowledge prompting mechanism also considers multimodal feature matrices to align with the diagnostic process used by physicians in clinical practice. Specifically, firstly, the mapped clinical information feature matrix FM... clinicalThe input is fed into a standard multi-head self-attention module to generate a feature matrix containing clinical information and case report biomarkers. This feature matrix serves as a cue in the clinical knowledge prompting mechanism to assist in the learning of MRI image information. Accordingly, the MRI image feature matrix is ​​mapped to FM. image Secondly, the unstructured feature matrix FM clinical and structured feature matrix FM image The data is fed into the cross-attention module for matrix fusion, during which the parameter matrix is ​​shared. This enhances the interrelationship of multimodal feature matrices and prevents the "semantic gap" from occurring during the fusion process. Finally, multilayer perceptron operations are used to recover the tumor detail information lost during the fusion process, and the fused feature matrix is ​​fused with the original MRI image feature matrix in the same way as the cross-attention module mentioned above.

[0060] The feature matrix fusion module is used to achieve matrix fusion based on a shared parameter matrix. Specifically, it utilizes the proposed parallel parameter shared attention module to fuse different feature matrices. The Query matrix and Key matrix are shared to enhance the connection between different feature matrices. Simultaneously, the spatial Value matrix and channel Value matrix are obtained from the data separately. Compared to existing methods, the computational cost is significantly reduced. Figure 2 As shown, the specific steps are as follows:

[0061] S1: Obtain different feature matrices. This involves using linear mapping operations to transform unstructured information (patient clinical data) into feature matrices, and using convolution operations in deep learning to map structured information (MRI image information from different time periods) into feature matrices. The feature matrices are then used in FM... clinical and FM image This indicates that, compared to existing methods, linear mapping can extract more complete unstructured information about the patient, avoiding the problem of excessively sparse information caused by simple feature matrix fusion. In addition, the structured feature matrix takes into account the information complementarity between MRI image features at different stages of the patient's treatment. Therefore, when fusing with the unstructured feature matrix, the structured feature matrix with more comprehensive tumor features is selected for fusion to fully explore the relationship between MRI images and clinical data at different stages of the patient's treatment.

[0062] S2: Calculate the spatial Value matrix and the channel Value matrix. Before feature matrix fusion, the spatial Value matrix and the channel Value matrix are calculated separately, resulting in the Query, Key, and Value matrices of both Value matrices for cross-attention operations.

[0063] S3: Calculate the shared parameter matrix based on the shared Query and Key. and To reduce computational cost and explore the correlation of feature matrices in spatial and channel dimensions, the two value matrices share the query and key, resulting in the following parameter matrix: and

[0064] S4: Share the parameter matrix and Cross-attention operations are performed on both the spatial and channel value matrices to obtain a unified feature matrix. The resulting matrix after fusing the parallel cross-attention feature matrices of the spatial and channel dimensions is as follows: and The calculation process of the parallel characteristic matrix is ​​shown in formulas (1) and (2):

[0065]

[0066]

[0067] in, and These represent the shared feature parameter matrices Query and Key, as well as the channel feature parameter matrix and the spatial feature parameter matrix, respectively.

[0068] Furthermore, the parallel cross-attention mechanism can extract the interaction information between the parameter matrices Query and Key, ensuring improved performance after feature matrix fusion. The two parallel attention modules use A... c and A s The specific calculation process is shown in formulas (3) and (4):

[0069]

[0070]

[0071] Where d is the size of the matrix, and the generated feature matrix A c and A s Vector stacking and fusion are performed to obtain a unified feature matrix.

[0072] The steps S2 to S4 above are the computation process of the self-attention mechanism, which aims to extract tumor detail information lost during feature fusion from the spatial and channel dimensions.

[0073] S5: Add the unified feature matrix to the original matrix to obtain the fused feature matrix. Here, the original matrix is ​​the mapped structured matrix FM. image(The matrix obtained after convolution of the MRI image). To preserve the detailed information of the tumor region in the original MRI image, including the shape, texture and volume of the tumor, the original feature matrix is ​​vector-wise added to the matrix generated in S4 to obtain the final feature matrix. The feature matrix fusion method of this invention has a significant improvement over the existing feature fusion methods, as shown in Table 2.

[0074] Table 2 compares the performance of the parameter-sharing parallel cross-attention module with other feature matrix fusion methods.

[0075]

[0076]

[0077] The objective function co-optimization module is configured to co-optimize the fused features using a multi-scale deep-supervised objective function. This allows the model to focus on breast cancer boundary, texture, and grayscale features, assisting the efficacy diagnosis model in learning supplementary information and obtaining breast cancer lesion information. By designing a multi-scale deep-supervised objective function to constrain the accuracy of feature extraction during the encoding stage, and by learning features at different scales, the shape, volume, and texture features of the tumor are preserved. The multi-scale deep-supervised objective function is used to co-optimize the non-invasive diagnostic tasks of breast cancer localization and neoadjuvant chemotherapy efficacy, achieving information sharing and complementarity among breast cancer-related tasks.

[0078] The specific steps for collaboratively optimizing the fused features based on a multi-scale deep-supervised objective function are as follows:

[0079] The main output l of the computation feature encoder main Calculate the deep supervised loss function L side The location information of the tumor and the information of the feature matrices at different scales during the feature extraction process are constrained respectively.

[0080] The main output of the feature encoder main With deep supervision loss function L side The summation yields the tumor localization loss function L. MS ;

[0081] Calculate the balanced cross-entropy loss function L CE The final objective function is obtained by combining the tumor localization loss function and the balanced cross-entropy loss function, and the weights of each part of the final objective function are adjusted by the hyperparameter α.

[0082] In one specific implementation, since pixel-level labels can provide information about the tumor surroundings, such as the shape, volume, and texture of the tumor, this invention designs an MS-loss function to supervise feature maps at different scales during the decoding process, as shown in formula (5):

[0083] LMS =l main +L side (5)

[0084] Among them, l main The main output of the feature encoder primarily constrains the location and regional features of the tumor in the patient's MRI images, with a deep supervision loss function L. side The side outputs of the feature encoder, composed of feature maps at different scales, are used to constrain the encoder to maintain attention during learning and keep it focused on tumor features. The objective function is specified as formula (6):

[0085] L MS =[l DICE (Y T ,Y P )+l BCE (Y T ,Y P )]+[l BCE (Y T ,Y N )] / N ⑹,

[0086] Among them, Y T and Y P Y represents the true value and the model prediction, respectively. N This represents the predicted value of the side output during the feature encoding process of the model, where N represents the number of side outputs. DICE and l BCE Let the objective functions of DICE and binary cross-entropy be represented respectively:

[0087]

[0088] l BCE (Y T ,Y P )=-[Y P ·log(Y T )+(1-Y P )·log(1-Y T (8),

[0089] L CE (Y T ,Y P )=-Y T ·log(Y P (9).

[0090] Furthermore, to ensure the acquisition of breast cancer location and diagnostic information during feature extraction, this invention proposes a multi-scale deep supervised objective function to collaboratively optimize the non-invasive diagnostic task of breast cancer localization and neoadjuvant chemotherapy efficacy. The weights of the objective function are adjusted using the hyperparameter α to achieve the optimal result, namely, the multi-scale deep supervised objective function and the balanced cross-entropy loss function L. CE Together, they constitute the overall objective function L for model co-optimization. total Therefore, the final objective function is shown in equation (10):

[0091] L total =α·L MS +(1-α)·L CE (10).

[0092] To verify the optimal value of the hyperparameter α, this invention was tested on a real dataset. The experiment shows that the best results are achieved when α = 0.3. The verification results are shown in Table 3.

[0093] Table 3. Verification table of the optimal value of α in the objective function

[0094]

[0095]

[0096] The quantitative analysis results, within the interpretability study module, are configured to perform quantitative and interpretable diagnostic analysis of breast cancer lesion information. In this embodiment, the diagnostic process for surgical efficacy is a quantitatively optimized system, and its accuracy is demonstrated through five-fold cross-validation and interpretability analysis.

[0097] This includes quantitative and parametric analysis of breast cancer lesion information;

[0098] The Grad-Cam algorithm was used to perform interpretability analysis on breast cancer lesion information.

[0099] Tables 1 to 3 in this invention present quantitative analyses. Compared with existing artificial intelligence methods and systems for diagnosing the efficacy of neoadjuvant chemotherapy in breast cancer, this invention achieves optimal results in five indicators: Accuracy, Precision, Recall, F1 score, and AUC, which are 0.96, 0.87, 1.00, 0.92, and 0.97, respectively. Furthermore, the Grad-cam algorithm is used to print out the focus areas of the breast cancer region using a gradient backpropagation computational mechanism.

[0100] The Grad-CAM algorithm can detect areas with significant gradient changes in deep learning models; specifically, it can detect the boundaries and regions of breast cancer in MR images. When a highlighted area appears in the target region, it indicates that the model has effectively focused on the target region.Figure 4 As shown, each row represents a patient, and (a), (b), (c), and (d) represent: pre-enhanced MR image, post-enhanced MR image, actual breast cancer condition, and model focus, respectively.

[0101] This invention provides a non-invasive artificial intelligence method and system for diagnosing the efficacy of neoadjuvant chemotherapy in breast cancer, belonging to the fields of postoperative efficacy diagnosis and medical artificial intelligence-assisted technology. The non-invasive efficacy diagnosis system includes: feature extraction from MRI images embedded with clinical knowledge; effective fusion of multi-omics data (MRI images and patient clinical manifestations); and synergistic optimization of tumor localization and non-invasive efficacy diagnosis. The clinical knowledge originates from the patient's post-biopsy pathology report and corresponding clinical follow-up indicators. A "clinical prompting" mechanism is used to embed the patient's clinical information matrix into the MRI image matrix. A parameter-sharing parallel cross-attention module is constructed to fuse features from different patient omics. A "multi-scale deep supervision" objective function is designed to synergistically optimize the non-invasive diagnostic task of breast cancer localization and neoadjuvant chemotherapy efficacy. Therefore, this invention can utilize patient information for non-invasive postoperative efficacy diagnosis, reducing patient suffering and alleviating the burden on doctors.

[0102] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0103] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A diagnostic system for the efficacy of neoadjuvant chemotherapy in breast cancer based on non-invasive artificial intelligence, characterized in that, include: The feature extraction module is configured to acquire clinical information and MRI images, and extract clinical information features and MRI image features respectively. Specifically, the clinical information and MRI images are mapped into clinical information feature matrices and MRI image feature matrices respectively. The clinical information feature matrices are fed into a multi-head self-attention model to extract a clinical knowledge feature matrix. The clinical knowledge feature matrix contains clinical information and case report biomarker information. The clinical knowledge feature matrix serves as a prompting information in the clinical knowledge prompting mechanism to assist in the learning of MRI image information. The feature embedding and fusion module is configured to fuse clinical information features and MRI image features using a bidirectional cross-attention mechanism to obtain fused features. Specifically, the feature embedding and fusion module includes a feature embedding module, which uses a bidirectional cross-self-attention mechanism to embed the clinical knowledge feature matrix into the MRI image feature matrix and shares the information between the two feature matrices. and The feature embedding and fusion module also includes a feature matrix fusion module, which is used to achieve matrix fusion based on a shared parameter matrix. The specific steps are: computation space... Matrix and Channel Matrix; based on shared and Calculate the shared parameter matrix and Shared parameter matrix and Separate in space Matrix and Channel Perform cross-attention operation on the matrix to obtain a unified feature matrix; add the unified feature matrix to the original matrix to obtain the fused feature matrix; The objective function co-optimization module is configured to co-optimize the fused features based on a multi-scale deep-supervised objective function to obtain breast cancer lesion information. The multi-scale deep-supervised objective function is used to co-optimize the non-invasive diagnostic tasks of breast cancer localization and neoadjuvant chemotherapy efficacy, thereby realizing information sharing and complementarity among breast cancer-related tasks. The quantitative analysis results, within the interpretability study module, are configured to perform quantitative and interpretable diagnostic result analysis of breast cancer lesion information.

2. The breast cancer neoadjuvant chemotherapy efficacy diagnostic system based on non-invasive artificial intelligence as described in claim 1, characterized in that, The specific steps for mapping clinical information and MRI images into clinical information feature matrices and MRI image feature matrices, respectively, are as follows: Unstructured clinical information is transformed into a clinical information feature matrix using linear mapping operations; The convolution operation in deep learning is used to map structured image information into nuclear magnetic resonance image feature matrices.

3. The breast cancer neoadjuvant chemotherapy efficacy diagnostic system based on non-invasive artificial intelligence as described in claim 1, characterized in that, The clinical information feature matrix has the same size as the MRI image feature matrix.

4. The breast cancer neoadjuvant chemotherapy efficacy diagnostic system based on non-invasive artificial intelligence as described in claim 1, characterized in that, The specific steps for collaboratively optimizing the fused features based on a multi-scale deep-supervised objective function are as follows: Calculate the main output of the feature encoder and the deep supervision loss function; The tumor localization loss function is obtained by adding the main output of the feature encoder to the deep supervision loss function; The balanced cross-entropy loss function is calculated, and the tumor localization loss function and the balanced cross-entropy loss function are combined to obtain the final objective function. The weights of each part of the final objective function are adjusted by hyperparameters.

5. The breast cancer neoadjuvant chemotherapy efficacy diagnostic system based on non-invasive artificial intelligence as described in claim 1, characterized in that, The specific steps for quantitative and interpretable diagnostic analysis of breast cancer lesion information are as follows: Quantitative and parametric analysis of breast cancer lesion information; The Grad-Cam algorithm was used to perform interpretability analysis on breast cancer lesion information.