Precise septal tumor diagnosis system based on artificial intelligence

Abstract

The invention provides a septal tumor accurate diagnosis system based on artificial intelligence, a multi-scale attention fusion module receives multi-modal data, focuses a septal region, dynamically adjusts the weight and outputs a key feature vector, an adversarial self-supervision pre-training module digs tumor morphological features based on the vector, robustness and generalization ability are enhanced, and the accuracy of diagnosis is improved. And the causal intervention diagnosis decision-making module constructs a decision-making tree by using enhanced features to perform preliminary diagnosis and transmits causal information to the meta-learning enhanced diagnosis module, and the meta-learning enhanced diagnosis module adjusts parameters by using a small number of samples for rare subtypes, outputs a final diagnosis result and feeds back the final diagnosis result. According to the system, multi-mode and dynamic attention, cooperative confrontation self-supervision pre-training and causal reasoning are fused, rare tumor diagnosis is optimized through meta-learning, a closed-loop feedback and shared knowledge graph is formed, and the diagnosis precision, interpretability and rare case diagnosis capacity are integrally improved.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and medical diagnostic technology, specifically relating to an intelligent diagnostic system for mediastinal tumors that integrates multimodal data analysis, causal reasoning, and meta-learning mechanisms. Background Technology

[0002] Current mediastinal tumor diagnostic techniques primarily rely on physicians' subjective judgment based on single-modality imaging data (such as CT images), limited pathological information, or clinical indicators. Diagnostic results are heavily influenced by physician experience and expertise. Furthermore, due to the diverse morphologies and rare subtypes of mediastinal tumors, traditional methods struggle to effectively handle the heterogeneity of multimodal data and lack the ability to learn from rare tumor samples, easily leading to misdiagnosis or missed diagnosis. In addition, most existing diagnostic techniques are based on correlation analysis, which struggles to accurately identify causal characteristics affecting diagnostic results, leading to causal confusion and compromising the accuracy and reliability of diagnoses, thus failing to meet the clinical need for precise diagnosis of mediastinal tumors. Summary of the Invention

[0003] To address the shortcomings and problems of existing technologies, this invention provides a mediastinal tumor precision diagnosis system that integrates multimodal attention mechanisms, adversarial learning, causal intervention, and meta-learning, thereby solving the problems of data heterogeneity and causal confusion.

[0004] The solution to the technical problem of this invention is: to adopt an artificial intelligence-based precision diagnosis system for mediastinal tumors, including a multi-scale attention fusion module, an adversarial self-supervised pre-training module, a causal intervention diagnosis decision module, and a meta-learning enhanced diagnosis module, etc. The multi-scale attention fusion module is configured to receive multimodal data from CT / MRI images, pathological text reports, and clinical indicators. It focuses on the mediastinal region using an anatomically perceptive attention mechanism and adjusts the weights of different modalities through dynamic modality complementarity scoring, outputting a feature vector containing key information about mediastinal tumors. The adversarial self-supervised pre-training module is configured to mine tumor morphological features based on the feature vector through a self-supervised task of lesion region masking and adversarial generation perturbation, enhancing feature robustness and generalization ability. The trained features are then fed back to the multi-scale attention fusion module and output to the causal intervention diagnostic decision module and the meta-learning enhanced diagnostic module. The causal intervention diagnostic decision module utilizes the enhanced features to identify causal features affecting diagnosis based on a causal graph model, constructs a diagnostic decision tree for preliminary diagnosis, outputs preliminary diagnostic results, and transmits the causal reasoning process and key causal features to the meta-learning enhanced diagnostic module. The meta-learning enhanced diagnostic module receives the causal information, combines it with a meta-learning mechanism, adjusts model parameters using a small number of samples for rare mediastinal tumor subtypes, outputs the final diagnostic results, and feeds back the diagnostic process and results to the multi-scale attention fusion module.

[0005] Preferably, the system further includes a shared information base, which comprises a shared feature base, a causal knowledge graph base, and a meta-learning experience pool. The shared feature base stores the original features extracted by the multi-scale attention fusion module, the features enhanced by the adversarial self-supervised pre-training module, and receives the feature-based analysis results from the causal intervention diagnosis decision module and the meta-learning enhanced diagnosis module to optimize feature extraction and enhancement strategies. The causal knowledge graph base stores the knowledge graph constructed by the causal intervention diagnosis decision module, which includes causal relationships between tumor location, tissue type, and clinical symptoms. This knowledge is used by the multi-scale attention fusion module to adjust the attention weights of modal data and by the meta-learning enhanced diagnosis module to accelerate the learning and diagnosis of rare tumors. The meta-learning experience pool stores the learning experience accumulated by the meta-learning enhanced diagnosis module in processing rare tumor types, which is used by the adversarial self-supervised pre-training module to optimize the training process.

[0006] Preferably, it also includes a joint training optimization module, configured to, during the model training phase, simultaneously adjust the parameters of the multi-scale attention fusion module, the adversarial self-supervised pre-training module, the causal intervention diagnosis decision module, and the meta-learning enhanced diagnosis module through the backpropagation algorithm, with the accuracy and F1 score of the final diagnostic result as optimization objectives.

[0007] Preferably, it also includes a feedback adjustment module, configured to establish a two-way feedback mechanism. Based on the output of each module and the final diagnostic result, when the meta-learning enhanced diagnostic module is not effective in diagnosing rare tumors, it triggers the causal intervention diagnostic decision module to re-examine the causal reasoning process, adjust the causal feature recognition and utilization, and at the same time triggers the multi-scale attention fusion module to strengthen the extraction and fusion of rare tumor-related features.

[0008] Preferably, in the multi-scale attention fusion module, the attention mechanism for anatomical structure perception is expressed by a formula. Calculate the attention weight A at position (i,j) of the image. ij , where f att For the attention computation function, I ij This represents the feature vector of the image at position (i,j); expressed by the formula Output the fused feature vector F, where S k f is the dynamic modal complementarity score for the k-th mode. k Let m be the feature vector extracted from the k-th mode, and m be the number of modes.

[0009] Preferably, in the adversarial self-supervised pre-training module, the loss function is reconstructed. Optimize the self-supervised task of lesion region masking, where N is the number of samples and F... i Fμ is the original feature vector. i The reconstructed feature vectors; using the adversarial loss function Ladv =E F [logD(F)]+E F Adversarial training is performed using [log(1-D(G(F)))], where G is the generator and D is the discriminator; the final training objective function is L = L rec +λL adv λ is the balance coefficient.

[0010] Preferably, in the causal intervention diagnostic decision-making module, the causal effect calculation formula is used. Calculate the causal effect TE(X→Y) of the characteristic variable X on the diagnostic outcome variable Y; using information gain As the primary factor in the splitting of decision tree nodes for diagnosis, the gain is based on causal effects. As the second factor in the splitting of the diagnostic decision tree nodes, the comprehensive gain M is used as the basis for splitting the diagnostic decision tree nodes. The comprehensive gain MIG = IG(s, a) + kCIG(s, a), where K is the weight and K is greater than 1, S is the sample set, a is the feature, and Values(a) is the set of values ​​for feature a. v Let be the subset of samples where feature a takes the value v, and H be the information entropy.

[0011] Preferably, in the meta-learning enhanced diagnostic module, based on the meta-learning mechanism, for task T... i Through formula Adjust the model parameters, where θ is the initial parameter. i′ These are the adjusted parameters, where α is the learning rate. For the model in task T i The loss is based on the parameter θ; the objective function is learned through meta-learning. Optimize the model, where M is the number of tasks.

[0012] The beneficial effects of this invention are as follows: 1. Deep integration of multimodal fusion and dynamic attention mechanisms: By focusing on the mediastinal region through an anatomical structure-aware attention mechanism, and utilizing dynamic modality complementarity scoring to adjust the weights of multimodal data such as CT / MRI images, pathological texts, and clinical indicators in real time. Existing technologies mostly employ fixed weights or single-modality analysis. This design can adaptively enhance key diagnostic information (such as the correlation between tumor location and pathological features), improving the feature extraction accuracy of complex cases. In the diagnosis of mediastinal metastasis in lung cancer, the weights of imaging tumor boundary features and pathological immunohistochemical indicators can be enhanced simultaneously, avoiding the omission of information from a single modality.

[0013] 2. Cross-module collaboration between adversarial self-supervised pre-training and causal inference: This approach mines tumor morphological features through a lesion region masking self-supervised task and adversarial generative perturbations, then constructs a diagnostic decision tree using a causal graphical model. Existing techniques largely rely on supervised learning pre-training, and the diagnostic process lacks causal interpretability. This design improves feature robustness even without a large amount of labeled data, while enhancing diagnostic interpretability through causal feature identification (e.g., the causal relationship between tumor size and metastasis risk). In the differential diagnosis of mediastinal lymphoma and sarcoidosis, causal inference can clarify the causal association between clinical symptoms (e.g., fever) and pathological type, reducing misdiagnosis.

[0014] 3. A meta-learning-driven optimization mechanism for rare tumor diagnosis utilizes meta-learning to rapidly adjust model parameters using a small number of samples for rare mediastinal tumor subtypes (such as thymoid carcinoids). A feedback adjustment module, in conjunction with a causal intervention module, re-examines the inference process. Existing technologies often suffer from low diagnostic accuracy for rare cases due to insufficient sample size; this design overcomes this data dependency bottleneck. In the diagnosis of rare thymoma subtypes, only 10-20 samples are needed to optimize the model through a meta-learning experience pool, while traditional methods require hundreds of samples.

[0015] 4. A closed-loop feedback and shared knowledge graph system optimization architecture: This architecture stores features, causal knowledge graphs, and meta-learning experience in a shared information repository, forming a closed loop of "feature extraction - diagnostic decision-making - experience feedback." Existing technologies are mostly unidirectional processes, lacking cross-module knowledge accumulation and optimization. This architecture can continuously accumulate diagnostic experience from rare cases; for example, the causal knowledge graph repository can store association rules between the location and tissue type of mediastinal tumors for reference in new cases. Attached Figure Description

[0016] Figure 1 This is a diagram showing the interaction between the various modules of the system and the shared information database in this invention.

[0017] Figure 2 This is a layered architecture diagram of the system of this invention;

[0018] Figure 3 This is a block diagram showing the progressive link between modules in the system of this invention;

[0019] Figure 4 This is a flowchart of the system operation of the present invention. Detailed Implementation

[0020] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0021] Example 1: Architecture of an AI-based precision diagnosis system for mediastinal tumors Figure 1As shown, this system mainly consists of a multi-scale attention fusion module, an adversarial self-supervised pre-training module, a causal intervention diagnosis decision module, a meta-learning enhanced diagnosis module, a shared information database, a joint training optimization module, and a feedback regulation module. These modules collaborate with each other to achieve accurate diagnosis of mediastinal tumors.

[0022] The multi-scale attention fusion module is responsible for receiving multimodal data such as CT / MRI images, pathology text reports, and clinical indicators. It uses an anatomical structure-aware attention mechanism to focus on the mediastinal region and adjusts the weights of different modal data according to the dynamic modality complementarity score. Finally, it outputs a feature vector containing key information about mediastinal tumors.

[0023] The adversarial self-supervised pre-training module, based on the feature vector output by the multi-scale attention fusion module, mines tumor morphological features through a self-supervised task of lesion region masking and adversarial generation of perturbations, enhancing the robustness and generalization ability of the features. The trained features are fed back to the multi-scale attention fusion module to optimize the weights of the original feature fusion; and output to the causal intervention diagnostic decision module and the meta-learning enhanced diagnostic module to support subsequent diagnosis.

[0024] The causal intervention diagnosis decision module utilizes enhanced features to identify causal features affecting diagnosis based on a causal graph model, constructs a diagnostic decision tree for preliminary diagnosis, outputs preliminary diagnostic results, and transmits the causal reasoning process and key causal features to the meta-learning enhanced diagnosis module.

[0025] Meta-learning-enhanced diagnostic module: Receives causal information from the causal intervention diagnostic decision module, combines it with the meta-learning mechanism, adjusts model parameters using a small number of samples for rare mediastinal tumor subtypes, outputs the final diagnostic results, and feeds the diagnostic process and results back to the multi-scale attention fusion module to further optimize system performance.

[0026] The shared information repository comprises a shared feature library, a causal knowledge graph library, and a meta-learning experience pool. The shared feature library stores the original features extracted by the multi-scale attention fusion module, the enhanced features from the adversarial self-supervised pre-training module, and the feature-based analysis results from the causal intervention diagnosis decision module and the meta-learning enhanced diagnosis module, to optimize feature extraction and enhancement strategies. The causal knowledge graph library stores the knowledge graph constructed by the causal intervention diagnosis decision module, containing causal relationships between tumor location, tissue type, and clinical symptoms. This is used by the multi-scale attention fusion module to adjust the attention weights of modal data and by the meta-learning enhanced diagnosis module to accelerate the learning and diagnosis of rare tumors. The meta-learning experience pool stores the learning experience accumulated by the meta-learning enhanced diagnosis module in processing rare tumor types, for the adversarial self-supervised pre-training module to reference and optimize the training process. The joint training optimization module, during model training, uses the backpropagation algorithm to simultaneously adjust the parameters of the multi-scale attention fusion module, the adversarial self-supervised pre-training module, the causal intervention diagnosis decision module, and the meta-learning enhanced diagnosis module, with the accuracy and F1 score of the final diagnostic results as optimization objectives, to improve the overall system performance.

[0027] The feedback adjustment module establishes a two-way feedback mechanism. Based on the output of each module and the final diagnostic results, when the meta-learning enhanced diagnostic module is not effective in diagnosing rare tumors, it triggers the causal intervention diagnostic decision module to re-examine the causal reasoning process, adjust the identification and utilization of causal features, and at the same time triggers the multi-scale attention fusion module to strengthen the extraction and fusion of rare tumor-related features.

[0028] Multimodal data input includes: CT / MRI images acquired through the hospital's Picture Archiving and Communication System (PACS). These images, acquired by specialized imaging equipment, clearly present the anatomical structures of the mediastinal region, providing crucial information such as the tumor's location, size, shape, density, signal characteristics, and relationship to surrounding tissues. For example, CT images accurately show the tumor's calcification and its proximity to blood vessels, helping to determine the tumor's nature and surgical risks; MRI images, with their high resolution for soft tissues, better display the tumor's internal structure and extent of invasion. Pathology reports originate from the pathology department's analysis of tumor tissue biopsies or surgically removed samples. These reports include the tumor's histological type, such as thymoma, teratoma, or lymphoma, crucial for determining the tumor's nature and origin; cell differentiation and malignancy, such as pathological grading, reflecting the tumor's biological behavior and prognosis; and immunohistochemical results, such as biomarker expression, which help determine the tumor's origin and biological behavior, providing a basis for subsequent treatment selection. Clinical data were extracted from the hospital's electronic medical record (EMR) system, covering laboratory tests, patient symptoms and signs, and medical history and treatment records. Tumor markers in laboratory tests, such as AFP, β-HCG, and neuron-specific enolase (NSE), are closely related to the occurrence and development of tumors. Blood routine tests and liver and kidney function tests reflect the patient's overall physical condition and assess treatment tolerance. Patient symptoms and signs included accompanying symptoms such as cough, chest pain, dyspnea, dysphagia, and myasthenia gravis, which are important for indicating the presence and possible type of tumor. Medical history and treatment records included the onset time, previous treatment plans, and complications, helping doctors understand the patient's disease progression and treatment response, providing a reference for comprehensive diagnosis.

[0029] The attention mechanism for anatomical structure perception focuses on the mediastinal region through an attention computation function. Assume the input image is I, I... ij Let A represent the feature vector of the image at position (i,j), and let A be the attention weight of the image at position (i,j). ij Calculated using the following formula:

[0030]

[0031] Among them, f attThe attention calculation function can be implemented using a multilayer perceptron (MLP) or similar methods. It obtains an attention score by performing a nonlinear transformation on the feature vectors. This mechanism evaluates the feature vectors at each location in the image, assigning higher attention weights to features in the mediastinal region and lower weights to features in other irrelevant regions, thereby highlighting the key information in the mediastinal region. In practical applications, the input CT / MRI images are first preprocessed, such as through normalization and cropping, to meet the model's input requirements. Then, the preprocessed image is input into the attention mechanism module, where the attention calculation function calculates the attention weight for each location, resulting in an attention map. Finally, the attention map is weighted and fused with the original image features to obtain a feature representation focused on the mediastinal region.

[0032] Dynamic modal complementarity scoring and weight adjustment: Dynamic modal complementarity scoring measures the degree of complementarity between different modalities. For data of the i-th and j-th modalities, a suitable algorithm is designed to calculate their complementarity score S. ij For example, machine learning-based methods use a large amount of labeled data to train models and learn the correlation and complementarity between different modalities of data. If the imaging shows that the tumor morphology matches the pathological results, and the tumor markers in the clinical examination also show a corresponding abnormal increase, it indicates that these modalities of data corroborate each other, and the complementarity score is high; conversely, if there is inconsistency or conflict in the information, the complementarity score is low.

[0033] The weights of different modal data are dynamically adjusted based on the complementarity score. Let S be the dynamic modal complementarity score of the i-th modality. k The feature vector extracted from the k-th mode is f k The fused feature vector F is then output using the following formula:

[0034]

[0035] Where m represents the number of modalities. Modal combinations with high complementarity scores are given higher weights, indicating that these modal data have strong consistency and complementarity in describing key information of mediastinal tumors and contribute significantly to the final feature vector; while modalities with low complementarity scores are given lower weights to reduce their impact on the final result.

[0036] The feature vector output is derived from features extracted from different modalities that are relevant to key information about mediastinal tumors. These features are then combined to form a feature vector containing key information about the mediastinal tumor. Morphological features of the tumor, such as volume, surface area, and shape factor, as well as density features, such as mean density and standard deviation, are extracted from CT images. Signal features, such as T1 and T2 signal intensity and signal homogeneity, are extracted from MRI images. Metabolic features, such as standard uptake value (SUV), are extracted from PET-CT. Tumor marker values ​​and patient symptoms are extracted from clinical examinations. These extracted features are arranged in a specific order to construct the feature vector. For example, the feature vector can be represented as [tumor location code, tumor size information, tumor density / signal value, relationship with surrounding tissues code, metabolic value, tumor marker value, symptom code, etc.], where each element corresponds to a specific feature. After adjusting the weights through dynamic modality complementarity scoring, the importance of features from different modalities in the feature vector is reasonably reflected, which is more conducive to accurately describing the characteristics of mediastinal tumors and providing valuable information for subsequent diagnosis, treatment, and research.

[0037] The adversarial self-supervised pre-training module is configured to mine tumor morphological features based on the aforementioned feature vectors through a self-supervised task of masking lesion regions and adversarial generation of perturbations, enhancing feature robustness and generalization ability. The trained features are then fed back to the multi-scale attention fusion module and output to the causal intervention diagnosis decision module and the meta-learning enhanced diagnosis module. This adversarial self-supervised pre-training module combines adversarial learning and self-supervised learning. Self-supervised learning utilizes the structural information of the data itself to design training tasks, allowing the model to autonomously learn data features without manual annotation. For example, in a mask prediction task, the model predicts the morphological features of the masked tumor based on the unmasked surrounding anatomical structures, thereby learning the spatial relationship between the tumor and surrounding tissues and its own morphological patterns. Adversarial learning introduces a generative adversarial mechanism, similar to a generative adversarial network (GAN). A generator produces perturbations or adversarial samples, which are added to the input data, forcing the model to learn more stable and interference-resistant features. Adding specific perturbations to CT images causes the model to misjudge tumor boundaries, but through adversarial training, the model gradually learns to ignore the perturbations and focus on the true morphological features. In this way, the model can maintain the stability of features when faced with noise, disturbances, or data bias, while also accurately extracting features from unseen new cases, thus improving the model's robustness and generalization ability.

[0038] In medical imaging, tumor regions are "masked" or marked using binary matrices, forming lesion area masks. Tumor regions in CT / MRI images are masked, and the model then predicts the morphological features of the masked tumor, such as shape and boundaries, based on the surrounding anatomical structures that are not masked. Through continuous prediction and correction, the model can deeply learn the spatial relationship between the tumor and surrounding tissues, as well as its own morphological patterns, thereby enhancing its understanding and extraction of tumor morphological features. During training, a reconstruction loss function L is defined. recon To optimize the self-supervised task of masking lesion regions:

[0039]

[0040] Where N is the sample size, F i The original feature vector, This represents the reconstructed feature vector. By minimizing the reconstruction loss function, the model can continuously adjust its parameters, improving the accuracy of predicting tumor morphological features.

[0041] Perturbations are small, imperceptible disturbances added to input data, such as slight offsets to pixel values ​​or the addition of noise. Adversarial networks dynamically generate perturbations that can "fool" the model and add them to feature vectors or images. Specific perturbations are added to CT images to cause the model to misjudge tumor boundaries. However, during adversarial training, the model gradually learns to ignore these perturbations and focus on the true morphological features of the tumor, thus improving the model's robustness. In adversarial training, an adversarial loss function L is defined. adv :L adv =E F [logD(F)]+E F [log(1-D(G(F)))] (4);

[0042] Where G is the generator and D is the discriminator. The generator aims to generate perturbations that cause the discriminator to misclassify; the discriminator aims to accurately distinguish between real data and perturbed data. Through adversarial training, the model can learn essential features that are unaffected by perturbations. The final training objective function is...

[0043] L = L rec +λL adv (5);

[0044] Where λ is the balance coefficient.

[0045] After adversarial self-supervised pre-training, the robustness and generalization ability of the feature vectors are enhanced. A portion of the trained feature vectors is fed back to the multi-scale attention fusion module to optimize the weights of the original feature fusion, enabling the module to better fuse multimodal data and extract more accurate features. Another portion of the feature vectors is output to the causal intervention diagnosis decision module and the meta-learning enhanced diagnosis module, providing more reliable feature representations for subsequent diagnostic decisions. During the feedback process, the weights and content of the fed-back feature vectors are dynamically adjusted based on the model's training performance and diagnostic performance evaluation, achieving continuous optimization of the entire diagnostic system.

[0046] The causal intervention diagnostic decision-making module is configured to utilize enhanced features, identify causal features influencing diagnosis based on a causal graph model, construct a diagnostic decision tree for preliminary diagnosis, output preliminary diagnostic results, and pass the causal reasoning process and key causal features to the meta-learning enhanced diagnostic module. The causal graph model uses a graph structure to represent causal relationships between variables. Nodes represent events, such as tumor size, tissue type, and biomarker levels, while edges represent causal links, such as "tumor invasion of nerves → chest pain." When constructing the causal graph model for mediastinal tumors, a large amount of clinical data is first collected, including patients' imaging characteristics, pathological results, clinical symptoms, and treatment responses. Then, based on medical domain knowledge and expert experience, the causal relationships between variables are determined. It is known that the pathological grade of a tumor is a direct factor affecting its malignancy, and tumor invasion of blood vessels can lead to clinical symptoms such as chest pain; thus, corresponding causal links are constructed. Causal discovery algorithms, such as the PC algorithm, are used to mine potential causal relationships from the data, further refining the causal graph model. The final constructed causal graph model clearly demonstrates the causal relationships between factors related to mediastinal tumors, providing a foundation for subsequent causal feature identification and diagnostic decision-making.

[0047] Causal characteristics are features that have a direct causal impact on the diagnostic outcome. In the diagnosis of mediastinal tumors, key causal characteristics are identified by analyzing causal diagram models and clinical data. The "pathological grade" of mediastinal tumors is a causal characteristic that directly determines the degree of malignancy; whether the tumor invades surrounding tissues such as the trachea and blood vessels is also an important causal characteristic, as it directly affects the patient's symptoms and prognosis. Some characteristics that are only correlated but not causal, such as the patient's place of residence and occupation, are excluded through rigorous causal analysis. These factors may be associated with tumor occurrence but are not the direct cause of the tumor. This exclusion improves the accuracy and reliability of the diagnosis.

[0048] A diagnostic decision tree is constructed based on the identified causal features. During construction, causal features are prioritized as decision nodes. For example, the first step is to determine whether the tumor has invaded blood vessels, as this feature significantly impacts the malignancy of the tumor and the patient's prognosis. Branching then occurs based on the feature's value, such as "if the tumor invades blood vessels → high probability of malignancy; if it has not invaded → high probability of benignity." At each decision node, information gain or gain based on causal effect is used as the basis for node splitting, selecting features that best differentiate between different diagnostic results to improve the accuracy and efficiency of the decision tree. In the causal intervention diagnostic decision module, the causal effect calculation formula is used...

[0049]

[0050] Calculate the causal effect TE(X→Y) of the characteristic variable X on the diagnostic outcome variable Y; using information gain

[0051]

[0052] As the primary factor in the splitting of decision tree nodes for diagnosis, the gain is based on causal effects.

[0053]

[0054] As the second factor in the splitting of decision tree nodes for diagnosis, the comprehensive gain M is used as the basis for splitting decision tree nodes.

[0055] MIG=IG(s,a)+kCIG(s,a) (9);

[0056] Where K is the weight, and K is greater than 1, S is the sample set, a is the feature, and Values(a) is the set of values ​​for feature a. v Let be the subset of samples where feature a takes the value v, and H be the information entropy.

[0057] In practical applications, the causal intervention diagnostic decision-making module has certain limitations in handling confounding factors. Confounding factors (such as patient age, comorbidities, etc.) may interfere with the causal relationship between characteristic variables and diagnostic outcome variables, leading to inaccurate estimation of causal effects and thus affecting the reliability of diagnostic decisions. To address these limitations, the following improvement scheme is proposed.

[0058] (1) Introduce front door adjustment and identify mediating variables: In the context of mediastinal tumor diagnosis, tumor morphology features are identified as mediating variables. These variables are key factors in the transmission path of patient age (confounding factor) affecting the probability of benign or malignant tumors (outcome variable). Construct a causal graph model: Draw a causal graph model containing patient age, tumor morphology features, and the probability of benign or malignant tumors to clarify the causal relationship and path between the variables. Apply the front door adjustment formula: Based on the front door adjustment formula, the influence of confounding factors (patient age) on the causal effect is adjusted by calculating the mediating variable (tumor morphology features), thereby obtaining a more accurate estimate of the causal effect.

[0059] (2) Use backdoor adjustment to identify confounding variables: In the diagnosis of mediastinal tumors, patient age, comorbidities, etc. are identified as potential confounding variables; establish a conditional probability model: construct a conditional probability model and include confounding variables, such as constructing a conditional probability relationship between the probability of benign or malignant tumors and tumor morphological characteristics, age, and comorbidities; apply the backdoor adjustment formula: use the backdoor adjustment formula to control confounding variables and calculate the causal effect of tumor morphological characteristics on the probability of benign or malignant tumors under the condition of controlling confounding factors.

[0060] (3) Combine meta-analysis to collect multi-source research data: collect multiple research data on the relationship between age, tumor morphology and benignity / malignancy to ensure the diversity and representativeness of the data; assess the degree of influence of confounding factors: use meta-analysis technology to comprehensively assess the degree of influence of confounding factors (such as age and comorbidities) in different studies; integrate data and revise causal effect estimates: adjust the causal effect estimates based on the meta-analysis results to make them more accurately reflect the real situation.

[0061] (4) Using counterfactual reasoning to construct a counterfactual model: Establishing a counterfactual reasoning model can simulate counterfactual scenarios of tumor morphology and benign / malignant probabilities under different age and other confounding factors; Analyzing the causal effects under counterfactual scenarios: By comparing the results under actual scenarios and counterfactual scenarios, we can deeply analyze the specific impact of confounding factors on causal effects, thereby more accurately estimating the causal effects of characteristic variables on diagnostic results.

[0062] (5) Optimize the adversarial self-supervised pre-training module and the meta-learning enhanced diagnosis module, and add an adversarial training strategy for the perception of confounding factors: In the adversarial self-supervised pre-training module, adjust the adversarial training process so that it can perceive and process confounding factors and enhance the robustness of feature extraction to confounding factors; Combine confounding factors to adjust the meta-learning algorithm: In the meta-learning enhanced diagnosis module, for rare tumor subtypes, modify the meta-learning algorithm so that it can effectively cope with the impact of confounding factors on diagnosis in the case of a small number of samples.

[0063] (6) Adjust the causal effect calculation formula in the causal intervention diagnosis decision module and expand the causal effect calculation formula: On the basis of the original causal effect calculation formula, add relevant terms to handle confounding factors so that the formula can fully consider the influence of confounding factors; use a dataset containing confounding factor information for model training: collect a mediastinal tumor diagnosis dataset containing confounding factor information, retrain the causal intervention diagnosis decision module, and optimize the model parameters.

[0064] (7) Improve the feedback adjustment module and add a mechanism for evaluating the effect of confounding factor treatment: In the feedback adjustment module, add evaluation indicators and methods for the effect of confounding factor treatment, so as to monitor the degree of influence of confounding factors in the causal reasoning process in real time; establish a trigger adjustment process: When the evaluation finds that the causal reasoning deviation caused by confounding factors exceeds the set threshold, the causal intervention diagnosis decision module is automatically triggered to re-examine the causal reasoning process, adjust the identification and utilization of causal features, and at the same time trigger the multi-scale attention fusion module to strengthen the extraction and fusion of rare tumor-related features and confounding factor-related features.

[0065] The causal effect calculation of the causal intervention diagnostic decision-making module was optimized from multiple perspectives. These improvements work together to effectively enhance the accuracy and reliability of the system when dealing with confounding factors, providing stronger support for the precise diagnosis of mediastinal tumors.

[0066] The preliminary diagnostic results are output by inputting the enhanced feature vectors into the constructed diagnostic decision tree. Through step-by-step judgment and reasoning, a preliminary diagnostic result is generated, such as "Thymoma suspected (benign possibility)" or "Suspected malignant lymphoma." Simultaneously with the output of the preliminary diagnostic results, the causal reasoning process is recorded and presented in the form of text explanations and visual causal graphs. The text explanations detail the basis for the diagnosis, such as "Due to tumor invasion of mediastinal vessels (causal feature A) and elevated NSE marker (causal feature B), malignant lymphoma is inferred." The visual causal graphs intuitively display key causal links, such as tumor → vascular invasion → dyspnea, helping doctors better understand the diagnostic process and its basis. Finally, the preliminary diagnostic results and causal reasoning process are passed to the meta-learning enhanced diagnostic module for further optimization of the diagnostic results.

[0067] The meta-learning-enhanced diagnostic module is configured to receive the causal information, combine it with a meta-learning mechanism, adjust model parameters using a small number of samples for rare mediastinal tumor subtypes, output the final diagnostic result, and feed the diagnostic process and result back to the multi-scale attention fusion module. In mediastinal tumor diagnosis, the meta-learning mechanism is used to learn from a large number of common tumor diagnostic tasks, extracting general diagnostic rules and experiences. From historical diagnostic data of common tumors (such as thymoma and lymphoma), it learns that immunohistochemical indicators are key to distinguishing lymphoma subtypes, and that "lobulated margins" and "vascular invasion" in imaging often indicate malignant subtypes. When faced with new diagnostic tasks, especially the diagnosis of rare mediastinal tumor subtypes, the model uses these learned experiences to quickly adjust diagnostic strategies, improving diagnostic efficiency and accuracy, and achieving rapid generalization of diagnostic rules from a small number of cases, similar to how human doctors improve their diagnostic abilities by accumulating experience. In the meta-learning-enhanced diagnostic module, based on the meta-learning mechanism, for task T... i Through formula

[0068]

[0069] Adjust the model parameters, where θ is the initial parameter. i′ These are the adjusted parameters, where α is the learning rate. For the model in task T i The loss is based on the parameter θ; the objective function is learned through meta-learning.

[0070]

[0071] Optimize the model, where M is the number of tasks.

[0072] To optimize for rare tumors, and address the scarcity of rare mediastinal tumor subtypes, the meta-learning-enhanced diagnostic module utilizes a small sample size to adjust model parameters. First, the model is pre-trained using common tumor data to develop general diagnostic capabilities. Then, when a small amount of rare subtype data is available, only parameters related to rare features are adjusted, such as those related to rare tumor immunohistochemical indicators and specific imaging features. Simultaneously, parameters for general features (such as tumor boundaries and basic morphology) are frozen to reduce sample requirements and avoid overfitting. A few-shot learning algorithm, such as ProtoNet, is employed to abstract the features of a small number of rare cases into "prototype vectors," serving as standard templates for rare tumors. New cases are compared with prototype vectors, and similarity is calculated for diagnosis, e.g., "New case features match prototype vectors by 90% → confirmed subtype."

[0073] The final diagnostic output combines the preliminary diagnostic results from the causal intervention diagnostic decision-making module with meta-learning optimization for rare subtypes. For cases initially diagnosed as "suspected malignant tumors," meta-learning analysis and judgment of rare subtypes further clarify the specific type of tumor, such as "diagnosed as rare mediastinal large B-cell lymphoma (CD30+)." The final diagnostic results are fed back to the multi-scale attention fusion module, which adjusts its extraction and fusion strategies for relevant features based on the feedback, enhancing its focus on "rare features," such as automatically focusing on CD30-related pathological areas in the next scan, thereby continuously optimizing the performance of the entire diagnostic system.

[0074] The shared feature library primarily stores the original features extracted by the multi-scale attention fusion module, the enhanced features from the adversarial self-supervised pre-training module, and the feature-based analysis results from the causal intervention diagnostic decision-making module and the meta-learning enhanced diagnostic module. The original features include multimodal data such as CT / MRI images, pathological texts, and clinical indicators. An anatomically structure-aware attention mechanism is used to focus on the mediastinal region, and the weights of different modalities are adjusted based on dynamic modality complementarity scores. The final output is a feature vector containing key information about mediastinal tumors.

[0075] These stored features and analysis results can be used to optimize feature extraction and enhancement strategies. For example, by analyzing the stored features, the multi-scale attention fusion module can understand which features are more important in diagnosis across different modalities, thus allowing for more targeted focus on key features during subsequent feature extraction, improving feature quality and relevance, and providing more accurate feature information for subsequent diagnosis.

[0076] The causal knowledge graph library stores a knowledge graph constructed by the causal intervention and diagnostic decision-making module, containing causal relationships between tumor location, tissue type, and clinical symptoms. This knowledge graph is used by the multi-scale attention fusion module to adjust the attention weights of modal data. For example, if the knowledge graph indicates a strong causal association between a certain location feature of a tumor and a specific tissue type, the multi-scale attention fusion module can appropriately increase the weights of modal data related to that location (such as features of that location in image data) during data fusion, thus giving it more attention in feature extraction and diagnosis.

[0077] The meta-learning-enhanced diagnostic module accelerates the learning and diagnosis of rare tumors. When encountering a rare tumor, the meta-learning-enhanced diagnostic module needs to refer to the causal relationships in the causal knowledge graph library to quickly find features and clues that may be related to the rare tumor. This allows it to use existing causal knowledge to guide the diagnostic process, reduce reliance on large amounts of data, and improve the efficiency and accuracy of rare tumor diagnosis.

[0078] The meta-learning experience pool stores the learning experience accumulated by the meta-learning-enhanced diagnostic module in handling rare tumor types, including experience in feature patterns, diagnostic strategies, and model parameter adjustments learned during the diagnosis of rare tumors. This experience can be used as a reference for the adversarial self-supervised pre-training module to optimize the training process. For example, the adversarial self-supervised pre-training module needs to understand, based on the experience in the meta-learning experience pool, which features are key and which training methods are effective in rare tumor diagnosis. This allows it to adjust training strategies in a targeted manner during pre-training, strengthening the learning and mining of rare tumor features, and improving the model's adaptability and performance on rare tumor diagnosis tasks.

[0079] When the above system is applied, it first collects multimodal patient data from the hospital's information system, receiving multimodal data such as chest CT images, pathology reports, and clinical indicators to ensure data integrity and accuracy. For CT / MRI images, it interfaces with the hospital's Picture Archiving and Communication System (PACS) to obtain the patient's scanned images, ensuring that the image resolution and quality meet diagnostic requirements. Pathology reports are extracted from the pathology department's electronic medical record system, detailing key information such as the tumor's histological type, cell differentiation degree, and immunohistochemical results. Clinical indicator data is collected from the hospital's electronic medical record system (EMR), covering various aspects such as laboratory tests, patient symptoms and signs, and medical history and treatment records.

[0080] In the data preprocessing stage, CT / MRI images are normalized to ensure that the grayscale or pixel values ​​are distributed within a uniform range, eliminating differences caused by different equipment or scanning parameters. Image enhancement techniques, such as histogram equalization and filtering, are used to improve image contrast and clarity, highlighting tumor features. For pathology reports, natural language processing (NLP) techniques are used for text cleaning to remove noise and irrelevant information, extract key diagnostic information, and transform it into structured data for subsequent analysis and processing. Clinical indicator data are standardized, converting the values ​​of different indicators into a unified dimension and scale to facilitate model learning and analysis. Simultaneously, missing values ​​are imputed using mean imputation, median imputation, or machine learning-based methods, depending on the characteristics and distribution of the data. Outliers are detected and processed using statistical analysis or machine learning algorithms, and corrected or deleted as appropriate.

[0081] During system operation, preprocessed multimodal data is first input into a multi-scale attention fusion module. This module utilizes an anatomical structure-aware attention mechanism to focus on the mediastinal region, assigning higher weights to mediastinal features through an attention calculation function to highlight key information. The weights of different modalities are adjusted based on dynamic modality complementarity scores, and complementarity scores between different modalities are calculated. Modality combinations with high complementarity scores are given higher weights, while modalities with low complementarity scores are given lower weights. The final output is a feature vector containing key information about mediastinal tumors.

[0082] Subsequently, the feature vectors are input into the adversarial self-supervised pre-training module. This module adds a mask to the tumor region in the image data through a self-supervised task of lesion region masking, allowing the model to predict the tumor morphology based on the unmasked information, thereby learning the spatial relationship between the tumor and surrounding tissues and its own morphological patterns. Simultaneously, adversarial perturbation generation adds small disturbances to the input data, forcing the model to learn more stable and interference-resistant features. Part of the trained feature vectors is fed back to the multi-scale attention fusion module to optimize the weights of the original feature fusion; the other part is output to the causal intervention diagnosis decision module and the meta-learning enhanced diagnosis module.

[0083] The causal intervention diagnostic decision-making module utilizes enhanced features to identify causal characteristics affecting the diagnosis based on a causal graph model. It constructs a causal graph model to determine the causal relationships between variables and identify key causal features. A diagnostic decision tree is built using these causal features as nodes. A preliminary diagnosis is performed through step-by-step judgment of the features, outputting preliminary diagnostic results. The causal reasoning process and key causal features are then passed to the meta-learning enhanced diagnostic module.

[0084] The meta-learning-enhanced diagnostic module receives causal information and, combined with a meta-learning mechanism, adjusts model parameters using a small number of samples for rare mediastinal tumor subtypes. It extracts "learning experience" from historical diagnostic data of common tumors, performs small-sample optimization for rare subtypes, adjusts model parameters, outputs the final diagnostic result, and feeds the diagnostic process and results back to the multi-scale attention fusion module.

[0085] The system outputs diagnostic results including preliminary and final diagnoses. The preliminary diagnosis, output by the causal intervention diagnostic decision-making module, is presented in text form, such as "Thymoma suspected (benign possibility)" or "Suspected malignant lymphoma," accompanied by a textual explanation of the causal reasoning process, illustrating the basis of the diagnosis. The final diagnosis, output by the meta-learning enhanced diagnosis module, is optimized for rare tumor subtypes based on the preliminary diagnosis, clarifying the specific type of tumor, such as "Confirmed as rare mediastinal large B-cell lymphoma (CD30+)." When interpreting the diagnostic results, clinicians first focus on the diagnostic conclusion, clarifying the nature and possible type of the tumor. They carefully read the causal reasoning process and diagnostic basis to understand the logic and key factors in the model's diagnosis, assessing the reliability of the results. The system's diagnostic results are then combined with the patient's clinical manifestations and other examination results for comprehensive analysis and judgment. Further examinations or consultations are conducted when necessary to develop a final treatment plan. In clinical treatment, the diagnostic results provide crucial evidence for the development of treatment plans. For patients diagnosed with benign tumors, if the tumor is small and asymptomatic, regular observation is an option; if the tumor is large or causes compressive symptoms, surgical removal may be considered. For patients diagnosed with malignant tumors, personalized treatment plans are developed based on the type and stage of the tumor and the patient's physical condition, including surgery, radiotherapy, chemotherapy, targeted therapy, or immunotherapy. During treatment, the patient's condition can be dynamically monitored and evaluated using a system to adjust the treatment plan in a timely manner and improve treatment outcomes.

[0086] The specific embodiments described above are merely illustrative or explanatory of the principles of the present invention and do not constitute a limitation thereof. Therefore, any modifications, equivalent substitutions, improvements, etc., made without departing from the spirit and scope of the present invention should be included within the protection scope of the present invention.

Claims

1. A precision diagnostic system for mediastinal tumors based on artificial intelligence, characterized in that, include: The multi-scale attention fusion module is configured to receive multimodal data from CT / MRI images, pathology text reports, and clinical indicators. It uses an anatomical structure-aware attention mechanism to focus on the mediastinal region and adjusts the weights of different modal data through dynamic modality complementarity scoring, outputting a feature vector containing key information about mediastinal tumors. The adversarial self-supervised pre-training module is configured to mine tumor morphological features based on the feature vector through a self-supervised task of lesion area masking and adversarial generation perturbation, thereby enhancing the robustness and generalization ability of the features. The trained features are fed back to the multi-scale attention fusion module and simultaneously output to the causal intervention diagnosis decision module and the meta-learning enhanced diagnosis module. The causal intervention diagnosis decision module is configured to utilize enhanced features, identify causal features affecting diagnosis based on a causal graph model, construct a diagnostic decision tree for preliminary diagnosis, output preliminary diagnostic results, and pass the causal reasoning process and key causal features to the meta-learning enhanced diagnosis module. The meta-learning-enhanced diagnostic module is configured to receive the causal information, combine it with the meta-learning mechanism, adjust the model parameters using a small number of samples for rare mediastinal tumor subtypes, output the final diagnostic result, and feed the diagnostic process and result back to the multi-scale attention fusion module.

2. The mediastinal tumor precision diagnostic system according to claim 1, characterized in that, It also includes a shared information repository, which includes: A shared feature library is used to store the original features extracted by the multi-scale attention fusion module, the features enhanced by the adversarial self-supervised pre-training module, and to receive the feature-based analysis results from the causal intervention diagnosis decision module and the meta-learning enhanced diagnosis module to optimize feature extraction and enhancement strategies. The causal knowledge graph library is used to store the knowledge graph containing the causal relationship between tumor location, tissue type, and clinical symptoms constructed by the causal intervention and diagnosis decision-making module. It is used by the multi-scale attention fusion module to adjust the attention weight of modal data, and by the meta-learning enhancement diagnosis module to accelerate the learning and diagnosis of rare tumors. The meta-learning experience pool is used to store the learning experience accumulated by the meta-learning enhanced diagnostic module in handling rare tumor types, so that the adversarial self-supervised pre-training module can refer to it to optimize the training process.

3. The mediastinal tumor precision diagnostic system according to claim 1, characterized in that, It also includes a joint training optimization module, configured to simultaneously adjust the parameters of the multi-scale attention fusion module, the adversarial self-supervised pre-training module, the causal intervention diagnosis decision module, and the meta-learning enhanced diagnosis module during the model training phase, with the accuracy and F1 score of the final diagnosis results as optimization objectives, through the backpropagation algorithm.

4. The mediastinal tumor precision diagnostic system according to claim 1, characterized in that, It also includes a feedback adjustment module, configured to establish a two-way feedback mechanism. Based on the output of each module and the final diagnostic results, when the meta-learning enhanced diagnostic module is not effective in diagnosing rare tumors, it triggers the causal intervention diagnostic decision module to re-examine the causal reasoning process, adjust the causal feature identification and utilization, and at the same time triggers the multi-scale attention fusion module to strengthen the extraction and fusion of rare tumor-related features.

5. The mediastinal tumor precision diagnostic system according to claim 1, characterized in that, In the multi-scale attention fusion module, the attention mechanism for anatomical structure perception is expressed by the formula... Calculate the attention weight A at position (i,j) of the image. ij , where f att For the attention computation function, I ij This represents the feature vector of the image at position (i,j); expressed by the formula Output the fused feature vector F, where S k f is the dynamic modal complementarity score for the k-th mode. k Let m be the feature vector extracted from the k-th mode, and m be the number of modes.

6. The mediastinal tumor precision diagnostic system according to claim 1, characterized in that, In the adversarial self-supervised pre-training module, the loss function is reconstructed. Optimize the self-supervised task of lesion region masking, where N is the number of samples and F... i The original feature vector, The reconstructed feature vectors; using the adversarial loss function L adv =E F [logD(F)]+E F Adversarial training is performed using [log(1-D(G(F)))], where G is the generator and D is the discriminator; the final training objective function is L = L rec +λL adv λ is the balance coefficient.

7. The mediastinal tumor precision diagnostic system according to claim 1, characterized in that, In the causal intervention diagnostic decision-making module, the causal effect calculation formula is used. Calculate the causal effect TE(X→Y) of the characteristic variable X on the diagnostic outcome variable Y; using information gain As the primary factor in the splitting of decision tree nodes for diagnosis, the gain is based on causal effects. As the second factor in the splitting of the diagnostic decision tree nodes, the comprehensive gain M is used as the basis for splitting the diagnostic decision tree nodes. The comprehensive gain M = IG(s, a) + kCIG(s, a), where K is the weight and K is greater than 1, S is the sample set, a is the feature, and Values(a) is the set of values ​​for feature a. v Let be the subset of samples where feature a takes the value v, and H be the information entropy.

8. The mediastinal tumor precision diagnostic system according to claim 1, characterized in that, In the meta-learning enhanced diagnostic module, based on the meta-learning mechanism, for task T... i Through formula Adjust the model parameters, where θ is the initial parameter. i′ These are the adjusted parameters, where α is the learning rate. For the model in task T i The loss is based on the parameter θ; the objective function is learned through meta-learning. Optimize the model, where M is the number of tasks.

Images