Chest x-ray disease diagnosis system based on multi-scale heterogeneous graph reasoning

Abstract

The invention discloses a chest x-ray disease diagnosis system based on multi-scale heterogeneous graph reasoning, and aims to solve the problems that an existing diagnosis system depends on a single mode, reasoning cannot be explained, and multi-scale and time sequence information is insufficient in utilization. The system comprises a multi-scale data input interface, a multi-scale heterogeneous graph construction module, a causal reasoning engine module and a diagnosis report generation module, wherein the multi-scale data input interface receives a chest X-ray image sequence and clinical text data; the multi-scale heterogeneous graph construction module extracts image lesions, anatomical nodes and text medical concepts, and generates a unified heterogeneous graph in combination with a medical knowledge graph; a causal reasoning engine conducts reasoning on the graph, a differential diagnosis set is generated, and the causal support score of each disease is calculated; the diagnosis report generation module outputs a structured report containing primary diagnosis, differential diagnosis and reasoning basis, diagnosis accuracy and robustness can be remarkably improved, and the method is suitable for auxiliary diagnosis of clinical chest diseases.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and more specifically, to a chest X-ray disease diagnosis system based on multi-scale heterogeneous graph reasoning. Background Technology

[0002] Chest X-ray examination, as one of the most commonly used imaging methods in clinical diagnosis, is widely used for screening and diagnosing various diseases such as lung diseases, mediastinal lesions, and pleural lesions. However, traditional chest X-ray diagnosis relies on the physician's experience and judgment, which has many limitations: First, the diagnostic results are easily affected by subjective factors, and different physicians may interpret the same image differently, especially for early or atypical lesions, with a high risk of misdiagnosis and missed diagnosis; Second, the diagnostic process requires combining the patient's clinical textual data (such as symptom descriptions, medical history, laboratory test results, etc.), but the efficiency of physicians manually integrating multi-source information is low, making it difficult to fully utilize all available evidence; Third, the diagnosis of complex cases requires differential diagnosis, that is, ruling out the possibility of similar diseases. This process depends on the physician's grasp of the causal relationship between diseases, and inexperienced physicians often find it difficult to complete systematic differential reasoning.

[0003] With the development of artificial intelligence technology, deep learning-based chest X-ray diagnostic systems have gradually emerged, but existing systems still have significant shortcomings. Most systems rely solely on a single image modality for analysis, neglecting crucial information contained in clinical textual data, resulting in incomplete diagnostic evidence. While some multimodal systems can integrate images and text, they lack a deep integration of medical knowledge, making it difficult to simulate the reasoning process of physicians combining medical knowledge. Furthermore, the reasoning process of existing systems mainly relies on a "black box" model, only outputting diagnostic results without providing clear reasoning basis, making it difficult to gain the trust of clinicians.

[0004] In terms of time-series analysis, multiple chest X-rays of the same patient can reflect the dynamic changes of lesions, which is crucial for judging disease progression and evaluating treatment effectiveness. However, most existing systems only process single-point-in-time data and cannot effectively utilize time-series information. Furthermore, medical knowledge updates rapidly, and the knowledge base embedded in existing systems cannot be updated in real time, potentially leading to inferences based on outdated knowledge. In addition, data quality issues are common in clinical practice (such as incomplete text records and poor image quality), and existing systems lack robust designs to handle non-ideal data conditions, making them prone to sudden performance degradation.

[0005] Therefore, building a chest X-ray disease diagnosis system that can deeply integrate multi-scale data and medical knowledge, achieve interpretable causal reasoning, and adapt to temporal changes and differences in data quality has become a key requirement for improving diagnostic accuracy and reliability, and is of great significance for assisting physicians in decision-making and improving diagnostic efficiency. Summary of the Invention

[0006] The purpose of this invention is to provide a chest X-ray disease diagnosis system based on multi-scale heterogeneous graph reasoning to address the aforementioned problems, such as insufficient utilization of multi-scale image structure modeling, neglect of multi-scale and temporal information, and limited recognition accuracy. Therefore, this invention designs a chest X-ray disease diagnosis model based on multi-scale heterogeneous graph reasoning to improve the accuracy of chest disease diagnosis.

[0007] To achieve the above objectives, an embodiment of the present invention provides the following technical solution:

[0008] A chest X-ray disease diagnostic system based on multi-scale heterogeneous graph inference, comprising:

[0009] A multi-scale data input interface for receiving chest X-ray image sequences and related clinical text data;

[0010] A multi-scale heterogeneous graph construction module is used to construct image subgraphs and text subgraphs based on the multi-scale information of the image and text data, and interact with a pre-set medical knowledge graph to generate a unified multi-scale heterogeneous graph.

[0011] A causal reasoning engine module is used to perform reasoning on the multi-scale heterogeneous graph, generate a differential diagnosis set for the primary diagnosis, and calculate the causal support score of each disease in the differential diagnosis set based on the evidence support relationship of the nodes in the graph.

[0012] A diagnostic report generation module is used to output a structured diagnostic report that includes the primary diagnosis, a list of differential diagnoses, and the reasoning behind them.

[0013] As a further improvement of the present invention, the multi-scale heterogeneous graph construction module includes:

[0014] Image subgraph construction unit is used to extract lesion nodes and anatomical nodes from images and construct their spatial relationships at different scales;

[0015] Text subgraph construction unit, used to extract medical concepts as nodes from clinical text and construct their semantic relationships;

[0016] The knowledge graph query unit is used to retrieve medical concepts and relationships related to image and text nodes from external medical knowledge graphs, forming knowledge subgraphs;

[0017] The graph fusion unit aligns and connects the nodes of the image subgraph, text subgraph, and knowledge subgraph in a shared semantic space through a cross-modal attention mechanism.

[0018] As a further improvement of the present invention, the workflow of the causal reasoning engine module includes:

[0019] Based on the activated disease nodes in the multi-scale heterogeneous graph, an auto-diagnosis set containing at least two candidate diseases is automatically generated.

[0020] For each disease in the differential diagnosis set, traverse the multi-scale heterogeneous graph and collect positive evidence nodes that support the disease and negative evidence nodes that weaken the disease.

[0021] By simulating the counterfactual reasoning process, the degree of matching between the expected evidence and the actual evidence in the graph when a certain disease is assumed to be true is evaluated, and the final score of each disease is calculated and corrected accordingly.

[0022] As a further improvement of the present invention, the causal reasoning engine module specifically includes:

[0023] A hypothesis injection unit is used to inject each disease hypothesis in the differential diagnosis set as a specific set of node feature vectors into the multi-scale heterogeneous graph.

[0024] An evidence query network takes a graph structure injected with a specific disease hypothesis as its input. The network calculates the importance weight of all evidence nodes in the graph relative to the current disease hypothesis through a multi-scale heterogeneous graph attention mechanism.

[0025] A counterfactual comparison unit receives the output of the evidence query network for different disease hypotheses, namely the evidence weight distribution corresponding to each disease; this unit quantifies the relative support strength among diseases in the differential diagnosis set by calculating the degree of difference in the evidence weight distribution under different disease hypotheses; wherein, the greater the difference between the evidence distribution and the most probable diagnosis, the higher the confidence level of the disease hypothesis being excluded.

[0026] As a further improvement of the present invention, in the structured diagnostic report output by the diagnostic report generation module, the reasoning basis includes the highlighted key areas of the image, the key symptoms extracted from the text, and the key pathophysiological associations referenced from the knowledge graph.

[0027] As a further improvement of the present invention, it further includes a time-series processing module for receiving and processing multiple sets of image-text data pairs of the same patient at different time points. The multi-scale heterogeneous graph construction module is able to construct a dynamic multi-scale heterogeneous graph sequence containing time-series change features. The causal inference engine is able to use time-series change information as strong causal evidence for inference.

[0028] As a further improvement of the present invention, the time-series processing module includes a high-precision image registration unit and a time-series feature calculation unit, which is used to quantify the morphological and density changes of the same lesion at different time points and integrate the change features as new node attributes into the graph.

[0029] As a further improvement of the present invention, a knowledge base management and update module is also included. This module loads a knowledge graph from an authoritative medical knowledge source during system initialization and provides an incremental learning interface, allowing the knowledge subgraphs embedded in the system to be updated and version managed online or offline based on the latest clinical guidelines or expert knowledge, ensuring the timeliness and accuracy of the reasoning basis.

[0030] As a further improvement of the present invention, a multi-stage collaborative training module is also included for end-to-end training of the system parameters; this training module adopts a phased strategy:

[0031] In the first stage, large-scale image-report pairs are used to pre-train image subgraph building units and text subgraph building units in a self-supervised or weakly supervised manner.

[0032] In the second stage, the parameters of the above units are fixed, and the causal reasoning engine module is specifically trained using a high-precision dataset containing expert annotations of the differential diagnosis process.

[0033] In the third stage, a smaller-scale, full-module end-to-end dataset is used to fine-tune the entire system to optimize the collaborative performance between modules.

[0034] As a further improvement of the present invention, a data quality awareness and robust processing module is also included, which is located after the multi-scale data input interface; this module is configured as follows:

[0035] When clinical text data is missing or of poor quality, the analysis weight of image subgraphs is automatically increased, and attempts are made to infer potentially related symptom nodes from the knowledge graph as a supplement.

[0036] When historical image data is missing, the system will be switched to a static inference mode based purely on single-time-point images, and the lack of time-series comparison information will be clearly noted in the diagnostic report.

[0037] This module assesses the integrity and quality of input data and dynamically adjusts the information flow and reasoning strategies within the system to ensure diagnostic reliability under non-ideal data conditions.

[0038] Compared with the prior art, the advantages of this invention are:

[0039] (1) This system breaks through the limitations of traditional single-modal diagnosis by deeply integrating chest X-ray images, clinical texts, and authoritative medical knowledge through multi-scale heterogeneous mapping. It can comprehensively utilize multi-dimensional evidence such as lesion morphology, symptom characteristics, and pathological correlations, avoiding misdiagnosis due to missing information. The causal reasoning engine accurately quantifies the evidence differences between different diseases through counterfactual comparison analysis, effectively distinguishing pneumonia from similar diseases such as pulmonary tuberculosis and pulmonary edema. In particular, the accuracy of identification is significantly improved for early atypical lesions. The time-series processing module provides quantitative analysis of the dynamic changes of lesions, further providing key evidence for judging disease progression and evaluating treatment effects, making the diagnostic results more in line with the actual clinical needs.

[0040] (2) Unlike traditional deep learning "black box" models, the system clearly presents the logical relationship between disease and evidence through multimodal, multi-scale heterogeneous graphs. The diagnostic report not only includes the primary diagnosis and differential diagnosis results, but also clearly lists the key lesions in the highlighted images, the key symptoms extracted from the text, and the medical knowledge cited, intuitively demonstrating "why this disease was diagnosed" and "why other diseases were excluded". The causal reasoning engine's calculation process for evidence weights and differences is traceable, allowing physicians to clearly understand the system's reasoning, facilitating the verification and supplementation of clinical judgments, significantly improving trust in the system's diagnostic results, and promoting its clinical application.

[0041] (3) The data quality awareness and robustness processing module of this system can flexibly cope with non-ideal data conditions commonly encountered in clinical practice, such as incomplete text records, poor image quality, and missing time-series data. By dynamically adjusting module weights and supplementing knowledge reasoning, it ensures stable diagnostic performance and avoids diagnostic failures caused by data problems. The knowledge base management and update module absorbs the latest clinical guidelines and expert experience in a timely manner through an incremental learning mechanism, ensuring the timeliness of system knowledge. The multi-stage collaborative training strategy optimizes the collaborative performance of each module, enabling the system to maintain efficient diagnostic capabilities in different clinical scenarios (such as community hospitals and tertiary hospitals), and has broad clinical applicability. Attached Figure Description

[0042] Figure 1 This is a block diagram of the overall system architecture of the present invention.

[0043] Figure 2 This is a schematic diagram illustrating the construction of the multi-scale heterogeneous graph of the present invention.

[0044] Figure 3 This is a flowchart of the causal reasoning engine of the present invention.

[0045] Figure 4 This is a schematic diagram of the counterfactual comparison unit of the present invention;

[0046] Figure 5 This is a block diagram of the timing processing module and dynamic graph construction of the present invention. Detailed Implementation

[0047] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0048] Example:

[0049] A chest X-ray disease diagnostic system based on multi-scale heterogeneous graph inference, comprising:

[0050] Multi-scale data input interface: This interface is the core of the connection between the system and clinical data. It needs to be compatible with common clinical multi-source data formats and transmission methods. The specific construction is as follows: First, for chest X-ray image sequences, it supports three mainstream formats: DICOM (clinical standard format), JPEG, and PNG. It has a built-in DICOM parser to extract image metadata (such as shooting position, tube voltage, and exposure time), and uses a format conversion tool to convert non-DICOM formats into the DICOM standard structure to ensure the integrity of image information. The image preprocessing stage employs a step-by-step optimization strategy: The first step is grayscale normalization, which transforms the original pixel values ​​(typically 0-4095) to the [0,1] range via linear mapping to eliminate grayscale differences between different devices; the second step is size standardization, which fixes the output to 512×512 pixels based on the anatomical proportions of the chest, using bilinear interpolation to avoid image distortion; the third step is noise suppression, which first smooths high-frequency device noise using a 3×3 Gaussian filter, then removes salt-and-pepper noise using a median filter, with particular emphasis on edge areas; the fourth step is contrast enhancement, which uses contrast-limited adaptive histogram equalization (CLAHE) with a contrast limit threshold set to 2.0 to highlight easily overlooked lesions such as small lung nodules and faint consolidation.

[0051] For clinical text data, the interface supports both structured data (such as tabular symptom records in electronic medical records) and unstructured data (such as handwritten examination request forms by physicians and texts of previous diagnostic reports). Structured data is directly imported into the system through field mapping (e.g., the "symptom" field corresponds to "cough" and "fever"); unstructured text is processed through a medical NLP pipeline: first, the jieba-medical word segmentation tool (adapted to Chinese medical terminology) is used for word segmentation to remove stop words without actual semantic meaning, such as "to proceed" and "to give"; then, a medical named entity recognition (NER) model based on BioBERT fine-tuning is used to extract symptoms (e.g., "cough for 3 days"), signs (e.g., "coarse breath sounds"), medical history (e.g., "hypertension history for 5 years"), and laboratory tests (e.g., "white blood cell count 12×10"). 9Entities such as " / L" are processed; finally, the entities are converted into structured key-value pairs (e.g., {"symptoms": "cough", "duration": "3 days"}) using a regular expression rule library (e.g., "[symptoms][duration][value][unit]"). Furthermore, the interface has a built-in data association engine that automatically matches multiple time-point image sequences of the same patient with corresponding text data using the patient's unique ID (e.g., medical card number) and examination timestamp (accurate to the second), generating a "patient-time-multi-scale data" association table to avoid data misalignment.

[0052] Multi-scale heterogeneous graph construction module: This module is the core of the system for realizing multi-source information fusion. It needs to construct three types of subgraphs and complete cross-modal alignment. The specific construction of each unit is as follows:

[0053] Image sub-graph construction unit: A three-step approach of "lesion detection - anatomical localization - relationship modeling" is adopted. The first step, lesion detection, is based on a pre-trained CheXNet model (121-layer DenseNet as the backbone network, fine-tuned on the ChestX-ray14 dataset), which outputs candidate bounding boxes for lesion regions of 14 common chest diseases (such as nodules, consolidation, pleural effusion). Each candidate box is filtered by non-maximum suppression (NMS, IOU threshold of 0.3) to determine the final lesion node. The node attributes include: lesion type (such as "consolidation"), location coordinates (pixels at the top left and bottom right corners of the bounding box), size (area = width × height, perimeter), density features (average gray value, gray standard deviation), and confidence (detection probability output by the model). The second step, anatomical localization, divides the lungs into five regions based on thoracic anatomy standards: right upper lobe, right middle lobe, right lower lobe, left upper lobe, and left lower lobe. The mediastinum, pleura, and diaphragm are each treated as independent anatomical structures. A pre-trained anatomical segmentation model (U-Net architecture) generates anatomical region masks. Then, spatial overlap (intersection / union of lesion regions and anatomical regions) is calculated to determine the anatomical node to which each lesion belongs, constructing a "lesion-anatomical" attribution relationship. The third step, relationship modeling, calculates spatial relationships at different scales between lesion nodes and between lesions and anatomical nodes: adjacency is determined by Euclidean distance (the minimum distance between the bounding boxes of two lesions is <10 pixels if they are adjacent), and containment is determined by spatial containment degree (the lesion region is completely within the anatomical region if it is contained). Weights are assigned to each relationship (containing relationship weight 1.0, adjacency relationship weight 0.7, no direct relationship weight 0.0), ultimately forming an image sub-image with "lesion node-anatomical node" as the core and containing spatial relationships at different scales.

[0054] Text Subgraph Construction Unit: Focusing on the process of "entity extraction - relation recognition - subgraph generation". The first step, entity extraction, is based on a fine-tuned BioBERT model (trained on the Chinese Medical NER dataset) to extract medical concept nodes from the text, covering symptoms (such as "fever"), signs (such as "heart rate 100 beats / min"), medical history (such as "history of diabetes"), laboratory tests (such as "elevated C-reactive protein"), and treatment history (such as "previously used cephalosporin antibiotics"). Each node attribute includes entity type, entity value, and extraction confidence (model output probability). The second step, relation identification, employs a fusion strategy of "rules + machine learning": explicit relations are identified based on a medical semantic rule base (e.g., "symptom + accompanying + symptom" is determined as an accompanying relation, and "disease + cause + symptom" is determined as a causal relation); for implicit relations (e.g., "cough for 3 days, chest X-ray shows solidification"), the relation type (e.g., "symptom-image association") is predicted through a trained relation classification model (based on BiLSTM + Attention architecture). Relationship attributes include relation type and association strength (calculated based on co-occurrence frequency and semantic similarity, ranging from 0 to 1). The third step, subgraph generation, constructs a text subgraph using extracted medical concept nodes as vertices and identified semantic relations as edges. For example, "fever" (symptom node) and "cough" (symptom node) are connected by an "accompanying" edge, and "cough" and "solidification" (image association entity) are connected by a "symptom-image association" edge.

[0055] The knowledge graph query unit is responsible for connecting with authoritative external medical knowledge and constructing a knowledge subgraph. First, the knowledge sources are identified as UMLS (Unified Medical Language System), SNOMEDCT (Systematic Medical Terminology Set), and a localized knowledge graph constructed from the domestic "Guidelines for the Diagnosis of Thoracic Diseases in Imaging." This knowledge graph contains four core relationships: "disease-symptom," "disease-imaging features," "disease-differential points," and "disease-pathophysiology." Second, an entity linking module is designed to map lesion nodes (e.g., "consolidation") in the image subgraph and medical concept nodes (e.g., "fever") in the text subgraph to standard terms in the knowledge graph. First, precise string matching is used (e.g., "consolidation" directly matches "pulmonary consolidation" in the knowledge graph). For nodes that fail to match (e.g., "fever"), semantic similarity is calculated using Word2Vec (matching is considered if the similarity to "fever" is >0.9), ensuring accurate node alignment. Finally, based on the matched nodes, the associated medical knowledge in the knowledge graph is queried: for example, "pulmonary consolidation" is associated with "common diseases: pneumonia, pulmonary edema", "typical symptoms: fever, cough", and "differential points: pneumonia with elevated white blood cell count, pulmonary edema with heart failure". These associated knowledge nodes (such as "pneumonia" and "pulmonary edema") and their relationships (such as "common diseases" and "differential points") are extracted to construct a knowledge subgraph. The knowledge node attributes include term ID, definition, and classification, and the relationship attributes include association type and evidence level (such as guideline recommendation as level 1 and literature report as level 2).

[0056] The graph fusion unit achieves unified fusion of three types of subgraphs through a cross-modal attention mechanism. The first step constructs a shared semantic space. Image subgraph nodes (lesions, anatomy) are converted into 512-dimensional feature vectors using a CNN feature extractor (MobileNetV3), text subgraph nodes are converted into 768-dimensional feature vectors using BioBERT, and knowledge subgraph nodes are converted into 512-dimensional feature vectors using a knowledge graph embedding model (TransE). A linear transformation then maps all three types of vectors to the 512-dimensional shared space. The second step calculates cross-modal attention weights. For each type of subgraph node, the semantic similarity (cosine similarity) between it and nodes from other modalities is calculated. For example, the similarity between the "pulmonary consolidation" node in the image subgraph and the "fever" node in the text subgraph, and the "pneumonia" node in the knowledge subgraph. The attention weights are then obtained through softmax normalization; higher weights indicate stronger inter-node connections. The third step is to construct a unified heterogeneous graph, retaining the edges within each subgraph (such as the "lesion-anatomy" edge in the image subgraph, the "symptom-accompanying" edge in the text subgraph, and the "disease-symptom" edge in the knowledge subgraph), and adding cross-modal edges based on cross-modal attention weights (threshold > 0.7), such as the "pulmonary consolidation" (image) - "pneumonia" (knowledge) edge, the "fever" (text) - "pneumonia" (knowledge) edge, and the "pulmonary consolidation" (image) - "fever" (text) edge. Finally, a unified multimodal heterogeneous graph containing three-modal nodes of "image-text-knowledge" and multiple types of edges is formed.

[0057] Causal Reasoning Engine Module: This module is the core of the system for disease differential diagnosis. It needs to cover the entire process of "differential diagnosis set generation - evidence collection - counterfactual assessment - score calculation". The specific construction of each unit and workflow is as follows:

[0058] The causal reasoning engine workflow is as follows: Step 1: Generate a differential diagnosis set. Based on the activation state of "disease nodes" in the multimodal heterogeneous graph (disease nodes associated with image and text nodes in the knowledge subgraph), calculate the initial association score (the sum of the weights of all associated edges) for each disease node. Select the top N diseases (N≥2, default N=5) as the differential diagnosis set, such as "pneumonia," "tuberculosis," "lung cancer," "pulmonary edema," and "lung abscess." Step 2: Collect positive and negative evidence. For each disease in the differential diagnosis set, traverse the multimodal heterogeneous graph: positive evidence nodes are those with a "supportive association" with the disease (e.g., "pulmonary consolidation" and "fever" support "pneumonia"), while negative evidence nodes are those with an "exclusionary association" with the disease (e.g., "no night sweats" and "smooth nodule boundaries" exclude "tuberculosis" and "lung cancer"). Record the association weight between each evidence node and the disease node. The third step is counterfactual assessment, which simulates a counterfactual scenario of "assuming a certain disease is true": Based on typical evidence of the disease in the knowledge graph (such as "typical evidence of pneumonia: pulmonary consolidation, fever, and elevated white blood cell count"), a set of expected evidence is constructed; the expected evidence is matched with the actual evidence in the heterogeneous graph, and the matching degree is calculated (the proportion of actual evidence covering expected evidence × the mean of evidence weights); if the matching degree is high (e.g., >0.8), the support score for the disease is enhanced; if the matching degree is low (e.g., <0.5), the support score is reduced, and the score is corrected.

[0059] Hypothesis Injection Unit: Responsible for integrating disease hypotheses into the heterogeneous graph. Specifically, it is constructed as follows: For each disease in the differential diagnosis set, a unique disease hypothesis feature vector is generated. The vector dimension is 512 (consistent with the shared semantic space), containing the core feature encoding of the disease (e.g., "pneumonia" is encoded as "pulmonary consolidation + fever + elevated white blood cell count," and "pulmonary tuberculosis" is encoded as "upper lobe nodules + low-grade fever + night sweats"). The feature values ​​are determined by the typical feature weights of the disease in the knowledge graph (typical feature weight 1.0, atypical feature weight 0.5). This feature vector is injected into the corresponding disease node in the heterogeneous graph through the node embedding layer of a graph neural network (GCN), updating the feature representation of the disease node. Through an attention propagation mechanism, the influence of the disease hypothesis is spread to related evidence nodes (e.g., after injecting the "pneumonia" hypothesis, the feature weights of the "pulmonary consolidation" and "fever" nodes increase by 20%), resulting in a "disease hypothesis-driven" feature distribution in the heterogeneous graph.

[0060] Evidence Query Network: Responsible for quantifying the support strength of evidence nodes for disease hypotheses. It is constructed using a Multi-HeadGAT architecture: the network contains three attention heads, each focusing on a specific type of evidence—Head 1 focuses on image evidence (lesions, anatomical nodes), Head 2 focuses on textual evidence (symptoms, medical history nodes), and Head 3 focuses on knowledge evidence (related nodes in the knowledge graph). The computation process for each attention head is as follows: for the heterogeneous graph after the disease hypothesis is injected, the attention coefficient between each evidence node i and the disease node j is calculated. Among them W h Let v be a linear transformation matrix. i v j These are the feature vectors of the evidence node and the disease node, respectively, where σ is the sigmoid activation function; α is normalized using softmax. ij The importance weight of the evidence node to the disease hypothesis is obtained; the weights of the three attention heads are fused through average pooling to obtain the final evidence weight (positive evidence weight is positive, and negative evidence weight is negative). For example, the weight of the "pulmonary consolidation" node to the "pneumonia" hypothesis is 0.9 (strong positive support), and the weight of the "no night sweats" node to the "tuberculosis" hypothesis is -0.7 (strong negative exclusion).

[0061] The counterfactual comparison unit is responsible for quantifying the differences between different disease hypotheses and achieving differential diagnosis ranking. Its specific construction is as follows: First, it collects the evidence weight distribution (a vector composed of the weights of all evidence nodes) corresponding to each disease hypothesis; second, it calculates the difference in evidence distributions between any two disease hypotheses, using a fusion index of "cosine distance + KL divergence": difference D = 0.5 × (1 - cosθ) + 0.5 × KL(P||Q), where cosθ is the cosine similarity between the two vectors, and KL(P||Q) is the KL divergence between the two distributions. A larger D indicates a more significant difference in the evidence distributions between the two diseases; finally, it adjusts the causal support scores of the diseases based on the difference: the disease with the highest initial score is taken as the "baseline disease," and the scores of other diseases = initial score × (1 - D). That is, the greater the difference between the disease and the baseline disease, the more significant the score reduction. Finally, it forms a differential diagnosis list ranked by score (the highest score is the primary diagnostic candidate). For example, if the initial score for "pneumonia" is 10 and the initial score for "tuberculosis" is 8, and the difference between the two is D = 0.6, then the corrected score for "tuberculosis" is 8 × (1 - 0.6) = 3.2, which is significantly lower than that for "pneumonia", so the primary diagnosis is "pneumonia".

[0062] Diagnostic Report Generation Module: This module needs to output structured and interpretable diagnostic reports. The specific structure is as follows: First, design a report template framework, including fixed columns: Patient basic information (name, gender, age, medical card number), examination information (examination type, imaging time, equipment model), imaging findings (key findings in the image), clinical information (key text extraction), primary diagnosis (the disease with the highest score), differential diagnosis list (sorted by the corrected score, including supporting scores and reasons for exclusion for each disease), reasoning basis (multimodal evidence association), and treatment recommendations (based on knowledge graph).

[0063] Secondly, the template content is dynamically filled: For the imaging presentation section, the 3-5 lesion nodes with the highest weight are extracted from the image sub-images (e.g., "large area of ​​consolidation in the lower lobe of the right lung with blurred boundaries"). Using image annotation tools, red bounding boxes are drawn to highlight the lesion areas on the original image, and a lesion location diagram is generated (labeling lung lobe areas). For the clinical information section, key symptoms and medical history are extracted from the text sub-images (e.g., "fever for 3 days, accompanied by cough and yellow sputum, no history of tuberculosis contact"). For the reasoning basis section, it is presented in layers according to "image evidence - text evidence - knowledge association": image evidence consists of highlighted lesion areas and corresponding imaging features (e.g., "consolidation in the lower lobe of the right lung"). The textual evidence consists of key symptoms extracted (e.g., "fever and yellow sputum support infectious diseases"), and the knowledge associations are the pathophysiological associations in the knowledge graph (e.g., "pneumonia is often caused by bacterial infection, manifesting as pulmonary consolidation and infection symptoms"). In the differential diagnosis list, each disease needs to indicate "supporting evidence" (e.g., "pulmonary tuberculosis: small nodules were seen in the upper lobe of the right lung") and "exclusion evidence" (e.g., "no night sweats, low-grade fever, nodules have disappeared") and the corrected score. In the treatment recommendation section, the standard recommendations for the primary diagnosis are retrieved from the knowledge graph (e.g., "for pneumonia, penicillin antibiotics are recommended as the first choice, and a chest X-ray should be repeated after 7 days of treatment").

[0064] Finally, the report format is editable, generating DOCX or PDF files. Physicians can manually modify the content (such as adding clinical judgments), and the system automatically records the modification history (modifier, modification time, and modified content) to ensure report traceability.

[0065] Time series processing module: This module is used to construct a dynamic heterogeneous graph using data from multiple time points of the same patient. The specific construction is as follows:

[0066] High-precision image registration unit: For chest X-ray images of the same patient at different time points (e.g., initial visit, follow-up one week after treatment), registration is achieved using a "feature point extraction-matching-transformation-verification" process. The first step, feature point extraction, uses the SIFT algorithm (Scale Invariant Feature Transform) to extract anatomical landmarks (e.g., hilar angle, costophrenic angle, thoracic vertebral body edges) from both images. Each feature point includes location, scale, and orientation descriptors. The second step, feature point matching, uses the FLANN matcher (Fast Nearest Neighbor Search library) to calculate the descriptor similarity of feature points between the two images, retaining corresponding point pairs with a matching score greater than 0.8. Then, the RANSAC algorithm (Random Sample Consensus) is used to remove abnormal matches (e.g., incorrect matches caused by artifacts), ensuring a matching accuracy greater than 95%. The third step, transformation matrix calculation, uses the least squares method to solve the affine transformation matrix (including translation, rotation, and scaling parameters) based on the matched feature point pairs, mapping the follow-up image to the coordinate system of the initial visit image. The fourth step is registration verification, which involves calculating the structural similarity index (SSIM) of the two registered images. The requirement is that SSIM > 0.85. If this is not met, the transformation matrix is ​​re-optimized to ensure registration accuracy.

[0067] The temporal feature calculation unit quantifies the dynamic changes of lesions and integrates them into the heterogeneous graph as new nodes. Specific calculation indicators include: morphological changes (lesion area change rate = (reviewed area - initial diagnosis area) / initial diagnosis area × 100%, boundary smoothness change = reviewed boundary pixel standard deviation - initial diagnosis boundary pixel standard deviation), density changes (average gray value change = reviewed average gray value - initial diagnosis average gray value, density uniformity change = reviewed gray value standard deviation - initial diagnosis gray value standard deviation), quantity changes (number of new lesions, number of disappeared lesions), and location changes (distance of lesion center coordinate offset). These temporal features are encapsulated into "temporal change nodes," with node attributes including change type (e.g., "area reduction" "density reduction"), change magnitude (e.g., "area reduction 30%"), and change time interval (e.g., "7 days"). "Temporal association" edges are established with corresponding lesion nodes and disease nodes (e.g., "area reduction" connects to the "pneumonia" node, with the edge attribute being "effective treatment support"), ultimately forming a dynamic multimodal heterogeneous graph sequence containing temporal information.

[0068] Knowledge Base Management and Update Module: This module ensures the timeliness and accuracy of system knowledge. Its specific construction is as follows: First, the knowledge import process is initialized using an ETL tool (Extract-Transform-Load) to import data in batches from multiple authoritative knowledge sources: standard terminology and relational data (XML format) are downloaded from UMLS and SNOMEDCT; Chinese medical knowledge is extracted from the latest literature in the "Guidelines for Imaging Diagnosis of Thoracic Diseases" and the "Chinese Journal of Tuberculosis and Respiratory Diseases" (text is extracted using a PDF parsing tool and then processed into structured knowledge using medical NLP); multi-source knowledge is uniformly converted into a "node-relationship-attribute" triple format (e.g., <pneumonia, common symptoms, fever>), imported into the system's built-in graph database (Neo4j), and an initial knowledge subgraph is constructed.

[0069] Secondly, the incremental learning interface design supports both online and offline update methods: When updating online, physicians manually enter new knowledge through the system's knowledge editing interface (which supports the addition / modification of visual nodes and edges). After submission, an expert review process is triggered (requiring approval from at least two associate chief physicians). Approved knowledge is automatically converted into triples and integrated into the knowledge subgraph. When updating offline, the system automatically retrieves updated content monthly from preset knowledge sources (such as the UpToDate clinical guideline update column and the latest literature from PubMedCentral), automatically parses it into structured knowledge using medical NLP, and updates it to the knowledge subgraph in one go after batch review by experts in the background.

[0070] Finally, the version management mechanism assigns a unique version number (e.g., V2024.05) to each version of the knowledge subgraph, records the version update time, update content (number of newly added / modified nodes and relationships), and the person responsible for the update; it supports version rollback functionality, allowing users to roll back to the previous stable version with one click when an update is found to have introduced incorrect knowledge (e.g., incorrect terminology); at the same time, the system will periodically (quarterly) perform consistency checks on the knowledge subgraph (checking for contradictory relationships, such as "pneumonia" being associated with both "no fever" and "must have fever") to ensure the correctness of the knowledge logic.

[0071] Multi-stage collaborative training module: This module is used for end-to-end optimization of system parameters, and its specific construction is as follows: It adopts a three-stage strategy of "pre-training - specialized training - end-to-end fine-tuning", and the details of each stage are as follows:

[0072] Phase 1: Pre-training of Image and Text Sub-image Units: The goal is to optimize the lesion detection and anatomical segmentation capabilities of image sub-images, and the entity extraction and relation recognition capabilities of text sub-images. Image sub-image unit training uses the ChestX-ray14 dataset (containing 112,000 chest X-ray images and corresponding report texts) and is trained in a self-supervised manner: FocalLoss (to address class imbalance) is used for lesion detection, with optimization objectives being the IOU of the lesion bounding box and classification accuracy; DiceLoss is used for anatomical segmentation, with optimization objectives being the Dice coefficient for anatomical region segmentation. Text sub-image unit training uses Chinese medical NER datasets (such as cMedQA2) and medical relation classification datasets (such as IMCS-V2) and is trained in a weakly supervised manner: Cross-entropy loss is used for entity extraction, with optimization objectives being the F1 score for entity recognition; contrastive loss is used for relation recognition, with optimization objectives being relation classification accuracy. Training parameters: Batch size is set to 32, learning rate is 1e-4, training is performed for a total of 50 epochs, the Adam optimizer is used by default, validation is performed every 5 epochs, the model parameters with the best performance on the validation set are saved, and the parameters of the image and text subgraph units are fixed after training is completed.

[0073] Phase Two: Specific Training of the Causal Inference Engine: The goal is to optimize the differential diagnosis capability of the inference engine, using a precisely labeled dataset (annotated by 100 respiratory physicians, containing 5000 chest X-ray images, corresponding text, and detailed differential diagnosis processes, with each image labeled with a "disease-evidence-support strength" triple). Image and text subgraph unit parameters are fixed, and only the causal inference engine is trained: the hypothesis injection unit training objective is the accuracy of the disease hypothesis feature vector representation (through contrastive learning, different descriptions of the same disease are mapped to similar vectors); the evidence query network training objective is the accuracy of evidence weight prediction (the loss function is the MSE of the predicted weights and the expert-annotated weights); the counterfactual comparison unit training objective is the reasonableness of the difference calculation (the loss function is the Spearman correlation coefficient between the corrected score and the expert ranking). Training parameters: batch size is set to 16, learning rate is 5e-5, total training is 30 epochs, the default AdamW optimizer is used, validation is performed every 3 epochs, and the model parameters with the highest inference accuracy are saved.

[0074] Phase 3: End-to-End Fine-Tuning of the Entire System: The goal is to optimize the collaborative performance between modules, using a small-scale fully labeled dataset (containing 1000 cases, each with a complete "multi-scale data - multi-scale heterogeneous graph - inference result - diagnostic report" chain). All module parameters are unfrozen, with "diagnostic accuracy + report reasonableness" as the joint optimization objective: diagnostic accuracy is assessed through the consistency evaluation between the primary diagnosis and expert diagnosis (ACC metric); report reasonableness is evaluated through natural language generation metrics (such as BLEU, ROUGE) and expert scores (1-5 points). Training parameters: Batch size is set to 8, learning rate is 1e-5, and training is conducted for a total of 10 epochs. Gradient accumulation (updating parameters every 4 steps) is used to avoid training instability caused by excessively small batch sizes. Validation is performed every 2 epochs, and the system model with the best joint performance is saved.

[0075] Data Quality Awareness and Robustness Processing Module: This module, located after the multi-scale data input interface, is used to address non-ideal data conditions. Its specific construction is as follows: First, a data quality assessment index system: Text data quality is assessed from three dimensions: "Completeness" (the rate of missing key information, such as the proportion of missing symptoms and medical history), "Consistency" (no contradictory descriptions, such as "no fever" and "body temperature 39℃" not appearing simultaneously), and "Accuracy" (no typos or ambiguous expressions, such as correcting "lung nodules" to "lung nodules"). Each dimension has a maximum score of 100 points, and a comprehensive score less than 60 points indicates poor quality. Image data quality is assessed from three dimensions: "Clarity" (the degree of blurriness, assessed by edge gradient values), "Artifacts" (motion artifacts, device artifacts, assessed by an artifact detection model), and "Completeness" (no missing anatomical structures, such as missing lower lobes of the lungs). A comprehensive score less than 60 points indicates poor quality. Historical image data quality is assessed based on "Existence" (whether there is at least one previous image) and "Temporal Continuity" (the maximum time interval between previous and current images is 3 months). Missing images are considered insufficient time-series data.

[0076] Secondly, a dynamic adjustment strategy is implemented: when the text data quality is poor (e.g., the missing rate of key symptoms is greater than 50%), the system automatically increases the weight of the image subgraph in inference from the base value of 0.5 to 0.7-0.8 (weight adjustment formula = base weight + (1 - text quality score / 100) × 0.3), and simultaneously triggers the knowledge graph supplementation mechanism—based on the lesion node of the image subgraph (e.g., "pulmonary consolidation"), it queries the knowledge graph for common accompanying symptoms of the lesion (e.g., "fever, cough"), generates "inferred symptom nodes" (labeled with the "knowledge inference" attribute), and supplements them to the text subgraph to reduce the impact of missing text; when the image ... When data quality is poor (e.g., fuzziness leading to a lesion detection confidence level of less than 0.5), the system automatically increases the weight of the text subgraph (from 0.5 to 0.7) and queries non-imaging evidence for the disease through the knowledge graph (e.g., "laboratory evidence of pneumonia: elevated white blood cell count"), prioritizing reasoning based on text and knowledge evidence. When historical imaging data is missing, the system switches to static reasoning mode, disables the time-series processing module, and only performs multimodal heterogeneous graph reasoning based on the current time point. The system also clearly notes in the "Limitations" section of the diagnostic report that "lack of previous image comparison makes it impossible to assess the dynamic changes of lesions, and the diagnosis needs to be combined with clinical follow-up."

[0077] Finally, the robustness verification mechanism: Before system deployment, low-quality data is generated through data augmentation (such as randomly deleting 10%-50% of key information from text, adding blur / artifacts to images, and randomly missing time-series data) to test the system's diagnostic accuracy under low-quality data (the accuracy drop should be less than 10% with higher-quality data); after deployment, the correlation between data quality score and diagnostic accuracy is monitored in real time. When a certain type of low-quality data causes an accuracy drop of more than 15%, an alarm is automatically triggered, prompting engineers to optimize the robustness of the corresponding module.

[0078] Working principle:

[0079] During the data input and preprocessing stage, clinical users (such as radiologists) upload patient chest X-ray image sequences (e.g., current examination images combined with a follow-up image from 3 months prior) and corresponding clinical text data (e.g., electronic medical records, examination request forms) through a multi-scale data input interface. The interface first performs grayscale normalization, size standardization, noise reduction, and contrast enhancement preprocessing on the images to ensure consistent image quality. It then performs word segmentation, entity extraction, and structured transformation on the text to generate key-value pair formatted text information. Simultaneously, the data quality awareness module assesses the quality of the input data: if the "symptoms" field in the text only records "discomfort" (completeness score below 40), it is judged as poor text quality. The image subgraph weight is automatically increased to 0.75, and the knowledge graph is used to query common symptoms ("fever, cough") corresponding to "pulmonary consolidation" lesions detected in the current image, generating "presumed symptom nodes" to supplement the text subgraph. If no previous images are uploaded, it is judged as missing time-series data, the time-series processing module is closed, and a report note stating "lacking time-series comparison" is recorded.

[0080] In the multi-scale heterogeneous map construction stage, the multi-scale heterogeneous map construction module initiates three sub-map construction and fusion processes: The image sub-map construction unit detects lesion nodes such as "right lower lobe consolidation" and "no pleural effusion" using the CheXNet model, and combines this with the anatomical segmentation model to determine that "right lower lobe consolidation is located in the right lower lobe," constructing an image sub-map containing spatial relationships of different scales of "lesion-anatomy"; The text sub-map construction unit extracts nodes such as "fever for 3 days," "cough," and "no history of tuberculosis contact" from structured text and supplementary "presumed symptoms," constructing an image sub-map containing "symptoms." - A text subgraph with "accompanying" relationships; the knowledge graph query unit maps "right lower lobe consolidation shadow" and "fever" to the knowledge graph, extracts associated disease nodes such as "pneumonia," "tuberculosis," and "pulmonary edema," and relationships such as "pneumonia-pulmonary consolidation" and "pneumonia-fever," and constructs a knowledge subgraph; the graph fusion unit calculates the semantic similarity (both greater than 0.8) between "right lower lobe consolidation shadow" and "pneumonia," and between "fever" and "pneumonia" through a cross-modal attention mechanism, adds cross-modal edges, and finally forms a unified heterogeneous graph containing nodes of the three modalities of "image-text-knowledge." If time-series data (such as previous images) exists, the image registration unit of the time-series processing module aligns the current image with the previous image, and the time-series feature calculation unit quantifies "the area of ​​right lower lobe consolidation shadow has decreased by 40% compared to before," generates a "time-series change node," adds it to the heterogeneous graph, and establishes a "treatment effective support" edge with the "pneumonia" node.

[0081] In the causal reasoning stage, the causal reasoning engine initiates the differential diagnosis process: First, based on the initial association scores of disease nodes in the heterogeneous graph, "pneumonia" (score 12), "tuberculosis" (score 9), and "pulmonary edema" (score 7) are selected as the differential diagnosis set. The hypothesis injection unit generates disease hypothesis feature vectors for "pneumonia," "tuberculosis," and "pulmonary edema," which are then injected into the heterogeneous graph to update the features of the corresponding disease nodes and associated evidence nodes. The evidence query network calculates the weight of each evidence node for the disease hypothesis through multi-head attention: "right lower lobe consolidation," "fever," and "reduction in consolidation area" have weights of 0.9, 0.8, and 0.7 for "pneumonia" (all positive). The weights for “no night sweats” and “no history of tuberculosis exposure” to “pulmonary tuberculosis” are -0.8 and -0.6 respectively (both negative), and the weights for “no lower extremity edema” and “no history of heart disease” to “pulmonary edema” are -0.9 and -0.7 respectively (both negative). The counterfactual comparison unit calculates the degree of difference: the degree of difference between “pneumonia” and “pulmonary tuberculosis” is D=0.7, the degree of difference between “pneumonia” and “pulmonary edema” is D=0.8. After correction, the score for “pulmonary tuberculosis” is 9×(1-0.7)=2.7, the score for “pulmonary edema” is 7×(1-0.8)=1.4. Finally, “pneumonia” has the highest score (12) and is determined as the primary diagnosis. “Pulmonary tuberculosis” and “pulmonary edema” are the differential diagnoses (sorted by score).

[0082] In the diagnostic report generation and knowledge update phase, the diagnostic report generation module fills in the template based on the reasoning results: the imaging findings section highlights "consolidation in the lower lobe of the right lung" and notes "area reduced by 40% compared to before"; the clinical information section records "fever for 3 days, cough, no history of tuberculosis contact or heart disease"; the primary diagnosis is "community-acquired pneumonia", and the differential diagnoses are "pulmonary tuberculosis (score 2.7, exclusion reason: no night sweats, history of tuberculosis contact)" and "pulmonary edema (score 1.4, exclusion reason: no history of heart disease, lower extremity edema)"; the reasoning basis section links "consolidation in the lower lobe of the right lung (image) + fever (text) + typical manifestations of pneumonia (knowledge)"; the treatment recommendation is "continue penicillin antibiotic treatment, repeat chest X-ray in 5 days" (from the knowledge graph). After the report is generated, the physician can add a clinical note such as "the patient has been taking amoxicillin for 3 days, and the symptoms have improved", and the system records the modification traces and outputs the final report. Meanwhile, the knowledge base management and update module monitors for new clinical experience (such as physicians adding "amoxicillin is effective for this patient"). If it meets the knowledge update criteria, it triggers an online update process. After expert review, the relationship between "community-acquired pneumonia - sensitive drugs - amoxicillin" is integrated into the knowledge subgraph, completing one knowledge iteration.

[0083] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0084] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style of the specification is merely for clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A chest X-ray disease diagnostic system based on multi-scale heterogeneous graph reasoning, characterized in that, include: A multi-scale data input interface for receiving chest X-ray image sequences and related clinical text data; A multi-scale heterogeneous graph construction module is used to construct image subgraphs and text subgraphs based on the multi-scale information of the image and text data, and interact with a pre-set medical knowledge graph to generate a unified multi-scale heterogeneous graph. A causal reasoning engine module is used to perform reasoning on the multi-scale heterogeneous graph, generate a differential diagnosis set for the primary diagnosis, and calculate the causal support score of each disease in the differential diagnosis set based on the evidence support relationship of the nodes in the graph. A diagnostic report generation module is used to output a structured diagnostic report that includes the primary diagnosis, a list of differential diagnoses, and the reasoning behind them.

2. The system according to claim 1, characterized in that, The multi-scale heterogeneous graph construction module includes: Image subgraph construction unit is used to extract lesion nodes and anatomical nodes from images and construct their spatial relationships at different scales; Text subgraph construction unit, used to extract medical concepts as nodes from clinical text and construct their semantic relationships; The knowledge graph query unit is used to retrieve medical concepts and relationships related to image and text nodes from external medical knowledge graphs, forming knowledge subgraphs; The graph fusion unit aligns and connects the nodes of the image subgraph, text subgraph, and knowledge subgraph in a shared semantic space through a cross-modal attention mechanism.

3. The system according to claim 2, characterized in that, The workflow of the causal reasoning engine module includes: Based on the activated disease nodes in the multi-scale heterogeneous graph, an auto-diagnosis set containing at least two candidate diseases is automatically generated. For each disease in the differential diagnosis set, traverse the multi-scale heterogeneous graph and collect positive evidence nodes that support the disease and negative evidence nodes that weaken the disease. By simulating the counterfactual reasoning process, the degree of matching between the expected evidence and the actual evidence in the graph when a certain disease is assumed to be true is evaluated, and the final score of each disease is calculated and corrected accordingly.

4. The system according to claim 3, characterized in that, The causal reasoning engine module specifically includes: A hypothesis injection unit is used to inject each disease hypothesis in the differential diagnosis set as a specific set of node feature vectors into the multi-scale heterogeneous graph. An evidence query network takes a graph structure injected with a specific disease hypothesis as its input. The network calculates the importance weight of all evidence nodes in the graph relative to the current disease hypothesis through a multi-scale heterogeneous graph attention mechanism. A counterfactual comparison unit receives the output of the evidence query network for different disease hypotheses, namely the evidence weight distribution corresponding to each disease; this unit quantifies the relative support strength among diseases in the differential diagnosis set by calculating the degree of difference in the evidence weight distribution under different disease hypotheses; wherein, the greater the difference between the evidence distribution and the most probable diagnosis, the higher the confidence level of the disease hypothesis being excluded.

5. The system according to claim 1, characterized in that, In the structured diagnostic report output by the diagnostic report generation module, the reasoning basis includes highlighted key areas of the image, key symptoms extracted from the text, and key pathophysiological associations referenced from the knowledge graph.

6. The system according to claim 1, characterized in that, It further includes a time-series processing module for receiving and processing multiple sets of image-text data pairs of the same patient at different time points. The multi-scale heterogeneous graph construction module can construct a dynamic multi-scale heterogeneous graph sequence containing time-series change features. The causal inference engine can use time-series change information as strong causal evidence for inference.

7. The system according to claim 6, characterized in that, The temporal processing module includes a high-precision image registration unit and a temporal feature calculation unit, which are used to quantify the morphological and density changes of the same lesion at different time points and integrate the change features as new node attributes into the image.

8. The system according to claim 2, characterized in that, It also includes a knowledge base management and update module, which loads the knowledge graph from authoritative medical knowledge sources during system initialization and provides an incremental learning interface, allowing the embedded knowledge subgraphs to be updated and versioned online or offline based on the latest clinical guidelines or expert knowledge, ensuring the timeliness and accuracy of the reasoning basis.

9. The system according to claim 1, characterized in that, It also includes a multi-stage collaborative training module for end-to-end training of the system's parameters; this training module employs a phased strategy: In the first stage, large-scale image-report pairs are used to pre-train image subgraph building units and text subgraph building units in a self-supervised or weakly supervised manner. In the second stage, the parameters of the above units are fixed, and the causal reasoning engine module is specifically trained using a high-precision dataset containing expert annotations of the differential diagnosis process. In the third stage, a smaller-scale, full-module end-to-end dataset is used to fine-tune the entire system to optimize the collaborative performance between modules.

10. The system according to claim 1, characterized in that, It also includes a data quality awareness and robustness processing module, located after the multi-scale data input interface; this module is configured as follows: When clinical text data is missing or of poor quality, the analysis weight of image subgraphs is automatically increased, and attempts are made to infer potentially related symptom nodes from the knowledge graph as a supplement. When historical image data is missing, the system will be switched to a static inference mode based only on single-time point images, and the lack of time-series comparison information will be clearly noted in the diagnostic report. This module assesses the integrity and quality of input data and dynamically adjusts the information flow and reasoning strategies within the system to ensure diagnostic reliability under non-ideal data conditions.

Images