Diagnosis method, device and equipment for multi-modal medical data fusion based on AI

Abstract

The invention provides a diagnosis method, device and equipment for multi-modal medical data fusion based on AI, and the method comprises the steps: constructing multi-source medical data, and constructing a dynamic association network based on the multi-source medical data; performing hierarchical feature interaction processing on the dynamic association network, updating node feature representation through bidirectional information conduction between nodes, and generating a cross-modal enhanced feature set; inputting the cross-modal enhanced feature set into a pre-trained diagnostic reasoning model to generate fusion diagnostic features; determining candidate disease types, corresponding support feature items and non-support feature items according to the overall matching degree and the sub-item matching degree of the fusion diagnosis features and the standardized disease typical feature spectrum; establishing a feature traceability chain of a diagnosis basis according to source features traced by the support feature items in the cross-modal enhanced feature set, and constructing a feature association map based on the feature traceability chain and a node association relationship of the dynamic association network; and outputting a diagnosis result according to the candidate disease type, the feature tracing chain and the feature association map.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a diagnostic method, apparatus and device based on AI-based multimodal medical data fusion. Background Technology

[0002] In the field of medical diagnosis, accurate and comprehensive analysis of a patient's condition is crucial for developing effective treatment plans. Traditional medical diagnosis mainly relies on single-modality medical data, such as observing changes in a patient's anatomical structure solely through imaging data, monitoring a patient's physiological function based solely on physiological signal data, or simply referring to textual medical records to understand a patient's medical history. However, the human body is a complex system, and symptoms often manifest in multiple ways; single-modality data can hardly fully present the patient's true condition.

[0003] While some methods for multimodal medical data fusion exist in existing technologies, most simply splice or overlay data from different modalities without fully considering the clinical correlation rules between the data items. The lack of deep interaction and effective integration of features from different modalities during the data fusion process prevents the fused data from fully uncovering the potential connections and complementary information between the various modalities. Furthermore, existing diagnostic reasoning models often fail to analyze the fused data based on the clinical attributes of different modalities, making it difficult to generate diagnostic features with clear clinical decision-making guidance. This affects the accuracy and reliability of diagnostic results, failing to meet the high precision and comprehensiveness requirements of modern medical diagnosis. Summary of the Invention

[0004] The purpose of this invention is to provide a diagnostic method, device, and equipment based on AI-driven multimodal medical data fusion, aiming to improve the accuracy and comprehensiveness of diagnostic results.

[0005] To achieve the above objectives, a first aspect of this disclosure provides a diagnostic method based on AI-driven multimodal medical data fusion, comprising: constructing multi-source medical data based on anatomical structure images and functional metabolic images transmitted by an image acquisition device under different imaging modalities, continuous waveform signals reflecting the functions of different physiological systems transmitted by a physiological monitoring device, and medical record reports transmitted by an electronic medical record system; constructing a dynamic association network based on clinical association rules for each data item in the multi-source medical data, wherein nodes in the dynamic association network correspond to data items in the multi-source medical data, and the attributes of the edges include association confidence and clinical dependence direction; performing hierarchical feature interaction processing on the dynamic association network, updating node feature representations through bidirectional information transmission between nodes, and generating a cross-modal enhanced feature set; and configuring the cross-modal enhanced features... A pre-trained diagnostic reasoning model is input into the dataset. A fusion diagnostic feature is generated using the model's dual-path feature parsing mechanism. Based on the overall matching degree between the fusion diagnostic feature and the typical feature spectra of each standardized disease, candidate disease types are determined. Furthermore, based on the sub-item matching degree between the fusion diagnostic feature and the typical feature spectra of each standardized disease, supporting and unsupported feature items for each candidate disease type are determined. Based on the source features traced to by the supporting feature items in the cross-modal enhanced feature set, a feature tracing chain for diagnostic criteria is established. Based on the feature tracing chain and the node association relationships of the dynamic association network, a feature association graph is constructed. Finally, based on the candidate disease type, the feature tracing chain, and the feature association graph, a diagnostic result including the disease type, differential diagnostic criteria, and the feature association graph is generated and output.

[0006] A second aspect of this disclosure provides a diagnostic device for multimodal medical data fusion based on AI, comprising: a first construction module configured to construct multi-source medical data based on anatomical structure images and functional metabolic images transmitted by an image acquisition device under different imaging modalities, continuous waveform signals reflecting the functions of different physiological systems transmitted by a physiological monitoring device, and medical record reports transmitted by an electronic medical record system; a second construction module configured to construct a dynamic association network based on clinical association rules of each data item in the multi-source medical data, wherein nodes in the dynamic association network correspond to data items in the multi-source medical data, and the attributes of the edges include association confidence and clinical dependence direction; a first generation module configured to perform hierarchical feature interaction processing on the dynamic association network, update node feature representations through bidirectional information transmission between nodes, and generate a cross-modal enhanced feature set; and a second generation module configured to process the cross-modal enhanced features. A strong feature set is input into a pre-trained diagnostic reasoning model, and fused diagnostic features are generated through the dual-path feature parsing mechanism of the diagnostic reasoning model. A determination module is configured to determine candidate disease types based on the overall matching degree between the fused diagnostic features and the typical feature spectra of each standardized disease, and to determine the supporting and unsupported feature items of each candidate disease type based on the sub-item matching degree between the fused diagnostic features and the typical feature spectra of each standardized disease. A third construction module is configured to establish a feature tracing chain for diagnostic basis based on the source features traced by the supporting feature items in the cross-modal enhanced feature set, and to construct a feature association graph based on the feature tracing chain and the node association relationship of the dynamic association network. A third generation module is configured to generate and output a diagnostic result including disease type, differential diagnostic basis, and feature association graph based on the candidate disease type, the feature tracing chain, and the feature association graph.

[0007] A third aspect of this disclosure provides an electronic device, comprising: a processor and a machine-readable storage medium connected to the processor; the machine-readable storage medium being used to store programs, instructions, or code; and the processor being used to execute the programs, instructions, or code in the machine-readable storage medium to implement the AI-based multimodal medical data fusion diagnostic method described in any one of the first aspects.

[0008] This invention provides a diagnostic method, apparatus, and device based on AI-driven multimodal medical data fusion. Compared with existing technologies, it offers the following advantages: By comprehensively acquiring multi-source medical data, including image data, physiological signal data, and text medical record data, and through modal identification processing and associative index information binding, the integrity and traceability of the data are ensured. Furthermore, when constructing a dynamic association network, the clinical association rules between various data items are fully considered, enabling the network to accurately reflect the clinical relationships between data. Hierarchical feature interaction processing, through bidirectional information transmission between nodes, achieves deep fusion and enhancement of cross-modal features, fully exploring the potential connections and complementary information between different modalities. The pre-trained diagnostic inference model employs a dual-path feature parsing mechanism, enabling targeted processing based on the clinical attributes of different modalities to generate fused diagnostic features with clinical decision-making orientation, significantly improving the accuracy and reliability of diagnosis. The final output diagnostic results include disease type, differential diagnostic criteria, and feature association maps, enhancing the accuracy and comprehensiveness of diagnostic results and providing doctors with comprehensive and detailed diagnostic information, helping them make more scientific and rational treatment decisions.

[0009] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0010] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating a diagnostic method based on AI-driven multimodal medical data fusion, as shown in the embodiments of the specification.

[0011] Figure 2 This is a block diagram of a diagnostic device based on AI-driven multimodal medical data fusion, as shown in the embodiments of the specification.

[0012] Figure 3 This is a block diagram illustrating a diagnostic device for AI-based multimodal medical data fusion, according to an embodiment of the specification. Detailed Implementation

[0013] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0014] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.

[0015] This disclosure provides a diagnostic method based on AI-driven multimodal medical data fusion. Figure 1 This is a flowchart illustrating an AI-based multimodal medical data fusion diagnostic method according to an embodiment. Specifically, the method includes: In step S11, multi-source medical data is constructed based on the anatomical structure images and functional metabolic images transmitted by the image acquisition device under different imaging modes, the continuous waveform signals transmitted by the physiological monitoring device to reflect the functions of different physiological systems, and the medical record reports transmitted by the electronic medical record system.

[0016] Imaging equipment is a medical device used to acquire information about the internal structure and function of the human body. Different imaging modalities can produce different types of images. For example, X-rays, CT scans, and MRIs are used to acquire anatomical images, while PET scans are used to acquire functional metabolic images. Anatomical images are used to display the anatomical structures of internal organs and tissues, clearly showing the shape, location, and size of each structure. For example, CT images can clearly show structures such as bones and lungs. Anatomical images can clearly show the size, shape, and location of the heart, as well as the structure of the heart's chambers and the distribution of blood vessels. Functional metabolic images reflect the metabolic state of human organs and tissues. They assess functional activity by detecting the distribution and changes of metabolic substances. For example, PET images can show the glucose metabolism in the brain, reflecting brain function.

[0017] Physiological monitoring equipment is used to monitor the function of human physiological systems in real time. It can transmit continuous waveform signals, such as an electrocardiograph (ECG) to monitor cardiac electrical activity and an electroencephalogram (EEG) to monitor brain electrical activity. Different physiological systems can include at least one of the circulatory, nervous, and respiratory systems. Physiological signals from the circulatory system, such as ECG waveforms, can reflect the rhythm of the heart's electrical activity and indicate whether the heart rate and rhythm are normal; blood pressure waveforms can show changes in intravascular pressure. Relevant signals from the nervous system can indirectly reflect its regulatory effect on cardiac function, while signals from the respiratory system may interact with cardiac function.

[0018] An electronic medical record system is a system for managing patient medical information electronically. Case reports can include patient chief complaint information, present illness history, past medical history, physical examination results, and auxiliary examination reports in text form. Chief complaint information is the main symptoms and duration stated by the patient at the time of consultation, such as "chest pain for three days"; present illness history records the occurrence and development of the disease in detail; past medical history records the diseases the patient has had in the past, especially cardiovascular diseases; physical examination results include information obtained by the doctor through inspection, palpation, percussion, auscultation, etc.; and auxiliary examination reports cover the results of various examinations previously performed.

[0019] Multi-source medical data is a collection of medical-related data from different data sources, including image data transmitted by imaging acquisition equipment, waveform signal data transmitted by physiological monitoring equipment, and medical record report data transmitted by electronic medical record systems.

[0020] In this embodiment, modal identification processing is performed on anatomical structure images, functional metabolic images, continuous waveform signals, and medical record reports, adding metadata tags to each type of data, including data type, acquisition scenario, and clinical significance. For example, the metadata tags added to anatomical structure images and functional metabolic images can indicate that they are CT images, acquired in an emergency scenario, and used to assess cardiac structure; the tags for continuous waveform signals can indicate that they are electrocardiograms, acquired during monitoring, and used to monitor cardiac electrical activity; and the tags for medical record reports can indicate that they are current medical history, come from outpatient records, and reflect the disease progression.

[0021] The imaging protocol parameters of anatomical and functional metabolic images, the acquisition frequency parameters of continuous waveform signals, and the recording timestamp parameters of medical record reports are then extracted and used as associated index information to bind to the corresponding data items. Imaging protocol parameters for anatomical and functional metabolic images may include scan slice thickness and tube voltage, which affect image quality and display effects; the acquisition frequency parameters of continuous waveform signals determine the temporal resolution of the signal; for example, the acquisition frequency of an electrocardiogram affects the capture of heart rate changes; and the recording timestamp parameters of medical record reports reflect the timeliness of the information. Binding these parameters to the corresponding data items facilitates tracing the data acquisition conditions in subsequent analyses.

[0022] Then, a clinical integrity verification mechanism is used to screen data, eliminating missing and contradictory items that do not conform to clinical data standards, and retaining data that meets the requirements for diagnostic analysis to construct multi-source medical data. This mechanism checks whether anatomical and metabolic images are missing key anatomical structures, whether continuous waveform signals have prolonged interruptions, and whether medical records contain missing or contradictory information. For example, if an electrocardiogram dataset has multiple missing leads, it will be discarded; if the symptoms recorded in the text medical record clearly contradict the examination results, it will also be verified or discarded to ensure the reliability of the data used for diagnostic analysis.

[0023] In this embodiment, the image acquisition device acquires anatomical and functional metabolic images of the human body using different imaging technologies (such as X-ray imaging and magnetic resonance imaging), and these images are stored and transmitted in the form of digital signals. The physiological monitoring device acquires physiological signals of the human body in real time, such as electrocardiograms and electroencephalograms, and converts them into continuous waveform signals for transmission. The electronic medical record system stores and manages patient medical records, including medical history, symptoms, examination results, and other information. Through data interfaces or network communication technologies, this data from different devices is integrated and stored to form a collection containing multiple types of data, i.e., multi-source medical data.

[0024] In step S12, a dynamic association network is constructed based on the clinical association rules of each data item in the multi-source medical data. The nodes in the dynamic association network correspond to the data items in the multi-source medical data, and the attributes of the edges include association confidence and clinical dependency direction.

[0025] Clinical association rules are used to describe the clinical relationships between data items in multi-source medical data, such as the association between changes in a physiological signal and specific anatomical structure image features. Dynamic association networks are network structures that graphically represent the relationships between data items in multi-source medical data. Association confidence is an indicator used to measure the degree of association between data items represented by two nodes in a dynamic association network; its value typically ranges from 0 to 1, with higher values ​​indicating stronger associations. Clinical dependency direction describes the direction of the clinical association between data items represented by two nodes in a dynamic association network; for example, a change in a physiological signal may lead to the appearance of specific anatomical structure image features, or vice versa.

[0026] In this embodiment, firstly, based on medical knowledge and clinical experience, the clinical relationships between data items in multi-source medical data are analyzed, and clinical association rules are formulated. For example, abnormal changes in certain physiological signals may be associated with lesions in specific anatomical structures. Then, each data item in the multi-source medical data is used as a node in a dynamic association network. For data items with clinical associations, edges are established to represent their associations. The confidence level of the edge's attribute association is determined through statistical analysis of a large amount of clinical data or expert experience assessment, reflecting the tightness of the association between the two data items. The direction of clinical dependence is determined based on medical knowledge and clinical logic; for example, if a change in a certain physiological indicator may lead to the appearance of a specific symptom, then the direction of the edge points from the physiological indicator node to the symptom node. In this way, a network structure that can dynamically reflect the clinical relationships between data items in multi-source medical data is constructed.

[0027] For example, in the context of cardiovascular disease diagnosis, there are close clinical correlations among the various data items in multi-source medical data. For instance, myocardial hypertrophy shown in anatomical images may be related to hypertension in physiological signal data, and a history of diabetes recorded in a medical record may be associated with the occurrence of cardiovascular disease. Constructing a dynamic correlation network aims to present these relationships in a network form, where nodes represent specific data items, and the attributes of the edges describe the strength of the correlation and the direction of the dependency.

[0028] In step S13, hierarchical feature interaction processing is performed on the dynamic association network, and the node feature representation is updated through bidirectional information transmission between nodes to generate a cross-modal enhanced feature set.

[0029] Hierarchical feature interaction processing is a method for processing node features in dynamic interconnected networks. Through bidirectional information transmission between nodes, node feature representations are updated at different levels to achieve feature interaction and enhancement. Cross-modal enhanced feature sets are a set of features generated after hierarchical feature interaction processing. These features integrate information from different modalities and possess richer diagnostic information.

[0030] In this embodiment of the disclosure, in the dynamic association network, each node has an initial feature representation, which may come from the original features of the corresponding data item or features that have undergone preliminary processing. The processing is divided into multiple levels, in which nodes interact with neighboring nodes through edges.

[0031] Specifically, nodes receive information from neighboring nodes and update themselves based on their own features. This information transmission is bidirectional; nodes can both send and receive information from neighboring nodes. Through multiple levels of processing, node features are continuously fused and enhanced, considering not only their own data items but also information from other related data items. Ultimately, after all levels of processing, a set of cross-modal enhanced features that integrates data from different modalities is obtained. These features possess richer diagnostic information and can better reflect the patient's condition.

[0032] In step S14, the cross-modal enhanced feature set is input into the pre-trained diagnostic inference model, and fused diagnostic features are generated through the dual-path feature parsing mechanism of the diagnostic inference model.

[0033] The dual-path feature parsing mechanism processes and analyzes the input cross-modal enhanced feature set through two different paths to generate fused diagnostic features. These fused diagnostic features, generated after processing by the dual-path feature parsing mechanism of the diagnostic inference model, integrate information from different modalities and paths, enabling a more accurate reflection of the patient's condition.

[0034] In this embodiment, the pre-trained diagnostic inference model typically employs a deep learning architecture, such as a convolutional neural network (CNN), a recurrent neural network (RNN), or its variants. During the training phase, this model utilizes a large amount of labeled medical data to learn the mapping relationship from input features to diagnostic results. The dual-path feature parsing mechanism can divide the input cross-modal enhanced feature set into two paths for processing. One path may focus on extracting local information from the features, while the other path focuses on extracting global information from the features.

[0035] In each path, features are progressively extracted and transformed through a series of neural network layers (such as convolutional layers, pooling layers, and fully connected layers). Then, the features processed by the two paths are fused to generate fused diagnostic features. This dual-path feature parsing mechanism fully leverages the advantages of different paths to extract more comprehensive and accurate diagnostic features.

[0036] In step S15, candidate disease types are determined based on the overall matching degree between the fusion diagnostic features and the typical feature spectra of each standardized disease, and supporting and unsupported feature items for each candidate disease type are determined based on the sub-item matching degree between the fusion diagnostic features and the typical feature spectra of each standardized disease.

[0037] The standardized typical feature spectrum is a predefined set of typical features for various diseases, used to match with the fusion diagnostic features to determine the disease type. The overall matching degree is the comprehensive degree of matching between the fusion diagnostic features and all features in the standardized typical feature spectrum. It is a quantitative assessment of the overall fit between the fusion diagnostic features and the typical feature spectrum of a certain disease, obtained by comprehensively considering the matching of all feature dimensions, and is used for preliminary screening of candidate disease types. The sub-item matching degree is the degree of matching between the fusion diagnostic features and each sub-item feature in the standardized typical feature spectrum. It evaluates the degree of fit between the fusion diagnostic features and the typical feature spectrum in each feature dimension, used to determine the supporting and unsupporting features for each candidate disease type. Supporting features are those with a high matching degree with a specific candidate disease type in the sub-item matching of the fusion diagnostic features and the standardized typical feature spectrum; these features support the diagnosis of that candidate disease type. Unsupporting features are those with a low matching degree or no match with a specific candidate disease type in the sub-item matching of the fusion diagnostic features and the standardized typical feature spectrum; these features do not support the diagnosis of that candidate disease type.

[0038] First, the overall matching degree between the fused diagnostic features and the typical feature spectra of each standardized disease is calculated. This can be achieved by calculating the similarity between features (such as cosine similarity, Euclidean distance, etc.), and selecting the disease types with the highest similarity as candidate disease types. Then, for each candidate disease type, the matching degree between the fused diagnostic features and each sub-feature in the standardized typical feature spectra of that disease is further calculated. Based on a pre-set threshold, features with matching degrees higher than the threshold are identified as supporting features, indicating that the patient may have the candidate disease; features with matching degrees lower than the threshold are identified as unsupporting features, indicating that the patient does not have the candidate disease. In this way, the degree of conformity between each candidate disease type and the patient's actual situation can be analyzed more accurately.

[0039] Then, based on the overall matching degree, several disease types with high overall matching degree are selected as candidate disease types. For each candidate disease type, based on the sub-item matching degree, the feature items corresponding to the feature dimensions with high sub-item matching degree are determined as supporting feature items (features that support the diagnosis of the disease), and the feature items corresponding to the feature dimensions with low sub-item matching degree are determined as unsupporting feature items (features that do not support the diagnosis of the disease).

[0040] In step S16, a feature tracing chain for diagnostic criteria is established based on the source features traced by the supporting feature items in the cross-modal enhancement feature set, and a feature association graph is constructed based on the feature tracing chain and the node association relationship of the dynamic association network.

[0041] The feature tracing chain is a chain established based on the source features traced to in the cross-modal enhanced feature set supporting feature items, used to clarify the source of diagnostic criteria. The feature association graph is a graph constructed based on the node association relationships of the feature tracing chain and the dynamic association network, showing the association relationships between features in the diagnostic criteria.

[0042] In this embodiment of the disclosure, for each candidate disease type, the supporting features are traced back to their origin in the cross-modal augmentation feature set, that is, which original cross-modal augmentation features were processed to obtain these supporting features. Through this tracing, a feature source chain from supporting features to cross-modal augmentation features is established, clarifying the source of the diagnostic basis and enhancing the credibility of the diagnosis.

[0043] In this embodiment of the disclosure, a feature association graph is constructed by combining the feature tracing chain and the association relationships of nodes in the dynamic association network. This graph graphically displays the clinical association paths between various diagnostic features (including supporting features, source features, etc.), such as the causal relationship and accompanying relationship between a supporting feature and other features.

[0044] In this embodiment, for each supporting feature item of a candidate disease type, its source features are traced within the cross-modal enhanced feature set. Since the cross-modal enhanced feature set is obtained through hierarchical feature interaction processing, feature tracing can be achieved through backpropagation or recording the information transmission path during feature processing. The traced source features are connected according to a certain logical relationship to form a feature tracing chain, clarifying the original data source of each supporting feature item. Then, combining the node association relationships in the dynamic association network, the features in the feature tracing chain and their relationships are integrated and visualized to construct a feature association graph. The feature association graph can intuitively display the source and association relationships between various features in the diagnostic criteria, providing doctors with clearer and more comprehensive diagnostic information.

[0045] In step S17, a diagnostic result including the disease type, differential diagnosis basis, and feature association map is generated and output based on the candidate disease type, the feature tracing chain, and the feature association map.

[0046] In this embodiment, the patient's condition is comprehensively analyzed based on the determined candidate disease types, the established feature tracing chain, and the feature association graph. The candidate disease types are used as the primary diagnostic conclusion, while supporting and unsupported features are used as differential diagnostic criteria, explaining the reasons for selecting these candidate disease types and excluding other disease types. Simultaneously, the feature association graph is output as part of the diagnostic results, providing doctors with more intuitive and detailed diagnostic information, helping them better understand the diagnostic process and basis, and improving the accuracy and reliability of the diagnosis. Finally, the diagnostic results, including disease type, differential diagnostic criteria, and feature association graph, are output to the doctor or relevant personnel in an appropriate form (such as a report, interface display, etc.).

[0047] The aforementioned technical solution comprehensively acquires multi-source medical data, including imaging data, physiological signal data, and text medical record data. Through modal identification processing and associative index information binding, it ensures data integrity and traceability. Furthermore, when constructing a dynamic association network, it fully considers the clinical association rules between data items, enabling the network to accurately reflect the clinical relationships between data. Hierarchical feature interaction processing, through bidirectional information transmission between nodes, achieves deep fusion and enhancement of cross-modal features, fully exploring the potential connections and complementary information between different modalities. The pre-trained diagnostic reasoning model employs a dual-path feature parsing mechanism, capable of targeted processing based on the clinical attributes of different modalities, generating fused diagnostic features with clinical decision-making guidance, significantly improving the accuracy and reliability of diagnosis. The final output diagnostic results include disease type, differential diagnostic criteria, and feature association maps, enhancing the accuracy and comprehensiveness of diagnostic results. This provides doctors with comprehensive and detailed diagnostic information, helping them make more scientific and rational treatment decisions.

[0048] Optionally, in step S14, the step of inputting the cross-modal enhanced feature set into the pre-trained diagnostic inference model and generating fused diagnostic features through the dual-path feature parsing mechanism of the diagnostic inference model includes: In step S141, the cross-modal enhanced feature set is input into the feature allocation layer of the diagnostic reasoning model to obtain the anatomical feature vector and functional feature vector obtained by the reasoning analysis of the diagnostic reasoning model.

[0049] Anatomical feature vectors are feature vectors extracted from modal data such as medical images, reflecting information related to human anatomical structures. For example, vectors formed by encoding features such as the shape, size, and positional relationships of organs are used to describe anatomical features. Functional feature vectors are feature vectors mainly extracted from modal data reflecting human physiological functions (such as physiological signals and functional images). For example, vectors formed by processing features such as the electrical activity of the heart and the functional metabolic features of the brain are used to describe functional features.

[0050] In this embodiment, the feature allocation layer of the diagnostic inference model is specifically designed to perform preliminary analysis and classification of the input cross-modal enhanced feature set. This layer analyzes the input features through a series of complex neural network structures (such as combinations of convolutional neural networks and fully connected layers). Different neurons or neural network submodules are trained to identify and extract anatomically relevant features as well as functionally relevant features. For example, when processing medical image features, some neurons may focus on identifying anatomical structural features such as organ boundaries and morphology, thereby generating anatomical feature vectors; while other neurons may be more sensitive to regions in the image that reflect functional information such as tissue metabolism and blood flow, thus extracting functional feature vectors. In this way, the feature allocation layer can decompose the cross-modal enhanced feature set into anatomical feature vectors and functional feature vectors.

[0051] In step S142, an adaptive fusion strategy based on clinical weights is adopted to perform feature fusion processing on the cross-validated anatomical feature vector and the functional feature vector to obtain a fused feature vector.

[0052] Among them, the adaptive fusion strategy based on clinical weights is used to dynamically adjust the contribution ratio of each feature during the fusion process according to the importance (clinical weight) of different features in clinical diagnosis, so that the fused features can better reflect the key information of clinical diagnosis.

[0053] In this embodiment, the core of the adaptive fusion strategy based on clinical weights lies in considering the importance of different features in clinical diagnosis. First, anatomical and functional feature vectors are evaluated using cross-validation to ensure good stability and reliability under different data partitioning conditions. Then, based on clinical knowledge and experience, corresponding clinical weights are assigned to anatomical and functional features respectively. These weights reflect the degree to which anatomical and functional information contributes to the diagnostic results of a specific disease. During the fusion process, weighted summation or other adaptive fusion algorithms are used to combine the anatomical and functional feature vectors according to the assigned clinical weights. For example, if functional features are more critical in the diagnosis of a certain disease, then the functional feature vector will be given a greater weight during fusion, making the fused feature vector more prominent in terms of functional information while also taking into account anatomical features, thus obtaining a fused feature vector that better meets the needs of clinical diagnosis.

[0054] In step S143, the fused feature vector is subjected to clinical semantic enhancement processing through the feature sublimation layer of the diagnostic reasoning model. The enhanced fused feature vector is then mapped to a clinically interpretable feature space in combination with the disease diagnosis criteria to obtain the feature vector to be used.

[0055] The feature enhancement layer further processes the fused feature vectors to enhance their clinical semantic information, making them more consistent with the logic and standards of clinical diagnosis. Clinical semantic enhancement, within the feature enhancement layer, incorporates clinical knowledge and diagnostic criteria to process the feature vectors, making the information contained within them easier for clinicians to understand and apply, thus improving the clinical interpretability of the features. Feature vectors in a clinically interpretable feature space can be directly linked to clinical symptoms and diagnostic results, facilitating diagnostic decisions by clinicians based on these features.

[0056] In this embodiment, a pre-built clinical knowledge graph or disease diagnostic standard library is introduced to match and associate the fused feature vector with this clinical information. For example, for a certain disease, the clinical knowledge graph defines the typical symptoms, signs, and normal and abnormal ranges of various examination indicators related to it. The feature enhancement layer analyzes the correlation between the fused feature vector and this clinical information using deep learning algorithms (such as attention mechanisms), strengthens the feature components in the fused feature vector that are related to the clinical disease, and suppresses irrelevant or interfering features. In this way, the fused feature vector is mapped to a clinically interpretable feature space, so that each dimension in the generated feature vector corresponds to a specific clinical meaning, which facilitates clinicians to make diagnostic inferences based on these features and obtain the feature vector to be used.

[0057] In step S144, the decision-focusing module of the diagnostic reasoning model is invoked to perform key feature enhancement processing on the candidate feature vector, thereby increasing the expression intensity of the candidate feature components that are directly related to the diagnostic decision and generating the target feature vector.

[0058] Dimension normalization is an operation that adjusts and normalizes the dimensions of feature vectors to give them a uniform dimensional format, which facilitates subsequent output and clinical decision-making applications, while also ensuring the numerical stability of feature vectors.

[0059] In this embodiment, the relationship between the candidate feature vector and a predefined diagnostic decision target is analyzed. Deep learning feature selection and enhancement techniques (such as gradient-based feature importance assessment and feature masking) are used to identify key feature components that play a crucial role in the diagnostic decision. For example, in diagnosing a disease, certain physiological indicators or imaging features may be essential for determining the severity of the condition or identifying a treatment plan. The decision focusing module adjusts the weights of these key feature components in the candidate feature vector or performs specialized feature transformations to make their expression more prominent in the feature vector, thereby generating a target feature vector. In this way, the target feature vector can provide more direct and powerful support for diagnostic decisions, reducing interference from irrelevant information.

[0060] In step S145, the target feature vector is subjected to dimensional normalization processing through the output layer of the diagnostic reasoning model to generate fusion diagnostic features with clinical decision-making orientation.

[0061] In this embodiment, the dimensionality of the target feature vector may become complex or inconsistent due to various feature processing operations, which is detrimental to its application in clinical decision-making. The output layer maps the target feature vector to a pre-defined fixed-dimensional space with clinical decision-making orientation by transforming the network structure through fully connected layers or other dimensional transformations. This fixed-dimensional space is designed according to the actual needs of clinical diagnosis, and each dimension corresponds to specific diagnostic information or decision indicators. For example, in the diagnosis of a certain disease, the dimensions of the output layer may correspond to the type of disease, severity classification, recommended treatment plan, etc. Through dimensional normalization processing, fused diagnostic features with clear clinical decision-making orientation are generated.

[0062] Optionally, in step S141, inputting the cross-modal enhanced feature set into the feature allocation layer of the diagnostic inference model to obtain the anatomical feature vector and functional feature vector obtained by the inference analysis of the diagnostic inference model includes: In step S1411, the cross-modal enhancement feature set is input into the feature allocation layer of the diagnostic reasoning model, and the cross-modal enhancement features are divided into anatomical feature stream and functional feature stream according to the clinical attributes of each cross-modal enhancement feature.

[0063] Clinical attributes refer to the characteristics of cross-modal augmented features that are directly related to clinical diagnosis, reflecting the significance and use of the features in a medical clinical context. For example, a feature may correspond to the shape and size of a human organ (anatomical-related clinical attributes) or reflect changes in an indicator of a certain physiological function (functional-related clinical attributes). These attributes determine the type of feature in the diagnostic process. Anatomical feature flow refers to the feature information related to human anatomical structures in the cross-modal augmented feature set, such as the shape, location, boundaries, and tissue density of organs, as reflected in medical images or other modal data. Functional feature flow refers to the physiological functional state of the human body in the cross-modal augmented feature set, such as information on the electrical activity patterns of the heart, the functional metabolic level of the brain, and respiratory rate and depth.

[0064] In this embodiment, each cross-modal augmentation feature is evaluated according to predefined clinical attribute classification rules. These rules are based on medical knowledge and experience; for example, if a feature primarily reflects the morphological structure of an organ, it is classified as an anatomical feature; if a feature is primarily related to physiological function indicators, it is classified as a functional feature. In this way, the cross-modal augmentation feature set is divided into anatomical feature streams and functional feature streams, preparing for subsequent anatomical and functional feature analysis. This division enables the model to process different types of features more specifically, improving the efficiency and accuracy of feature extraction and analysis.

[0065] In step S1412, the anatomical feature flow is processed through the anatomical feature parsing path of the diagnostic reasoning model, and feature refinement extraction, spatial relationship modeling and pathological morphology recognition operations are performed in sequence to generate anatomical feature vectors.

[0066] Feature refinement extraction is a process of deeper and more detailed mining of features in the anatomical feature stream within the anatomical feature parsing path. Through specific algorithms and model structures, more representative and discriminative sub-features are extracted from the original anatomical features to more accurately describe the detailed features of the anatomical structure. Spatial relationship modeling analyzes the spatial relationships between various anatomical structures, such as relative positions, distances, and angles, and models these spatial relationships using mathematical models or neural networks. This helps to understand the spatial organization and mutual influence of anatomical structures, providing more comprehensive anatomical information for diagnosis. Pathological morphology recognition, within the anatomical feature parsing path, uses trained models or algorithms to identify abnormal morphologies within the anatomical features. These abnormal morphologies are often associated with diseases. For example, the specific shape of tumors, organ atrophy, or hypertrophy are considered important components of anatomical features.

[0067] In this embodiment, feature refinement extraction is first performed using structures such as Convolutional Neural Networks (CNNs) to conduct multi-level convolution and pooling operations on the features in the anatomical feature stream. Convolutional kernels can capture anatomical structural features at different scales; as the network depth increases, more subtle and discriminative features are gradually extracted. Next, spatial relationship modeling is performed using techniques such as Graph Neural Networks (GNNs) or Spatial Transform Networks (STNs) to analyze and model the spatial relationships between anatomical structures. For example, GNNs can treat anatomical structures as nodes in a graph, with spatial connections between them as edges. Through information transmission and aggregation between nodes, the spatial organization of anatomical structures is understood. Finally, pathological morphology recognition is performed using object detection or image segmentation algorithms, combined with pre-trained pathological morphology templates, to identify abnormal morphologies in the anatomical features. The refined features, spatial relationship model, and pathological morphology recognition results are integrated to generate an anatomical feature vector, which comprehensively and accurately describes the feature information of human anatomical structures.

[0068] In step S1413, the functional feature flow is processed through the functional feature parsing path of the diagnostic reasoning model, and the time-series trend analysis, physiological rhythm extraction and functional state assessment operations are performed in sequence to generate functional feature vectors.

[0069] Among these, temporal trend analysis analyzes the changing trends of features in the functional feature stream over time within the functional feature analysis path. By observing the numerical changes of functional indicators at different time points, it reveals the dynamic patterns of physiological function changes, such as the fluctuations in blood sugar levels throughout the day. Circadian rhythm extraction identifies and extracts inherent physiological rhythm information from functional features, such as the body's diurnal rhythm and sleep-wake cycle, and their impact on physiological function. This rhythmic information is crucial for accurately assessing physiological function status. Functional status assessment, based on the results of temporal trend analysis and circadian rhythm extraction, comprehensively evaluates the body's physiological function status, determining whether function is normal, whether functional impairment exists, and the degree of functional impairment.

[0070] In this embodiment, a temporal trend analysis is first performed using recurrent neural networks (RNNs) and their variants (such as Long Short-Term Memory networks (LSTM) or gated recurrent units (GRUs) to model the temporal data in the functional feature stream. These networks can capture the changing patterns of data over time and predict future trends in functional indicators by analyzing feature values ​​at different time points. Next, physiological rhythm extraction is performed. Signal processing methods such as Fourier transform or wavelet transform are used to decompose the functional feature data into different frequency components, identifying frequency components related to human physiological rhythms, thereby extracting physiological rhythm information. Finally, functional status assessment is performed. Combining the results of temporal trend analysis and physiological rhythm extraction, classification or regression algorithms (such as support vector machines (SVM) or decision trees) are used to assess the physiological functional status of the human body. The results of these three steps are combined to generate a functional feature vector, which accurately reflects the characteristic information of human physiological function.

[0071] In step S1414, the anatomical feature vector and the functional feature vector are input into the cross-validation layer of the diagnostic reasoning model. The clinical consistency between the anatomical feature vector and the functional feature vector is tested through the feature mutual verification mechanism. The feature components with contradictions are marked, and the anatomical feature vector and functional feature vector obtained by the reasoning analysis of the diagnostic reasoning model are obtained.

[0072] In this embodiment, the generated anatomical feature vector and functional feature vector are input into the cross-validation layer of the diagnostic reasoning model. In the cross-validation layer, a feature verification mechanism is used to test these two feature vectors. Specifically, for corresponding feature components in the anatomical and functional feature vectors (these feature components should theoretically describe different aspects of the same clinical phenomenon or disease), their consistency in clinical significance is compared using a pre-established clinical knowledge association model. For example, if anatomical features show enlargement of an organ, while the corresponding indicator reflecting the function of that organ in the functional features shows normality, this may be contradictory. By setting certain thresholds and judgment rules, when this contradiction exceeds a certain level, the contradictory feature component is marked. After verification by the feature verification mechanism, anatomical and functional feature vectors that have passed clinical consistency verification are obtained.

[0073] Optionally, in step S1412, the anatomical feature flow is processed through the anatomical feature parsing path of the diagnostic reasoning model, sequentially performing feature refinement extraction, spatial relationship modeling, and pathological morphology recognition operations to generate an anatomical feature vector, including: The anatomical feature stream is input into the multi-scale extraction submodule of the anatomical feature parsing path, and feature information of anatomical structures of different sizes is captured by feature extraction units with different receptive fields to generate multi-scale anatomical features.

[0074] The multi-scale extraction submodule can process the input anatomical feature stream using feature extraction units with different receptive fields. The receptive field refers to the size of the input region seen by each neuron in the neural network. Feature extraction units with different receptive fields can capture anatomical structural feature information of different sizes, thereby generating multi-scale anatomical features to comprehensively describe the details and overall features of the anatomical structure.

[0075] In this embodiment, the multi-scale extraction submodule typically employs a convolutional neural network (CNN) architecture, which includes multiple convolutional kernels of different sizes. Different sized kernels have different receptive fields; larger kernels can capture a wider range of anatomical features, suitable for extracting overall, macroscopic anatomical information; while smaller kernels are more sensitive to detailed features and can capture minute changes in anatomical structures. When the anatomical feature stream is input to this submodule, feature extraction units in different receptive fields perform convolution operations on the input data in parallel, gradually extracting features at different levels through multi-layer convolution, pooling, and other operations. Finally, the features extracted from different receptive fields are fused to generate multi-scale anatomical features that comprehensively cover anatomical structural information from microscopic to macroscopic levels.

[0076] Based on the clinical importance of anatomical structures, redundant non-critical features are removed from the multi-scale anatomical features to obtain key anatomical features. These key anatomical features are then input into the spatial relationship modeling submodule, and a spatial positional relationship graph between anatomical structures is constructed using a graph structure learning algorithm. The nodes of the spatial positional relationship graph represent anatomical structures, and the edges represent the relative positional relationships between the structures.

[0077] Spatial location relationship graphs can be used to represent the spatial relationships between anatomical structures using a graph structure. Nodes in the graph represent different anatomical structures, while edges represent the relative positional relationships between these structures, such as adjacency or containment. This graph structure allows for an intuitive understanding and analysis of the spatial layout and mutual influences of anatomical structures.

[0078] In this embodiment, firstly, based on medical expertise and clinical experience, predefined criteria for assessing the clinical importance of anatomical structures are established. These criteria may include the degree of association between anatomical structures and common diseases, their crucial role in diagnosis, etc. Then, these criteria are used to screen multi-scale anatomical features, eliminating non-critical features that contribute little to diagnosis or are redundant. After obtaining the key anatomical features, they are input into a spatial relationship modeling submodule. This submodule employs a graph structure learning algorithm, mapping each key anatomical feature to a node, and determining the connections (edges) between nodes by analyzing the relative positional information between anatomical structures. For example, if two anatomical structures are spatially adjacent, an edge is established between their corresponding nodes. In this way, a spatial positional relationship graph between anatomical structures is constructed, intuitively representing the spatial layout and interrelationships of the anatomical structures.

[0079] The topological features and node attribute features of the spatial location association graph are extracted to generate a spatial relationship feature vector. The spatial relationship feature vector is then input into the pathological morphology recognition submodule. An abnormal morphological pattern in the spatial relationship feature vector is detected by a pre-trained morphological recognition unit, and possible pathological morphological regions are marked.

[0080] Topological features refer to the connectivity and structural characteristics between nodes and edges in a spatial relationship graph, such as the degree of a node (the number of edges connected to that node) and the connectivity of the graph. These features reflect the overall structure and pattern of spatial relationships between anatomical structures. Node attribute features are the attribute information possessed by each node in the spatial relationship graph. Besides representing the anatomical structure itself, they may also include some characteristic parameters of the anatomical structure, such as size and shape. These attribute features provide more detailed information for further analysis of the anatomical structure. The spatial relationship feature vector is a vector extracted and combined from the topological features and node attribute features of the spatial relationship graph. It integrates the spatial positional relationships between anatomical structures and the attribute information of the nodes themselves, and can comprehensively describe the spatial characteristics of the anatomical structure.

[0081] In this embodiment, for the constructed spatial location association graph, graph feature extraction methods are used to extract its topological features and node attribute features. Topological features can be obtained by calculating indicators such as graph connectivity and node degree distribution; these features reflect the overall structural pattern of spatial relationships between anatomical structures. Node attribute features can be extracted from the anatomical structure corresponding to each node, such as the size and shape of the anatomical structure. The topological features and node attribute features are combined to generate a spatial relationship feature vector. Then, this feature vector is input into the pathological morphology recognition submodule. The pre-trained morphological recognition unit in this submodule is trained based on a large amount of medical imaging data and pathological annotation information; it can learn the feature patterns of normal and abnormal anatomical morphologies. By matching and comparing the spatial relationship feature vector with the feature patterns in the pre-trained model, abnormal morphological patterns are identified, and possible pathological morphological regions are marked.

[0082] The marked pathological morphological regions are subjected to feature quantization processing to extract the morphological parameters and edge features of the pathological morphological regions and generate pathological morphological features.

[0083] Morphological parameters are quantitative indicators used to describe the characteristics of pathological morphological regions, such as area, perimeter, roundness, and aspect ratio. These parameters objectively quantify the characteristics of pathological morphology, helping doctors to more accurately determine the nature and extent of lesions. Edge features are the geometric and textural features of the edges of pathological morphological regions, such as edge smoothness, curvature, and texture complexity. Edge features play an important role in distinguishing different types of pathological morphologies and can provide detailed information about the boundaries of lesions.

[0084] In this embodiment, image processing and feature extraction algorithms are used to quantify the marked pathological morphological regions. For morphological parameters, the size and shape of the pathological morphological region are objectively described by calculating indicators such as area fraction, perimeter, roundness, and aspect ratio. For edge features, edge detection algorithms (such as the Canny edge detection algorithm) are used to extract the edges of the pathological morphological region, and then features such as edge smoothness, curvature, and texture complexity are calculated to reflect the geometric and textural information of the lesion boundary. These morphological parameters and edge features are integrated to generate pathological morphological features. These features accurately describe the characteristics of the pathological morphology and provide crucial pathological information for the generation of anatomical feature vectors.

[0085] The spatial relationship feature vector and the pathological morphological features are combined to obtain the anatomical feature vector.

[0086] In this embodiment, a feature concatenation method can be used to connect two feature vectors in a certain order to form a longer feature vector. Alternatively, a weighted fusion method can be used, assigning different weights to the two feature vectors based on the importance of spatial relationships and pathological morphology in diagnosis, and then adding the weighted feature vectors to obtain the final anatomical feature vector. This fusion method can fully utilize information from both spatial relationships and pathological morphology, enabling the anatomical feature vector to more accurately describe the characteristics of anatomical structures and pathological states.

[0087] Optionally, in step S1413, the functional feature flow is processed through the functional feature parsing path of the diagnostic inference model, sequentially performing time-series trend analysis, physiological rhythm extraction, and functional state assessment operations to generate a functional feature vector, including: The functional feature stream is input into the time-series processing submodule of the functional feature parsing path. The time-series processing submodule sorts the features in the functional feature stream according to the timestamp order to construct a time-series feature sequence.

[0088] In this embodiment, the feature data is rearranged in chronological order by reading these timestamps. During the sorting process, the validity of each feature data's timestamp is checked, and data with invalid or missing timestamps is processed accordingly, such as being removed or interpolated reasonably based on the time of preceding and following data. After sorting, the functional feature data are arranged sequentially in chronological order, forming a time-series feature sequence, providing an ordered data foundation for subsequent time-series analysis, enabling accurate analysis of changes in functional features over time.

[0089] The sliding window analysis method is used to capture the changing trends of each feature in the time series within different time intervals, generate trend feature components, and input the trend feature components into the physiological rhythm recognition submodule. The periodic rhythm pattern of the trend feature components is extracted by the spectrum analysis algorithm to determine the periodic and amplitude characteristics of the rhythm.

[0090] Amplitude characteristics refer to the magnitude of periodic changes in a physiological rhythm pattern, reflecting the intensity of these rhythmic changes. For example, the fluctuation range of blood pressure throughout the day can be described by amplitude characteristics; a larger amplitude indicates more drastic blood pressure changes.

[0091] In this embodiment, the sliding window analysis method sets a fixed-size window on the time-series feature sequence. The size of the window can be set according to the analysis requirements and the characteristics of the data. The window starts from the beginning of the time-series feature sequence and slides gradually along the time axis. At each window position, statistics of the feature data within the window are calculated, such as mean, variance, and slope. These statistics reflect the trend of the feature within that time interval. By continuously sliding the window and calculating the statistics, a series of values ​​describing the trend of feature change are obtained. Combining these values ​​generates the trend feature component.

[0092] After the trend feature components are input into the physiological rhythm recognition submodule, they are processed using a spectrum analysis algorithm. The spectrum analysis algorithm transforms the trend feature components from the time domain to the frequency domain, and decomposes the time-domain signal into a superposition of sine and cosine functions of different frequencies using mathematical methods such as Fourier transform. In the frequency domain, the frequency components and amplitudes of the signal are analyzed to identify the periodic rhythmic patterns present. The periodicity is determined by finding the period corresponding to the main frequency components in the frequency domain, while the amplitude is represented by the amplitude corresponding to that frequency component. In this way, the periodic rhythmic patterns in the trend feature components can be accurately extracted, and the periodicity and amplitude characteristics of the rhythm can be determined.

[0093] Based on the periodic and amplitude characteristics, a physiological rhythm model is constructed to describe the regularity of functional characteristics changing over time. The physiological rhythm model is then compared and analyzed with a preset normal physiological rhythm template to calculate the rhythm deviation parameter.

[0094] Among them, the rhythm deviation parameter is obtained by comparing and analyzing the physiological rhythm model with a preset normal physiological rhythm template. It is used to quantify the degree of difference between an individual's physiological rhythm and the normal physiological rhythm. These parameters can reflect aspects such as the period deviation and amplitude deviation of the rhythm.

[0095] In this embodiment, a physiological rhythm model is constructed based on the extracted periodic and amplitude features. Physiological rhythm models are typically represented by mathematical functions, such as sine or cosine functions, which can effectively simulate periodic changes. By using the periodic and amplitude features as parameters of the function, a physiological rhythm model capable of describing the regularity of functional characteristics changing over time is constructed.

[0096] Then, the constructed circadian rhythm model is compared and analyzed with a pre-defined normal circadian rhythm template. The normal circadian rhythm template is a standard model pre-defined based on a large amount of physiological data from healthy individuals and medical research results; it represents the characteristics of a normal circadian rhythm. During the comparison analysis, the differences between the circadian rhythm model and the normal circadian rhythm template in terms of period and amplitude are calculated. Rhythm deviation parameters can be determined by calculating the absolute difference, relative difference, or using other statistical methods. These rhythm deviation parameters can quantify the degree of deviation between an individual's circadian rhythm and a normal circadian rhythm. The rhythm deviation parameters are input into the functional status assessment submodule, and functional status level characteristics are generated by combining them with clinical functional assessment standards.

[0097] Among them, the functional status level features are generated by the functional status assessment submodule based on rhythm deviation parameters and clinical functional assessment standards. They are used to represent different levels of human functional status, such as normal, mildly abnormal, moderately abnormal, and severely abnormal. These level features can help doctors quickly understand the patient's functional status.

[0098] In this embodiment, the functional status assessment submodule receives rhythm deviation parameters and then performs a functional status assessment based on pre-set clinical functional assessment standards. The clinical functional assessment standards consider factors such as the normal range of different physiological indicators, manifestations of functional abnormalities, and their relationship to diseases, and set corresponding thresholds and assessment rules for each rhythm deviation parameter.

[0099] For example, for the period deviation parameter, if its value is within the normal range, the rhythm's period is considered normal; if it exceeds a certain range, it is judged as mild, moderate, or severe abnormality based on the degree of deviation. Similarly, amplitude deviation parameters are processed in a similar manner. By synthesizing the evaluation results of various rhythm deviation parameters and combining them with the comprehensive judgment rules in clinical functional assessment standards, functional status level characteristics are generated, such as normal, mildly abnormal, moderately abnormal, and severely abnormal. These functional status level characteristics can intuitively reflect the body's functional state.

[0100] The functional feature vector is obtained by integrating the trend feature components, the rhythm deviation parameters, and the functional state level features.

[0101] In this embodiment of the disclosure, to obtain a comprehensive and integrated functional feature vector, it is necessary to fuse trend feature components, rhythm deviation parameters, and functional state level features. A feature concatenation method can be used to connect these three features in a specific order to form a longer feature vector. During the concatenation process, it is necessary to ensure that the dimensions and order of each feature are consistent for subsequent processing and analysis.

[0102] Alternatively, a weighted fusion method can be used, assigning different weights to the three features—trend characteristics, rhythm deviations, and functional status assessments—based on their relative importance within the functional characteristics. For example, if functional status assessment is more critical in diagnosis, the functional status level feature can be given a higher weight. The weighted features are then summed or other fusion operations are performed to obtain the final functional feature vector. This fusion approach fully utilizes information from trend characteristics, rhythm deviations, and functional status assessments, enabling the functional feature vector to more accurately describe the body's functional characteristics.

[0103] Optionally, in step S1414, the step of inputting the anatomical feature vector and the functional feature vector into the cross-validation layer of the diagnostic reasoning model, verifying the clinical consistency of the anatomical feature vector and the functional feature vector through a feature mutual verification mechanism, marking contradictory feature components, and obtaining the anatomical feature vector and functional feature vector obtained by the reasoning analysis of the diagnostic reasoning model includes: Extract the key anatomical feature components and their corresponding anatomical clinical significance from the anatomical feature vector, and extract the key functional feature components and their corresponding functional clinical significance from the functional feature vector.

[0104] Key anatomical feature components are clinically significant feature data extracted from anatomical feature vectors. These components reflect crucial information about human anatomical structures, such as organ size, shape, and location abnormalities, and play a vital indicative role in diagnosing related diseases. The anatomical clinical significance description is a textual explanation of the meaning and role of key anatomical feature components in clinical diagnosis. It details the association between the anatomical feature and a specific disease or physiological state, helping to understand the importance of anatomical features in the diagnostic process. The functional clinical significance description explains the clinical significance and role of key functional feature components. It illustrates the connection between changes in functional features and the occurrence, development, and treatment effects of diseases.

[0105] In this embodiment, the anatomical feature vector is used to identify key anatomical feature components with significant diagnostic value as critical anatomical feature components through a preset feature selection algorithm and a clinical knowledge base. These algorithms can select features based on indicators such as variance, correlation, and disease association. Simultaneously, anatomical clinical significance descriptions corresponding to these critical anatomical feature components are obtained from the clinical knowledge base. These descriptions are based on extensive medical research and clinical practice, detailing the role and significance of each anatomical feature in disease diagnosis.

[0106] For functional feature vectors, a similar method is used to extract key functional feature components. For example, key functional features are determined based on factors such as the range of variation in functional indicators and sensitivity to diseases. Then, corresponding clinical significance descriptions of these functions are obtained from a clinical knowledge base to explain the relationship between changes in these functional features and diseases, such as how a rapid heart rate may be related to heart disease or hyperthyroidism.

[0107] Based on clinical diagnostic logic, and combining the descriptions of anatomical clinical significance and functional clinical significance, the expected correlation between the key anatomical feature components and the key functional feature components is established.

[0108] The anticipated correlation is a theoretical link between key anatomical and functional features, pre-established based on clinical diagnostic logic and combining descriptions of anatomical and functional clinical significance. This relationship reflects the mutual influence between anatomical structure and functional state under normal conditions and serves as a standard for practical comparison.

[0109] In this embodiment of the disclosure, the interaction between anatomical structure and functional state is analyzed based on the description of anatomical clinical significance and the description of functional clinical significance. For example, if a key anatomical feature component indicates hepatomegaly, according to clinical knowledge, hepatomegaly may affect liver function, leading to abnormalities in certain liver function indicators, thereby establishing a anticipated correlation between the anatomical feature of hepatomegaly and the functional feature of abnormal related liver function indicators.

[0110] Establishing such anticipated correlations requires consideration of multiple factors, including the physiological relationship between anatomical structure and function, and the patterns of anatomical and functional changes during disease progression. By integrating this information, a complete set of anticipated correlations that conforms to clinical practice can be formed.

[0111] Compare each of the actual key anatomical feature components and key functional feature components to see if they match the expected correlation, and record contradictory feature pairs that do not match the correlation.

[0112] Inconsistent feature pairs are feature combinations that do not conform to the expected correlation when comparing the actual key anatomical feature components with the key functional feature components one by one. The appearance of these feature pairs indicates an inconsistency between anatomical and functional features, which may suggest an underlying disease or abnormal condition.

[0113] In this embodiment of the disclosure, the actually acquired key anatomical feature components and key functional feature components are compared one by one according to the pre-established expected correlation. For each pair of key anatomical features and key functional features, it is checked whether the actual relationship between them is consistent with the expected correlation.

[0114] For example, if liver enlargement is expected to be associated with an elevated liver function indicator, but in actual data, the liver is enlarged while the liver function indicator is normal, then this pair of features is considered to have a contradictory relationship and is recorded as a contradictory feature pair. In this way, the consistency between anatomical and functional features can be comprehensively examined, and potential anomalies can be identified.

[0115] The contradictory features are quantified to determine their degree of contradiction, and a contradiction index is calculated based on the probability and impact of the contradiction in clinical diagnosis.

[0116] In this embodiment, firstly, based on clinical data and medical literature, the probability of each contradictory feature pair occurring in clinical diagnosis is statistically analyzed. For example, certain contradictory combinations may be more common in specific diseases but rarely occur in others. Simultaneously, the degree of influence of the contradiction on the diagnostic result is assessed. Some contradictions may have a minor impact on the diagnosis, while others may directly alter the diagnostic conclusion. By establishing a mathematical model, the probability of contradiction occurrence and the degree of influence are weighted and calculated to obtain a specific contradiction index. This index can objectively reflect the severity of the contradiction.

[0117] The feature components whose contradiction index exceeds a preset contradiction index threshold of the contradictory feature pair are marked to mark the feature components that are contradictory, thus obtaining contradiction markings.

[0118] In this embodiment of the disclosure, when the contradiction index of a contradictory feature pair exceeds a certain threshold, the contradiction is considered to have sufficient clinical significance and may have a significant impact on the diagnostic result. These feature components exceeding the threshold are specially marked, for example, using different colors, symbols, or adding specific annotations. These contradiction markings can draw the doctor's attention during subsequent diagnostic reasoning, prompting the doctor to further analyze and judge these contradictory features to determine whether a disease or abnormality exists.

[0119] Based on the anatomical feature vector, the functional feature vector, and the corresponding contradiction markers, the anatomical feature vector and functional feature vector obtained by the diagnostic reasoning model are obtained.

[0120] In this embodiment, the anatomical feature vectors, functional feature vectors, and corresponding contradiction markers after the above processing are integrated. The anatomical feature vectors and functional feature vectors contain key information about the human anatomical structure and functional state, while the contradiction markers highlight feature components where inconsistencies exist.

[0121] The integrated results provide more comprehensive and accurate information for diagnostic reasoning models. Physicians can conduct more in-depth diagnostic analysis based on the normality and abnormality of anatomical and functional features, combined with the potential problems indicated by contradictory markers, thereby improving the accuracy and reliability of diagnoses.

[0122] Optionally, in step S12, constructing a dynamic association network based on the clinical association rules of each data item in the multi-source medical data includes: The anatomical location identifiers, physiological indicator names, and disease description keywords corresponding to each data item are extracted and parsed to obtain the clinical semantic information of each data item in the multi-source medical data.

[0123] Anatomical location identifiers are used to clearly identify the names or codes of specific parts of the human anatomy, such as "heart," "liver," and "lungs," providing precise information for locating the anatomical position of diseases. Physiological indicator names are the names of various indicators used to measure human physiological functions and states, such as "blood pressure," "blood sugar," and "heart rate." These indicators reflect the normal functioning of various organs and systems in the body, as well as potential health problems. Disease description keywords are key words extracted from the disease description text to summarize the main characteristics of the disease, such as "pain," "fever," and "cough." These keywords help to quickly understand the basic situation of the disease.

[0124] In this embodiment of the disclosure, each data item in the multi-source medical data is parsed using a combination of natural language processing technology and a medical knowledge base. First, the text content of the data item is preprocessed using text segmentation and part-of-speech tagging techniques, breaking it down into individual words or phrases. Then, by matching and identifying relevant terms such as anatomical locations, physiological indicators, and disease descriptions in the medical knowledge base, anatomical location identifiers, physiological indicator names, and disease description keywords are extracted.

[0125] For example, in a medical record describing "the patient complains of chest pain, and an electrocardiogram shows ST segment elevation," parsing can extract the anatomical location marker "chest" and the keywords "pain" and "ST segment elevation" in the symptom description. Similarly, for a laboratory test result "blood glucose level 7.8 mmol / L," the physiological indicator name "blood glucose" and its value can be extracted. In this way, each data item is transformed into information with clear clinical semantics.

[0126] By introducing a medical knowledge graph to standardize and map clinical semantic information, the semantic information of different data items is unified into a standard medical terminology system, resulting in standardized clinical semantic information.

[0127] The standard medical terminology system is a set of terms used to uniformly describe medical concepts and information. Adopting standard medical terminology can avoid misunderstandings and communication barriers caused by inconsistencies in terminology, and improve the quality and interoperability of medical data.

[0128] In this embodiment, the medical knowledge graph contains a large number of medical concepts, entities, and relationships between them, and these concepts and entities are represented using standard medical terminology. The extracted clinical semantic information is matched and mapped with nodes in the knowledge graph, and the most suitable standard medical terminology is determined through methods such as semantic similarity calculation and concept reasoning.

[0129] For example, for the extracted symptom description keyword "chest pain," the standard term "chest pain" with similar semantics is found in the knowledge graph and mapped to it. Similarly, the anatomical location marker "abdomen" is mapped to the standard term "abdominal skin." This standardized mapping process eliminates semantic differences arising from different data sources and expression methods, unifying the semantic information of all data items within the standard medical terminology system, thus improving data consistency and comparability.

[0130] Based on standardized clinical semantic information, a pre-defined clinical association rule base is retrieved to determine the clinical association type between any two data items. The clinical association type includes at least one of causal association, concomitant association, and exclusion association.

[0131] Causal association refers to a clear causal relationship between two data items, meaning that a change in one data item causes a change in the other. For example, bacterial infection (cause) leads to fever (effect). Concomitant association indicates that two data items often coexist clinically, but there is no direct causal relationship between them. For example, hypertension and hyperlipidemia often coexist in patients, but hypertension is not the cause of hyperlipidemia, and vice versa. Exclusionary association means that two data items are mutually exclusive in clinical diagnosis; if one data item is present, the other is unlikely to be present. For example, in diagnosing a specific disease, the presence of certain specific symptoms may rule out the possibility of other diseases.

[0132] In this embodiment, the pre-defined clinical association rule base is established based on a large amount of clinical research, expert experience, and medical literature summaries, and includes association rules between various medical concepts. For each pair of data items corresponding to standardized clinical semantic information, retrieval and matching are performed in the clinical association rule base.

[0133] The clinical association type between data items is determined by calculating semantic similarity and checking for corresponding association conditions in the rule base. For example, if the rule base contains "bacterial infection → fever," a causal association can be determined when the data items "bacterial infection" and "fever" are retrieved. If the rule base contains "hypertension ∧ hyperlipidemia" (indicating that hypertension and hyperlipidemia often occur simultaneously), then a concomitant association is determined when these two data items are retrieved. In this way, the clinical association type between data items in multi-source medical data can be comprehensively determined.

[0134] Based on the clinical association type, the frequency of citation of the clinical association type in clinical guidelines, and the level of evidence-based medicine, the association confidence parameters between the calculated data items are determined.

[0135] The association confidence parameter is a numerical indicator used to measure the reliability and strength of the association between two data items. This parameter takes into account factors such as clinical association rules and evidence-based medicine, and its value usually ranges from 0 to 1. The higher the value, the more reliable the association.

[0136] In this embodiment of the disclosure, firstly, different types of clinical associations have different effects on the confidence level. Generally speaking, the confidence level of causal association is relatively high, followed by co-occurrence association, while the confidence level of exclusion association needs to be evaluated according to the specific circumstances.

[0137] Next, refer to the citation frequency of this clinical association type in clinical guidelines. The higher the citation frequency, the more widely the association is recognized in clinical practice, and the higher its confidence level. At the same time, consider the level of evidence-based medicine. The level of evidence-based medicine is divided into multiple levels, such as Level A (high-quality randomized controlled trials), Level B (moderate-quality studies), etc. The higher the level of evidence, the stronger the reliability of the association, and the higher the confidence parameter.

[0138] By establishing a mathematical model, the clinical association type, citation frequency, and evidence-based medicine level are weighted and calculated to obtain the final association confidence parameter. For example, the following formula can be used: Confidence parameter = α × association type weight + β × citation frequency weight + γ × evidence level weight, where α, β, and γ are weight coefficients set according to the actual situation.

[0139] Based on the disease progression sequence and examination process logic, a directed correlation is formed, and the clinical dependence direction between data items is obtained.

[0140] In this context, the disease progression sequence refers to the chronological order of the various stages from the onset to the development and changes of a disease. Understanding the disease progression sequence is crucial for determining the clinical dependencies between data items; for example, earlier symptoms may be the cause of subsequent abnormal test results.

[0141] In this embodiment of the disclosure, the disease development sequence reflects the evolution of the disease over time, and the order in which different symptoms and examination results appear often implies a causal relationship or dependency between them. For example, if a patient first develops a cough and then undergoes a chest X-ray that reveals lung inflammation, then there is a clinical dependency between the cough symptoms and the lung inflammation examination result, moving from symptoms to examination results.

[0142] The logical sequence of examination procedures refers to the rational order in which medical institutions conduct various examinations, and it also determines the sequential relationship between different examination results. For example, in blood tests, a routine blood test is usually performed first, followed by further biochemical or specialized tests as needed. Therefore, there is a clinically dependent order between the results of the routine blood test and subsequent biochemical tests.

[0143] By comprehensively analyzing the disease progression sequence and examination process logic, and combining the previously identified clinical association types, a clear direction is determined for each association, forming a directed association, thereby more accurately describing the clinical dependencies between data items.

[0144] Each data item in the multi-source medical data is used as a node in the dynamic association network, and the corresponding association confidence parameter and the clinical dependence direction are used as attributes of the edges between nodes to initialize the topology of the dynamic association network.

[0145] In this embodiment of the disclosure, when constructing the dynamic association network, each medical data item is regarded as a node in the network. Each node has a unique identifier to distinguish different data items. The edges between nodes represent the association relationships between data items, and the attributes of the edges include association confidence parameters and clinical dependence directions.

[0146] Based on the relationships, association confidence parameters, and clinical dependency directions determined in the previous steps, the corresponding nodes are connected by edges, and each edge is assigned a corresponding attribute value. For example, if there is a causal relationship between data item A and data item B, the association confidence parameter is 0.8, and the clinical dependency direction is from A to B, then an edge is drawn in the network pointing from node A to node B, and the association confidence parameter of the edge is set to 0.8. In this way, the topology of the dynamic association network is initialized.

[0147] Based on preset clinical scenario parameters, the attribute values ​​of the edges are dynamically adjusted to improve the association confidence parameter and strengthen the constraint of the dependency direction for key relationships in critical care scenarios.

[0148] In this embodiment of the disclosure, different clinical scenarios have different requirements and impacts on the relationships between data items. Preset clinical scenario parameters include factors such as disease severity, patient age, gender, and underlying diseases. For example, in acute and critical care scenarios, patients' conditions change rapidly, requiring higher accuracy and timeliness in key relationships.

[0149] For critical relationships in acute and critical care scenarios, their reliability is enhanced by increasing the association confidence parameter. For example, in normal scenarios, the association confidence parameter between two data items might be 0.6, while in acute and critical care scenarios, it is increased to 0.8. Simultaneously, the constraint of dependency direction is strengthened to ensure accurate grasp of the order and causal relationship between data items during clinical decision-making. For instance, for dependencies between some key treatment measures and examination results, in acute and critical care scenarios, it is required to strictly follow the dependency direction to avoid worsening of the condition due to incorrect order. By dynamically adjusting the attribute values ​​of the edges, the dynamic association network can better adapt to the needs of different clinical scenarios.

[0150] The dynamic association network is structurally optimized by using a network evolution algorithm, and low-confidence association edges are pruned according to the clinical importance of nodes to form the dynamic association network.

[0151] In this embodiment, the network evolution algorithm typically includes operations such as adding, deleting, and adjusting nodes and edges, with the goal of making the network structure more rational and efficient according to certain optimization criteria. In dynamically connected networks, a network evolution algorithm is used to optimize the initialized network.

[0152] First, edges in the network are evaluated based on the clinical importance of nodes. Edges connected to nodes with high clinical importance generally have higher value, while edges with low confidence that connect to nodes with lower clinical importance can be pruned. For example, a confidence threshold can be set, and edges whose association confidence parameter falls below this threshold are removed from the network. Simultaneously, the network evolution algorithm can dynamically add new nodes and edges, or adjust the attributes of existing edges, based on data updates and changes in the clinical context. Through continuous iterative optimization, the structure of the dynamically related network gradually becomes stable and reasonable, more accurately reflecting the complex relationships between data items in multi-source medical data.

[0153] Optionally, in step S13, the hierarchical feature interaction processing performed on the dynamic association network, updating the node feature representation through bidirectional information transmission between nodes, and generating a cross-modal enhanced feature set, includes: The basic features corresponding to each node in the dynamic association network are extracted. The basic features include anatomical morphological features represented by anatomical structure images, functional morphological features represented by functional metabolic images, functional trend features in continuous waveform signals, and clinical semantic features carried by case reports.

[0154] In this embodiment, image processing and computer vision techniques, such as edge detection, morphological analysis, and feature extraction algorithms (such as SIFT and HOG), are used to extract anatomical and functional morphological features. Anatomical morphological features can describe the shape, size, and location of human organs and tissues, providing basic information about the patient's body structure for subsequent analysis.

[0155] For continuous waveform signals, signal processing techniques such as Fourier transform and wavelet transform are used to convert the time-domain signal into the frequency domain or other feature domains, and then functional trend features are extracted. For example, heart rate variability features are extracted from electrocardiogram (ECG) signals, and brain rhythm features are extracted from electroencephalogram (EEG) signals. These features reflect the dynamic changes in human physiological functions.

[0156] For medical record reports, natural language processing techniques, such as lexical analysis, syntactic analysis, and semantic understanding, are used to extract clinical semantic features. By identifying key information in the medical record, such as medical terminology, symptom descriptions, and diagnostic results, and converting them into semantic representations that computers can understand and process, the clinical semantic connotations of the patient's condition can be captured.

[0157] Modality adaptation encoding is performed on the basic features of each node to transform the basic features of different modalities into a unified feature space, generating modality-compatible features.

[0158] In this embodiment of the disclosure, an encoder-decoder structure or a multimodal fusion model from deep learning is used to achieve modality-adaptive encoding. For example, different neural network encoders can be designed for image features and physiological signal features respectively (such as convolutional neural networks (CNN) for image feature encoding, and recurrent neural networks (RNN) or their variants LSTM and GRU for physiological signal feature encoding), mapping the basic features extracted from them to a low-dimensional latent feature space.

[0159] For text features, pre-trained language models (such as BERT, GPT, etc.) can be used to encode them, converting them into vector representations. Then, by designing a shared fusion or mapping layer, these modally encoded features are further transformed into a unified feature space. In this process, the model learns the correspondence and transformation rules between features of different modalities, enabling the generated modality-compatible features to be represented and interact at the same semantic level, thereby eliminating the impact of modality differences on subsequent feature fusion and analysis.

[0160] Based on the clinical dependence direction of the edges in the dynamic association network, the information transmission order between nodes is determined, and node features are transmitted forward along the clinical dependence direction and feature update information is fed back in the reverse dependence direction.

[0161] In this embodiment of the disclosure, the edges in the dynamic association network are constructed based on the actual relationships between clinical knowledge and data, and each edge is assigned a clinical dependency direction. This direction reflects the direction of information flow during the clinical decision-making process, for example, from a symptom node to a disease diagnosis node, indicating that symptom information is dependent on disease diagnosis.

[0162] During the forward propagation process, following the clinical dependency direction, the source node packages its own modality-compatible features and association confidence parameters into a feature packet and transmits it to the target node. Upon receiving the feature packet, the target node performs a weighted integration of the feature packets transmitted from different source nodes based on the association confidence parameters. This weighted integration method can reasonably fuse information from different source nodes according to the tightness of the association between nodes, making the feature representation of the target node richer and more accurate.

[0163] In the reverse feedback process, after completing feature integration, the target node analyzes the discrepancies generated during the integration process. These discrepancies reflect changes in the target node's own features after receiving features from the source node, as well as potential information conflicts or additions. The target node feeds this discrepancy information back to the source node along the inverse dependency direction, and the source node adjusts the expression dimension of its modality-compatible features based on the received discrepancies. This reverse feedback mechanism promotes information interaction and collaborative optimization between nodes, making the feature representations in the entire dynamic association network more consistent and effective.

[0164] During the forward propagation process, the source node transmits a feature packet, including its own modality compatibility features and associated confidence parameters, to the target node. The target node then performs weighted integration of the received feature packet based on the associated confidence parameters.

[0165] In this embodiment of the disclosure, when preparing to transmit the feature packet, the source node combines its own modality-compatible features (feature vectors after modality-adaptive encoding) with a pre-calculated association confidence parameter. The association confidence parameter is calculated based on various factors such as the clinical relationship and data similarity between the source node and the target node, and it reflects the importance of the source node's features to the target node's feature updates.

[0166] After receiving feature packets from multiple source nodes, the target node performs a weighted summation of the modality-compatible features in each feature packet based on the corresponding association confidence parameters. Through this weighted integration method, the target node can comprehensively consider the influence of different source nodes, highlighting the features of source nodes closely related to itself, thereby generating feature representations that are more consistent with clinical practice and data relationships.

[0167] During the reverse feedback process, the target node transmits the difference information generated during the feature integration process to the source node, and the source node adjusts the expression dimension of its modality-compatible features based on the corresponding difference information.

[0168] In this embodiment, after the target node completes the weighted integration of features through forward propagation, it calculates the difference between the integrated features and its original features. This difference can be measured in various ways, such as calculating the Euclidean distance or cosine similarity between feature vectors. The difference information reflects the changes in the target node's own features after receiving features from the source node, as well as any potential information mismatches or additions.

[0169] The target node feeds back this discrepancy information to the corresponding source node along the inverse dependency direction. Upon receiving this discrepancy information, the source node adjusts the expression dimensions of its modality-compatible features accordingly. For example, if the discrepancy information indicates a significant change in a certain feature dimension of the target node, it means that the source node's feature in that dimension has a greater impact on the target node, and the source node can appropriately enhance the expression of that feature dimension; conversely, if the discrepancy information shows that a certain dimension has a smaller impact on the target node, the source node can weaken the expression of that feature dimension. Through this adjustment, the source node can better adapt to the feature update needs of the target node, improving the efficiency and accuracy of feature interactions in the entire dynamic association network.

[0170] Based on the node's hierarchical position in the dynamic association network, multiple rounds of feature interaction processing are performed. Nodes at the upstream level of clinical decision-making complete feature updates first and then transmit the updated features to downstream nodes.

[0171] In this embodiment of the disclosure, in the dynamic relational network, nodes are divided into different levels according to the process and logic of clinical decision-making. For example, in a disease diagnosis scenario, symptom nodes may be at the upstream level, disease diagnosis nodes at the midstream level, and treatment plan nodes at the downstream level. This hierarchical division reflects the flow order and dependencies of information in clinical decision-making.

[0172] In the multi-round feature interaction processing, features are updated preferentially for nodes at the upstream level. After the upstream nodes update their features, they propagate the updated features to downstream nodes along the clinical dependency direction. Downstream nodes, upon receiving the features transmitted from upstream nodes, further update and integrate them with their existing features. This hierarchical processing method simulates the step-by-step process of clinical decision-making, making feature interactions more consistent with actual clinical scenarios. It ensures that the feature updates of downstream nodes fully consider the information from upstream nodes, thereby improving the overall model's decision-making accuracy and reliability.

[0173] After each round of feature interaction, the association confidence parameters between nodes are recalculated, and the weight allocation of information transmission is dynamically adjusted.

[0174] In this embodiment of the disclosure, the feature relationships between nodes may change after each round of feature interaction processing. For example, feature updates of some nodes may cause changes in the degree of association between them and other nodes. To adapt to this change, it is necessary to recalculate the association confidence parameters between nodes.

[0175] Methods for recalculating association confidence parameters can be based on various factors, such as the similarity between node features, updates to clinical knowledge, and historical interaction information. For example, the degree of association between two node feature vectors can be initially assessed by calculating the cosine similarity, and then the similarity value can be adjusted by combining prior clinical knowledge and historical interaction data to obtain the final association confidence parameter.

[0176] By dynamically adjusting the association confidence parameter, the changes in the association relationship between nodes can be reflected in real time. In subsequent feature interaction processes, the priority of information transmission can be allocated according to the new association weight, so that information can flow and merge more effectively in the dynamic association network, thereby improving the quality and efficiency of feature interaction.

[0177] In response to a feature update magnitude lower than a preset magnitude threshold, hierarchical feature interaction processing is stopped, and a cross-modal enhanced feature set is formed based on the updated feature representations of all nodes.

[0178] In this embodiment of the disclosure, after each round of feature interaction processing, the feature update magnitude of all nodes in the current round is calculated. As mentioned above, the feature update magnitude can be calculated using various metrics, such as calculating the average Euclidean distance change of the feature vectors of all nodes.

[0179] The calculated feature update magnitude is compared with a preset magnitude threshold. If the feature update magnitude is lower than the preset threshold, it indicates that after multiple rounds of feature interaction, the node's features have stabilized, and continuing feature interaction processing will not significantly improve the feature representation. In this case, hierarchical feature interaction processing is stopped to avoid unnecessary consumption of computational resources and time.

[0180] Finally, the updated feature representations of all nodes are combined to form a cross-modal enhanced feature set. This set contains multimodal data features that have undergone multiple rounds of feature interaction and optimization, enabling it to more comprehensively and accurately reflect the patient's condition information and providing strong support for subsequent clinical decision-making, disease diagnosis, and treatment plan development. By performing hierarchical feature interaction processing on the dynamic association network, information from multimodal data can be effectively integrated to generate a cross-modal enhanced feature set with stronger representational capabilities and clinical value.

[0181] This disclosure also provides a diagnostic device based on AI-driven multimodal medical data fusion, see [link to relevant documentation]. Figure 2The system includes: a first construction module 210, configured to construct multi-source medical data based on anatomical and functional metabolic images transmitted by an image acquisition device under different imaging modalities, continuous waveform signals reflecting the functions of different physiological systems transmitted by a physiological monitoring device, and medical record reports transmitted by an electronic medical record system; a second construction module 220, configured to construct a dynamic association network based on clinical association rules of each data item in the multi-source medical data, wherein nodes in the dynamic association network correspond to data items in the multi-source medical data, and the attributes of the edges include association confidence and clinical dependence direction; a first generation module 230, configured to perform hierarchical feature interaction processing on the dynamic association network, update node feature representations through bidirectional information transmission between nodes, and generate a cross-modal enhanced feature set; and a second generation module 240, configured to input the cross-modal enhanced feature set into a pre-trained diagnostic inference system. The model generates fused diagnostic features through the dual-path feature parsing mechanism of the diagnostic reasoning model; the determination module 250 is configured to determine candidate disease types based on the overall matching degree between the fused diagnostic features and the typical feature spectra of each standardized disease, and to determine the supporting and unsupported feature items of each candidate disease type based on the sub-item matching degree between the fused diagnostic features and the typical feature spectra of each standardized disease; the third construction module 260 is configured to establish a feature tracing chain for diagnostic basis based on the source features traced by the supporting feature items in the cross-modal enhanced feature set, and to construct a feature association graph based on the feature tracing chain and the node association relationship of the dynamic association network; the third generation module 270 is configured to generate and output a diagnostic result including disease type, differential diagnostic basis and feature association graph based on the candidate disease type, the feature tracing chain and the feature association graph.

[0182] Optionally, the second generation module 240 is configured to: input the cross-modal enhanced feature set into the feature allocation layer of the diagnostic reasoning model to obtain the anatomical feature vector and functional feature vector obtained by the reasoning analysis of the diagnostic reasoning model; adopt an adaptive fusion strategy based on clinical weights to perform feature fusion processing on the cross-validated anatomical feature vector and the functional feature vector to obtain a fused feature vector; perform clinical semantic enhancement processing on the fused feature vector through the feature sublimation layer of the diagnostic reasoning model, and map the enhanced fused feature vector to a clinically interpretable feature space in combination with the disease diagnosis criteria to obtain a candidate feature vector; call the decision focusing module of the diagnostic reasoning model to perform key feature enhancement processing on the candidate feature vector to improve the expression intensity of the candidate feature components directly related to the diagnostic decision, and generate a target feature vector; and perform dimensional normalization processing on the target feature vector through the output layer of the diagnostic reasoning model to generate a fused diagnostic feature with clinical decision orientation.

[0183] Optionally, the second generation module 240 is configured to: input the cross-modal enhanced feature set into the feature allocation layer of the diagnostic reasoning model; divide the cross-modal enhanced features into anatomical feature streams and functional feature streams according to the clinical attributes of each cross-modal enhanced feature; process the anatomical feature streams through the anatomical feature parsing path of the diagnostic reasoning model, sequentially performing feature refinement extraction, spatial relationship modeling, and pathological morphology recognition operations to generate anatomical feature vectors; process the functional feature streams through the functional feature parsing path of the diagnostic reasoning model, sequentially performing temporal trend analysis, physiological rhythm extraction, and functional state assessment operations to generate functional feature vectors; input the anatomical feature vectors and the functional feature vectors into the cross-validation layer of the diagnostic reasoning model, verify the clinical consistency of the anatomical feature vectors and the functional feature vectors through a feature mutual verification mechanism, mark the feature components with contradictions, and obtain the anatomical feature vectors and functional feature vectors obtained by the reasoning analysis of the diagnostic reasoning model.

[0184] Optionally, the second generation module 240 is configured to: input the anatomical feature stream into the multi-scale extraction submodule of the anatomical feature parsing path, capture feature information of anatomical structures of different sizes through feature extraction units with different receptive fields, and generate multi-scale anatomical features; based on the clinical importance of the anatomical structures, remove redundant non-critical features from the multi-scale anatomical features to obtain key anatomical features, and input the key anatomical features into the spatial relationship modeling submodule, construct a spatial positional association graph between anatomical structures through a graph structure learning algorithm, wherein the nodes of the spatial positional association graph represent anatomical structures and the edges represent the relative positional relationships between structures; extract the topological features and node attribute features of the spatial positional association graph to generate a spatial relationship feature vector, and input the spatial relationship feature vector into the pathological morphology recognition submodule, detect abnormal morphological patterns in the spatial relationship feature vector through a pre-trained morphological recognition unit, and mark possible pathological morphological regions; perform feature quantization processing on the marked pathological morphological regions, extract the morphological parameters and edge features of the pathological morphological regions, and generate pathological morphological features; fuse the spatial relationship feature vector and the pathological morphological features to obtain the anatomical feature vector.

[0185] Optionally, the second generation module 240 is configured to: input the functional feature stream into the time-series processing submodule of the functional feature parsing path; the time-series processing submodule sorts the features in the functional feature stream according to the timestamp order to construct a time-series feature sequence; capture the changing trends of each feature in the time-series feature sequence in different time intervals using a sliding window analysis method to generate trend feature components; input the trend feature components into the physiological rhythm recognition submodule; extract the periodic rhythm pattern of the trend feature components using a spectrum analysis algorithm to determine the periodic and amplitude features of the rhythm; construct a physiological rhythm model based on the periodic and amplitude features to describe the regularity of functional features changing over time; compare and analyze the physiological rhythm model with a preset normal physiological rhythm template to calculate the rhythm deviation parameter; input the rhythm deviation parameter into the functional state assessment submodule; generate functional state level features by combining clinical functional assessment standards; and fuse the trend feature components, the rhythm deviation parameter, and the functional state level features to obtain the functional feature vector.

[0186] Optionally, the second generation module 240 is configured to: extract key anatomical feature components and corresponding anatomical clinical significance descriptions from the anatomical feature vector, and extract key functional feature components and corresponding functional clinical significance descriptions from the functional feature vector; establish an expected correlation between the key anatomical feature components and the key functional feature components based on clinical diagnostic logic, combined with the anatomical clinical significance descriptions and the functional clinical significance descriptions; compare the actual key anatomical feature components and key functional feature components one by one to see if they conform to the expected correlation, and record contradictory feature pairs that do not conform to the correlation; quantify the degree of contradiction for the contradictory feature pairs, and calculate a contradiction index based on the probability and degree of influence of the contradiction in clinical diagnosis; mark the feature components whose contradiction index exceeds a preset contradiction index threshold to mark the contradictory feature components, and obtain contradiction markers; and obtain the anatomical feature vector and functional feature vector obtained by the diagnostic reasoning model based on the anatomical feature vector, the functional feature vector, and the corresponding contradiction markers.

[0187] Optionally, the second construction module 220 is configured to: extract and parse the anatomical site identifier, physiological indicator name, and disease description keywords corresponding to each data item to obtain the clinical semantic information of each data item in the multi-source medical data; introduce a medical domain knowledge graph to perform standardized mapping processing on the clinical semantic information, unifying the semantic information of different data items to a standard medical terminology system to obtain standardized clinical semantic information; based on the standardized clinical semantic information, search a preset clinical association rule base to determine the clinical association type between any two data items, wherein the clinical association type includes at least one of causal association, concomitant association, and exclusion association; and, based on the clinical association type and the frequency of citation of the clinical association type in clinical guidelines and the level of evidence-based medicine, determine the clinical association type between any two data items. The process involves determining the association confidence parameters between data items; establishing directed associations based on the disease progression timeline and examination process logic to obtain the clinical dependency direction between data items; initializing the topology of the dynamic association network by using each data item in the multi-source medical data as a node, and using the corresponding association confidence parameters and clinical dependency direction as attributes of the edges between nodes; dynamically adjusting the attribute values ​​of the edges based on preset clinical scenario parameters, and improving the association confidence parameters and strengthening the constraint of the dependency direction for key associations in acute and critical illness scenarios; optimizing the structure of the initialized dynamic association network using a network evolution algorithm, and pruning low-confidence association edges according to the clinical importance of the nodes to form the dynamic association network.

[0188] Optionally, the first generation module 230 is configured to: extract basic features corresponding to each node in the dynamic association network, the basic features including anatomical morphological features represented by anatomical structure images, functional morphological features represented by functional metabolic images, functional trend features in continuous waveform signals, and clinical semantic features carried by case reports; perform modality adaptation encoding processing on the basic features of each node, transforming the basic features of different modalities into a unified feature space to generate modality-compatible features; determine the information transmission order between nodes based on the clinical dependency direction of the edges in the dynamic association network, and transmit node features forward along the clinical dependency direction and feedback feature update information backward along the inverse dependency direction; during the forward transmission process, the source node transmits its own modality-compatible features and association confidence parameters to the target node. The target node performs weighted integration of the received feature packets based on the association confidence parameter. During the reverse feedback process, the target node transmits the difference information generated during the feature integration process to the source node, and the source node adjusts the expression dimension of its own modality-compatible features based on the corresponding difference information. Multiple rounds of feature interaction processing are performed according to the hierarchical position of the nodes in the dynamic association network. Nodes at the upstream level of clinical decision-making complete feature updates first and transmit the updated features to downstream nodes. After each round of feature interaction, the association confidence parameter between nodes is recalculated, and the weight allocation of information transmission is dynamically adjusted. In response to the feature update magnitude being lower than a preset magnitude threshold, the hierarchical feature interaction processing is stopped, and a cross-modal enhanced feature set is formed based on the updated feature representations of all nodes.

[0189] This disclosure also provides an electronic device, including: a processor and a machine-readable storage medium connected to the processor; the machine-readable storage medium is used to store programs, instructions, or code; the processor is used to execute the programs, instructions, or code in the machine-readable storage medium to implement the AI-based multimodal medical data fusion diagnostic method described in any of the foregoing embodiments.

[0190] Figure 3 The AI-based multimodal medical data fusion diagnostic device 100 shown includes a processor 1001 and a memory 1003. The processor 1001 and memory 1003 are connected, for example, via a bus 1002. Optionally, the AI-based multimodal medical data fusion diagnostic device 100 may further include a communication component 1004, which can be used for data interaction between the device 100 and other devices, such as data transmission and / or data reception. It should be noted that in actual scheduling, the communication component 1004 is not limited to one, and the structure of this AI-based multimodal medical data fusion diagnostic device 100 does not constitute a limitation on the embodiments of this application.

[0191] Processor 1001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 1001 may also be a combination that implements computing functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0192] Bus 1002 may include a pathway for transmitting information between the aforementioned components. Bus 1002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 1002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0193] The memory 1003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing program code and capable of being read by a computer, without limitation herein.

[0194] The memory 1003 is used to store program code for executing embodiments of the present disclosure, and its execution is controlled by the processor 1001. The processor 1001 is used to execute the program code stored in the memory 1003 to implement the steps shown in the foregoing embodiments of the AI-based multimodal medical data fusion diagnostic method.

[0195] This disclosure also provides a computer-readable storage medium storing program code. When the program code is executed by a processor, it can implement the steps and corresponding content of the aforementioned AI-based multimodal medical data fusion diagnostic method embodiment.

[0196] The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings. However, the present disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of the present disclosure, various changes, modifications, substitutions and variations can be made to these embodiments, and all such changes, modifications, substitutions and variations fall within the protection scope of the present disclosure.

[0197] It should also be noted that the various specific technical features described in the above embodiments can be combined in any suitable manner without contradiction, and such combinations should also be considered as part of this disclosure. To avoid unnecessary repetition, this disclosure will not further describe the various possible combinations. The technical scope of this application is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. A diagnostic method based on AI-driven multimodal medical data fusion, characterized in that, include: Multi-source medical data is constructed based on anatomical and functional metabolic images transmitted by imaging acquisition devices in different imaging modalities, continuous waveform signals transmitted by physiological monitoring devices to reflect the functions of different physiological systems, and medical record reports transmitted by electronic medical record systems. Based on the clinical association rules of each data item in the multi-source medical data, a dynamic association network is constructed, wherein the nodes in the dynamic association network correspond to the data items in the multi-source medical data, and the attributes of the edges include association confidence and clinical dependency direction. Hierarchical feature interaction processing is performed on the dynamic association network, and the node feature representation is updated through bidirectional information transmission between nodes to generate a cross-modal enhanced feature set; The cross-modal enhanced feature set is input into a pre-trained diagnostic reasoning model, and fused diagnostic features are generated through the dual-path feature parsing mechanism of the diagnostic reasoning model. Based on the overall matching degree between the fusion diagnostic features and the typical feature spectrum of each standardized disease, candidate disease types are determined, and based on the sub-item matching degree between the fusion diagnostic features and the typical feature spectrum of each standardized disease, supporting and unsupported feature items of each candidate disease type are determined. Based on the source features traced back to the cross-modal enhanced feature set by the supporting feature items, a feature tracing chain for diagnostic criteria is established, and a feature association graph is constructed based on the feature tracing chain and the node association relationship of the dynamic association network. Based on the candidate disease type, the feature tracing chain, and the feature association map, a diagnostic result including the disease type, differential diagnosis criteria, and feature association map is generated and output.

2. The diagnostic method based on AI-driven multimodal medical data fusion according to claim 1, characterized in that, The step of inputting the cross-modal enhanced feature set into a pre-trained diagnostic inference model and generating fused diagnostic features through the dual-path feature parsing mechanism of the diagnostic inference model includes: The cross-modal enhanced feature set is input into the feature allocation layer of the diagnostic reasoning model to obtain the anatomical feature vector and functional feature vector obtained by the reasoning analysis of the diagnostic reasoning model; An adaptive fusion strategy based on clinical weights is adopted to perform feature fusion processing on the cross-validated anatomical feature vector and the functional feature vector to obtain a fused feature vector; The fused feature vector is enhanced with clinical semantics through the feature enhancement layer of the diagnostic reasoning model. The enhanced fused feature vector is then mapped to a clinically interpretable feature space in combination with the disease diagnosis criteria to obtain the feature vector to be used. The decision-focusing module of the diagnostic reasoning model is invoked to perform key feature enhancement processing on the candidate feature vector, thereby increasing the expression intensity of the candidate feature components that are directly related to the diagnostic decision and generating the target feature vector. The target feature vector is dimensionally normalized through the output layer of the diagnostic reasoning model to generate fusion diagnostic features with clinical decision-making orientation.

3. The diagnostic method based on AI-driven multimodal medical data fusion according to claim 2, characterized in that, The step of inputting the cross-modal enhanced feature set into the feature allocation layer of the diagnostic inference model to obtain the anatomical feature vector and functional feature vector obtained by the inference analysis of the diagnostic inference model includes: The cross-modal enhancement feature set is input into the feature allocation layer of the diagnostic reasoning model, and the cross-modal enhancement features are divided into anatomical feature stream and functional feature stream according to the clinical attributes of each cross-modal enhancement feature; The anatomical feature flow is processed through the anatomical feature parsing path of the diagnostic reasoning model, and feature refinement extraction, spatial relationship modeling and pathological morphology recognition operations are performed in sequence to generate anatomical feature vectors. The functional feature flow is processed through the functional feature parsing path of the diagnostic reasoning model, and time-series trend analysis, physiological rhythm extraction and functional state assessment are performed in sequence to generate functional feature vectors. The anatomical feature vector and the functional feature vector are input into the cross-validation layer of the diagnostic reasoning model. The clinical consistency between the anatomical feature vector and the functional feature vector is tested through the feature mutual verification mechanism. The feature components with contradictions are marked, and the anatomical feature vector and functional feature vector obtained by the reasoning analysis of the diagnostic reasoning model are obtained.

4. The diagnostic method based on AI-driven multimodal medical data fusion according to claim 3, characterized in that, The anatomical feature flow is processed through the anatomical feature parsing path of the diagnostic reasoning model, sequentially performing feature refinement extraction, spatial relationship modeling, and pathological morphology recognition operations to generate anatomical feature vectors, including: The anatomical feature stream is input into the multi-scale extraction submodule of the anatomical feature parsing path, and feature information of anatomical structures of different sizes is captured by feature extraction units with different receptive fields to generate multi-scale anatomical features. Based on the clinical importance of anatomical structures, redundant non-critical features in the multi-scale anatomical features are removed to obtain key anatomical features. These key anatomical features are then input into the spatial relationship modeling submodule, and a spatial position association graph between anatomical structures is constructed using a graph structure learning algorithm. The nodes in the spatial position association graph represent anatomical structures, and the edges represent the relative positional relationships between the structures. Extract the topological features and node attribute features of the spatial location association graph to generate a spatial relationship feature vector, and input the spatial relationship feature vector into the pathological morphology recognition submodule. The abnormal morphological patterns in the spatial relationship feature vector are detected by the pre-trained morphological recognition unit, and possible pathological morphological regions are marked. The marked pathological morphological regions are subjected to feature quantization processing to extract the morphological parameters and edge features of the pathological morphological regions and generate pathological morphological features. The spatial relationship feature vector and the pathological morphological features are combined to obtain the anatomical feature vector.

5. The diagnostic method based on AI-driven multimodal medical data fusion according to claim 3, characterized in that, The functional feature stream is processed through the functional feature parsing path of the diagnostic reasoning model, sequentially performing time-series trend analysis, physiological rhythm extraction, and functional state assessment operations to generate functional feature vectors, including: The functional feature stream is input into the time-series processing submodule of the functional feature parsing path. The time-series processing submodule sorts the features in the functional feature stream according to the timestamp order to construct a time-series feature sequence. The sliding window analysis method is used to capture the changing trends of each feature in the time series within different time intervals, generate trend feature components, and input the trend feature components into the physiological rhythm recognition submodule. The periodic rhythm pattern of the trend feature components is extracted by the spectrum analysis algorithm to determine the periodic and amplitude characteristics of the rhythm. Based on the periodic and amplitude characteristics, a physiological rhythm model is constructed to describe the regularity of functional characteristics changing over time. The physiological rhythm model is then compared and analyzed with a preset normal physiological rhythm template to calculate the rhythm deviation parameter. The rhythm deviation parameters are input into the functional status assessment submodule, and functional status level characteristics are generated by combining them with clinical functional assessment standards. The functional feature vector is obtained by integrating the trend feature components, the rhythm deviation parameters, and the functional state level features.

6. The diagnostic method based on AI-driven multimodal medical data fusion according to claim 3, characterized in that, The process involves inputting the anatomical feature vector and the functional feature vector into the cross-validation layer of the diagnostic reasoning model. A feature verification mechanism is used to test the clinical consistency between the anatomical feature vector and the functional feature vector, marking contradictory feature components, and obtaining the anatomical feature vector and functional feature vector obtained from the diagnostic reasoning model's inference analysis. This includes: Extract the key anatomical feature components and their corresponding anatomical clinical significance from the anatomical feature vector, and extract the key functional feature components and their corresponding functional clinical significance from the functional feature vector; Based on clinical diagnostic logic, and combining the descriptions of anatomical clinical significance and functional clinical significance, the expected correlation between the key anatomical feature components and the key functional feature components is established. Compare the actual key anatomical feature components with the key functional feature components one by one to see if they meet the expected correlation, and record the contradictory feature pairs that do not meet the correlation. The contradictory features are quantified to determine their degree of contradiction, and a contradiction index is calculated based on the probability and impact of the contradiction in clinical diagnosis. The feature components whose contradiction index exceeds a preset contradiction index threshold of the contradictory feature pair are marked to mark the feature components that are contradictory, thus obtaining contradiction marks; Based on the anatomical feature vector, the functional feature vector, and the corresponding contradiction markers, the anatomical feature vector and functional feature vector obtained by the diagnostic reasoning model are obtained.

7. The diagnostic method based on AI-driven multimodal medical data fusion according to any one of claims 1-6, characterized in that, The construction of a dynamic association network based on clinical association rules for each data item in the multi-source medical data includes: Extract and parse the anatomical site identifiers, physiological indicator names, and disease description keywords corresponding to each data item to obtain the clinical semantic information of each data item in the multi-source medical data; By introducing a medical knowledge graph to standardize and map clinical semantic information, the semantic information of different data items is unified into a standard medical terminology system, resulting in standardized clinical semantic information. Based on standardized clinical semantic information, a preset clinical association rule base is retrieved to determine the clinical association type between any two data items. The clinical association type includes at least one of causal association, concomitant association, and exclusion association. Based on the clinical association type, the frequency of citation of the clinical association type in clinical guidelines, and the level of evidence-based medicine, the association confidence parameters between the calculated data items are determined. Based on the disease progression sequence and examination process logic, a directed correlation is formed, and the clinical dependency direction between data items is obtained. Each data item in the multi-source medical data is used as a node in the dynamic association network, and the corresponding association confidence parameter and the clinical dependence direction are used as attributes of the edges between nodes to initialize the topology of the dynamic association network. Based on preset clinical scenario parameters, the attribute values ​​of the edges are dynamically adjusted to improve the association confidence parameters and strengthen the constraint of the dependency direction for key associations in critical care scenarios. The dynamic association network is structurally optimized by using a network evolution algorithm, and low-confidence association edges are pruned according to the clinical importance of nodes to form the dynamic association network.

8. The diagnostic method based on AI-driven multimodal medical data fusion according to any one of claims 1-6, characterized in that, The hierarchical feature interaction processing performed on the dynamic association network updates the node feature representations through bidirectional information transmission between nodes, generating a cross-modal enhanced feature set, including: Extract the basic features corresponding to each node in the dynamic association network. The basic features include anatomical morphological features represented by anatomical structure images, functional morphological features represented by functional metabolic images, functional trend features in continuous waveform signals, and clinical semantic features carried by case reports. Modality adaptation encoding is performed on the basic features of each node to transform the basic features of different modalities into a unified feature space, generating modality-compatible features. Based on the clinical dependence direction of the edges in the dynamic association network, the information transmission order between nodes is determined, and node features are transmitted forward along the clinical dependence direction and feature updates are fed back in the reverse dependence direction. During the forward propagation process, the source node transmits a feature packet, including its own modality compatibility features and associated confidence parameters, to the target node. The target node then performs weighted integration of the received feature packet based on the associated confidence parameters. During the reverse feedback process, the target node transmits the difference information generated during the feature integration process to the source node, and the source node adjusts the expression dimension of its own modality-compatible features based on the corresponding difference information. Based on the hierarchical position of the node in the dynamic association network, multiple rounds of feature interaction processing are performed. Nodes at the upstream level of clinical decision-making complete feature updates first and transmit the updated features to downstream nodes. After each round of feature interaction, the association confidence parameters between nodes are recalculated, and the weight allocation of information transmission is dynamically adjusted. In response to a feature update magnitude lower than a preset magnitude threshold, hierarchical feature interaction processing is stopped, and a cross-modal enhanced feature set is formed based on the updated feature representations of all nodes.

9. A diagnostic device based on AI-driven multimodal medical data fusion, characterized in that, include: The first building module is configured to construct multi-source medical data based on anatomical and functional metabolic images transmitted by the image acquisition device in different imaging modalities, continuous waveform signals transmitted by the physiological monitoring device to reflect the functions of different physiological systems, and medical record reports transmitted by the electronic medical record system. The second construction module is configured to construct a dynamic association network based on the clinical association rules of each data item in the multi-source medical data. The nodes in the dynamic association network correspond to the data items in the multi-source medical data, and the attributes of the edges include association confidence and clinical dependency direction. The first generation module is configured to perform hierarchical feature interaction processing on the dynamic association network, update the node feature representation through bidirectional information transmission between nodes, and generate a cross-modal enhanced feature set. The second generation module is configured to input the cross-modal enhanced feature set into a pre-trained diagnostic inference model and generate fused diagnostic features through the dual-path feature parsing mechanism of the diagnostic inference model. The determination module is configured to determine candidate disease types based on the overall matching degree between the fused diagnostic features and the typical feature spectrum of each standardized disease, and to determine the supporting and unsupported feature items of each candidate disease type based on the sub-item matching degree between the fused diagnostic features and the typical feature spectrum of each standardized disease. The third construction module is configured to establish a feature tracing chain for diagnostic criteria based on the source features traced back to the cross-modal enhanced feature set by the supported feature items, and to construct a feature association graph based on the feature tracing chain and the node association relationship of the dynamic association network. The third generation module is configured to generate and output a diagnostic result including the disease type, differential diagnosis basis, and feature association map based on the candidate disease type, the feature tracing chain, and the feature association map.

10. An electronic device, characterized in that, include: A processor and a machine-readable storage medium, wherein the machine-readable storage medium is connected to the processor; The machine-readable storage medium is used to store programs, instructions, or code; The processor is configured to execute the program, instructions, or code in the machine-readable storage medium to implement the AI-based multimodal medical data fusion diagnostic method according to any one of claims 1-8.

Images