Systemic lupus erythematosus assessment method and device based on multi-modal medical map and medical knowledge fusion, terminal equipment and medium

Abstract

The invention discloses a systemic lupus erythematosus assessment method and device based on multi-modal medical map and medical knowledge fusion, terminal equipment and a medium, and relates to the technical field of medical artificial intelligence. The method comprises the following steps: acquiring multi-dimensional medical indexes of a patient, and constructing a multi-modal medical knowledge graph of the patient associated with the indexes; pre-training the self-supervised graph neural network to obtain a patient and medical index embedding vector; grouping and constructing a feature subset combination according to medical knowledge, and training and selecting an optimal sub-model; dynamically weighting and fusing the prediction probabilities of the sub-models to obtain a comprehensive prediction probability; and screening the key indexes based on the embedded vector and quantifying the activity evaluation contribution degree of the key indexes to obtain an activity comprehensive score. The systemic lupus erythematosus evaluation method improves the accuracy and robustness of preliminary evaluation of systemic lupus erythematosus, solves the problems of data missing and sample imbalance, and is explainable in output evaluation, adaptive to primary medical treatment and capable of supporting hierarchical diagnosis and treatment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical artificial intelligence, and in particular to a systemic lupus erythematosus evaluation method and device based on multi-modal medical atlas and medical knowledge fusion, a terminal device and a medium. BACKGROUND

[0002] Systemic lupus erythematosus (SLE) is a complex and highly heterogeneous chronic autoimmune disease, and its evaluation relies on rheumatology and immunology specialists to combine dominant symptoms, laboratory indicators and international classification standards for judgment, but the traditional mode has limitations. Specifically, the evaluation standard is highly subjective, and it is easy to make inaccurate evaluations for early and atypical patients. Moreover, the quality of the evaluation results is highly dependent on the experience of experts, and it is difficult for primary hospitals to popularize due to limited resources. The evaluation of SLE based on international standards is mostly based on Caucasian population data, and lacks localization adaptation for Chinese population.

[0003] When a patient is diagnosed with SLE, the current SLEDAI (Systemic Lupus Erythematosus Disease Activity Index) scoring tool for evaluating activity also has obvious deficiencies. Specifically, this scoring tool relies on the subjective judgment of doctors, and the consistency of non-quantitative symptom scoring is poor. Moreover, the scoring focuses on organ damage, and does not fully cover non-specific symptoms such as fatigue and mental debilitation. In addition, this scoring is only suitable for short-term evaluation and lacks long-term prognosis prediction ability, and the evaluation dimension is single, without integrating complications, quality of life and other factors, and the population applicability is questionable.

[0004] Although there have been explorations of existing AI-assisted technologies, they still face various problems. Specifically, the data quality of patients is low, the samples are imbalanced, the model is prone to overfitting, and traditional models are difficult to capture high-dimensional feature nonlinear relationships, multi-modal information fusion is insufficient, and dynamic modeling is lacking. In addition, the model has poor interpretability, lacks clinical trust, and has not effectively integrated medical knowledge, and it is difficult to integrate into clinical systems.

[0005] Therefore, there is an urgent need for an accurate and robust evaluation method that adapts to primary care, to address data missing, sample imbalance, and output uninterpretable problems. SUMMARY

[0006] The technical problem to be solved by the present application is that in the field of systemic lupus erythematosus evaluation, the existing technology is difficult to guarantee the accuracy and robustness of the evaluation, is also easily affected by data missing and sample imbalance problems, and the evaluation results lack interpretability, and cannot be well adapted to the needs of primary medical scenarios, making it difficult to effectively support graded diagnosis and treatment, and there is an urgent need for an effective solution.

[0007] To solve the above technical problems, the technical solutions adopted by the present application are as follows: In a first aspect, the present application provides a method for evaluating systemic lupus erythematosus based on multi-modal medical atlas and medical knowledge fusion, the method comprising: Obtaining multi-dimensional medical indicators of patients, and taking the patients and the medical indicators as nodes of a graph structure respectively, establishing the association between the patients and the medical indicators, and obtaining a multi-modal medical knowledge graph; Based on the multi-modal medical knowledge graph, a graph neural network model for self-supervised learning is constructed and pre-trained to obtain patient embedding vectors and medical indicator embedding vectors, wherein the patient embedding vectors are low-dimensional dense vectors representing information of all medical indicators corresponding to a patient, and the medical indicator embedding vectors are low-dimensional dense vectors representing information of all patients corresponding to a medical indicator; Based on medical knowledge, the multi-dimensional medical indicators are divided into several modal groups, several feature subset combinations are constructed based on the modal groups, and for each feature subset combination, several sub-models are trained and an optimal sub-model is selected, wherein each feature subset combination contains several medical indicators, and the sub-model is used to predict the binary classification prediction probability of systemic lupus erythematosus through the corresponding feature subset combination; Based on the medical knowledge and the optimal sub-model corresponding to each feature subset combination, a dynamic weighted fusion module is constructed, and through the dynamic weighted fusion module, the binary classification prediction probabilities corresponding to all feature subset combinations are fused to obtain the comprehensive prediction probability of systemic lupus erythematosus; Based on the patient embedding vectors and the medical indicator embedding vectors, several key medical indicators are screened and the activity evaluation contribution of the key medical indicators is quantified, and based on the key medical indicators of patients and the corresponding activity evaluation contribution, the activity comprehensive score of systemic lupus erythematosus is obtained.

[0008] In an implementation manner, the medical indicators include hematological indicators, immunological indicators, kidney function indicators, liver function indicators, and imaging features, and the obtaining of the multi-dimensional medical indicators of the patients and the taking of the patients and the medical indicators as the nodes of the graph structure respectively to establish the association between the patients and the medical indicators to obtain the multi-modal medical knowledge graph comprises: Each patient is taken as an independent node, each medical indicator is taken as an independent node, each patient and the corresponding each medical indicator are taken as an edge, and the standardized value of the measured value of the patient and the corresponding medical indicator is taken as the edge weight, and a patient and medical indicator bipartite graph is constructed.

[0009] In an implementation manner, the multi-modal medical knowledge graph is used to construct a self-supervised learning graph neural network model and pre-train the self-supervised learning graph neural network model to obtain patient embedding vectors and medical index embedding vectors, wherein the patient embedding vector is a low-dimensional dense vector representing information of all medical indexes corresponding to a patient, and the medical index embedding vector is a low-dimensional dense vector representing information of all patients corresponding to a medical index, and the method comprises the following steps: randomly masking patient nodes or index nodes in the multi-modal medical knowledge graph with a preset probability; constructing a self-supervised learning graph neural network model, and predicting information of a masked node by using the self-supervised learning graph neural network model to train the self-supervised learning graph neural network model with the information of the masked node as a target, wherein the information of the masked node includes existence and global attributes of the masked node; generating patient embedding vectors and medical index embedding vectors by using the trained self-supervised learning graph neural network model.

[0010] In an implementation manner, the multi-modal medical knowledge graph is used to construct a self-supervised learning graph neural network model and pre-train the self-supervised learning graph neural network model to obtain patient embedding vectors and medical index embedding vectors, wherein the patient embedding vector is a low-dimensional dense vector representing information of all medical indexes corresponding to a patient, and the medical index embedding vector is a low-dimensional dense vector representing information of all patients corresponding to a medical index, and the method comprises the following steps: dividing the medical indexes into five modal groups based on medical knowledge, wherein the five modal groups include a hematology group, an immunology group, a kidney function group, a liver function group, and an imaging group; constructing a non-empty power set of the five modal groups to obtain a plurality of feature subset combinations; using the self-supervised learning graph neural network model to predict and fill in missing values of the medical indexes in each feature subset combination to obtain a filled feature subset combination; performing minority class oversampling on the filled feature subset combination to obtain a plurality of synthetic samples for expanding patient samples; training a plurality of different types of binary classification models for each feature subset combination, wherein the binary classification models include a support vector machine, a K-nearest neighbor algorithm, a random forest, an extreme gradient boosting algorithm, and a multi-layer perceptron; using grid search and 5-fold cross-validation to select an optimal model for each feature subset combination from the trained plurality of different types of binary classification models, wherein the average AUC-ROC value of cross-validation is used as a criterion for evaluating model performance.

[0011] In an implementation, the synthesizing the padded feature subset combinations with minority class oversampling generates a plurality of synthetic samples for augmenting the patient samples, including: In the minority class SLE positive patient samples, for each sample, a plurality of neighbors are randomly selected from its k-nearest neighbors; In the feature space, for each selected minority class sample, a point is randomly selected on the line between the sample and a randomly selected neighbor to generate a new synthetic sample, wherein the synthetic sample is a linear combination of the original sample features; The generated series of synthetic samples are added to the training data set of the binary classification model, wherein the synthetic samples statistically approximate the feature vectors of real patients.

[0012] In an implementation, the constructing a dynamic weighted fusion module based on the medical knowledge and the optimal sub-models corresponding to the feature subset combinations, and fusing the binary classification prediction probabilities of all feature subset combinations corresponding to the optimal sub-models through the dynamic weighted fusion module to obtain the comprehensive prediction probability of SLE, including: Based on medical knowledge, the medical importance scores of each modality group are calibrated, and the medical importance scores are dynamically adjusted based on the clinical status of the patient; Based on the AUC-ROC values of each optimal sub-model, a performance weight is calibrated, wherein the performance weight is proportional to the AUC-ROC value of the optimal sub-model; Based on the number of medical indicators contained in the feature subset combination corresponding to each optimal sub-model, a complexity weight is calibrated, wherein the complexity weight is proportional to the number of medical indicators contained in the feature subset combination; Based on the average value of the medical importance scores of the modality groups contained in the feature subset combination corresponding to each optimal sub-model, a knowledge weight is calibrated, wherein the knowledge weight is proportional to the average value of the medical importance scores; The performance weight, the complexity weight, and the knowledge weight are integrated to obtain a dynamic weight; Based on the dynamic weight, the binary classification prediction probabilities of SLE output by all optimal sub-models are weighted and averaged to obtain the comprehensive prediction probability of SLE.

[0013] In an implementation, the screening a plurality of key medical indicators based on the patient embedding vector and the medical indicator embedding vector, and quantifying the activity assessment contribution degree of the key medical indicators, and obtaining the activity comprehensive score of SLE based on the key medical indicators of the patient and the corresponding activity assessment contribution degrees, including: For each patient embedding vector, the cosine similarity between the patient embedding vector and all medical indicator embedding vectors is calculated, and the three medical indicator embedding vectors with the highest cosine similarity are selected as the medical indicator embedding vectors of the key medical indicators; The medical indicator embedding vectors of the key medical indicators selected for each patient are recorded in a global list, and the occurrence frequency of each medical indicator embedding vector recorded in the global list is counted, wherein the occurrence frequency is used as the activity evaluation contribution degree of the medical indicator; Based on the occurrence frequency, the probability distribution of each medical indicator in the patient space is calculated to obtain a global probability distribution; For each patient, based on the cosine similarity between the patient embedding vector and all medical indicator embedding vectors, and the global probability distribution, the sum of the expected values is calculated, and linear normalization is performed to obtain the activity comprehensive score of systemic lupus erythematosus.

[0014] In a second aspect, the embodiments of the present application also provide a systemic lupus erythematosus evaluation device based on multi-modal medical atlas and medical knowledge fusion, the device comprising: A multi-modal medical knowledge graph construction module is configured to obtain medical indicators of a patient in multiple dimensions, and establish an association relationship between the patient and the medical indicators by taking the patient and the medical indicators as nodes of a graph structure, to obtain a multi-modal medical knowledge graph. A self-supervised learning graph neural network model construction and training module is configured to construct a self-supervised learning graph neural network model and pre-train the model based on the multi-modal medical knowledge graph, to obtain patient embedding vectors and medical indicator embedding vectors, wherein the patient embedding vectors are low-dimensional dense vectors representing information of all medical indicators corresponding to a patient, and the medical indicator embedding vectors are low-dimensional dense vectors representing information of all patients corresponding to a medical indicator. An optimal sub-model construction module is configured to divide the medical indicators in multiple dimensions into a plurality of modal groups based on medical knowledge, construct a plurality of feature subset combinations based on the modal groups, and for each feature subset combination, train a plurality of sub-models and select an optimal sub-model, wherein each feature subset combination includes a plurality of medical indicators, and the sub-model is configured to predict a binary classification prediction probability of systemic lupus erythematosus through the corresponding feature subset combination. A comprehensive prediction probability calculation module is configured to construct a dynamic weighted fusion module based on medical knowledge and the optimal sub-model corresponding to each feature subset combination, and fuse the binary classification prediction probabilities corresponding to all feature subset combinations through the dynamic weighted fusion module to obtain a comprehensive prediction probability of systemic lupus erythematosus. The activity evaluation contribution degree calculation module is configured to filter a plurality of key medical indexes and quantify activity evaluation contribution degrees of the key medical indexes based on the patient embedding vector and the medical index embedding vector, and obtain a comprehensive activity score of systemic lupus erythematosus based on the key medical indexes of the patient and the corresponding activity evaluation contribution degrees.

[0015] In a third aspect, an embodiment of the present application further provides a terminal device, which comprises a memory, a processor, and a systemic lupus erythematosus evaluation program based on fusion of multi-modal medical atlas and medical knowledge stored in the memory and executable on the processor. When the processor executes the systemic lupus erythematosus evaluation program based on fusion of multi-modal medical atlas and medical knowledge, the steps of the systemic lupus erythematosus evaluation method based on fusion of multi-modal medical atlas and medical knowledge in any of the above solutions are implemented.

[0016] In a fourth aspect, an embodiment of the present application further provides a computer readable storage medium, which stores a systemic lupus erythematosus evaluation program based on fusion of multi-modal medical atlas and medical knowledge. When a processor executes the systemic lupus erythematosus evaluation program based on fusion of multi-modal medical atlas and medical knowledge, the steps of the systemic lupus erythematosus evaluation method based on fusion of multi-modal medical atlas and medical knowledge in any of the above solutions are implemented.

[0017] Beneficial effects: The application discloses a systemic lupus erythematosus evaluation method and device based on multi-modal medical atlas and medical knowledge fusion, a terminal equipment and a medium. The method first acquires medical indexes of a patient in multiple dimensions, and takes the patient and the medical indexes as nodes of a graph structure respectively, establishes the correlation between the patient and the medical indexes, and obtains a multi-modal medical knowledge graph. Then, based on the multi-modal medical knowledge graph, a graph neural network model of self-supervised learning is constructed and pre-trained to obtain patient embedding vectors and medical index embedding vectors. The patient embedding vectors are low-dimensional dense vectors representing information of all medical indexes corresponding to a patient, and the medical index embedding vectors are low-dimensional dense vectors representing information of all patients corresponding to a medical index. Next, the medical indexes in multiple dimensions are divided into several modal groups based on medical knowledge, several feature subset combinations are constructed based on the modal groups, and for each feature subset combination, several sub-models are trained and an optimal sub-model is selected. Each feature subset combination contains several medical indexes, and the sub-model is used to predict the binary classification prediction probability of systemic lupus erythematosus through the corresponding feature subset combination. Then, based on the medical knowledge and the optimal sub-model corresponding to each feature subset combination, a dynamic weighted fusion module is constructed, and the binary classification prediction probability corresponding to all feature subset combinations is fused through the dynamic weighted fusion module to obtain the comprehensive prediction probability of systemic lupus erythematosus. Further, based on the patient embedding vectors and the medical index embedding vectors, several key medical indexes are screened and the activity evaluation contribution degree of the key medical indexes is quantified, and based on the key medical indexes of the patient and the corresponding activity evaluation contribution degree, the activity comprehensive score of systemic lupus erythematosus is obtained. The application improves the accuracy and robustness of systemic lupus erythematosus evaluation, solves the problems of data missing and sample imbalance, the evaluation result is interpretable, is suitable for primary medical care and supports hierarchical diagnosis and treatment, and enhances clinical practicability. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 The flowchart of the specific implementation mode of the systemic lupus erythematosus evaluation method based on multi-modal medical atlas and medical knowledge fusion provided by the embodiment of the application is provided.

[0019] Figure 2 The systemic lupus erythematosus evaluation method based on multi-modal medical atlas and medical knowledge fusion provided by the embodiment of the application is a whole architecture schematic diagram.

[0020] Figure 3 The systemic lupus erythematosus evaluation method based on multi-modal medical atlas and medical knowledge fusion provided by the embodiment of the application is a dynamic weighted fusion schematic diagram.

[0021] Figure 4is a schematic diagram of experimental results of a systemic lupus erythematosus evaluation method based on multi-modal medical atlas and medical knowledge fusion provided by an embodiment of the present application.

[0022] Figure 5 is a principle block diagram of a systemic lupus erythematosus evaluation device based on multi-modal medical atlas and medical knowledge fusion provided by an embodiment of the present application.

[0023] Figure 6 is a principle block diagram of an internal structure of a terminal device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0024] To make the objectives, technical solutions and effects of the present application clearer and more explicit, the present application is further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0025] The flowchart shown in the drawings is only an example and does not necessarily include all contents and operations or steps, nor does it necessarily execute in the order described. For example, some operations or steps can be further divided, combined or partially combined, so the actual execution order may be changed according to the actual situation.

[0026] It should be understood that the terms used in the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the present application specification and the appended claims, unless otherwise clear from the context, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0027] It should be understood that, in order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, the terms "first", "second" and the like are used to distinguish the same or similar items with basically the same function and effect. For example, the first control information and the second control information are only used to distinguish different control information, and do not limit the order.

[0028] Those skilled in the art can understand that the terms "first", "second" and the like do not limit the quantity and execution order, and the terms "first", "second" and the like do not necessarily mean different.

[0029] It should also be understood that the term "and / or" used in the present application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0030] Systemic lupus erythematosus (SLE) is a complex and highly heterogeneous chronic autoimmune disease. Its assessment mainly relies on rheumatology and immunology specialists to comprehensively judge patients' dominant symptoms (such as skin rash, arthritis) and laboratory test indicators (such as autoantibodies, blood routine, urine routine, complement levels, etc.) according to international classification standards, such as the 2019 EULAR / ACR standard or the 2012 SLICC standard. However, this traditional assessment model has the following inherent limitations. First, the assessment standard is highly subjective and lagging, and the current standard contains more subjective judgments, which is more suitable for patients with typical symptoms. For early, atypical or incomplete SLE patients who do not meet the classification criteria, it is easy to miss or misjudge. Second, it relies on the experience of specialists and uneven medical resources, and the quality of assessment is highly dependent on the experience of specialists, and often relies on a combination of specific detection indicators with high cost, which is difficult to popularize in primary hospitals. This leads to difficulties in ensuring the quality of medical services in primary community or regional hospitals with relatively weak resources, exacerbating the differences in medical service quality and hindering the promotion of hierarchical diagnosis and treatment. Third, it lacks local adaptability. The existing international standards are mostly based on Caucasian population data and may not be suitable for the clinical characteristics of Chinese population, lacking local adjustment for Chinese patients.

[0031] In addition, for patients diagnosed with SLE, there are still difficulties in assessing disease activity. The widely used SLEDAI scoring tool also has obvious shortcomings. First, the subjectivity of the score is too high. SLEDAI score relies on the subjective judgment of doctors, especially in assessing non-quantitative symptoms such as fatigue and pain, with large differences in scoring by different doctors and poor consistency. Second, the symptoms covered by the score are not complete. The score focuses on obvious organ damage and pays insufficient attention to non-specific symptoms such as fatigue, mental debilitation, and weight loss that affect quality of life. According to the Pediatric Rheumatology International Trials Organization survey, these symptoms may appear in the early stages of SLE. Therefore, SLEDAI score is difficult to fully reflect the disease manifestations and individual differences. Third, the scoring mechanism lacks a long-term perspective. SLEDAI is mainly used for short-term activity assessment and has limited predictive ability for long-term disease progression and prognosis, while long-term follow-up is crucial for SLE management. Fourth, the assessment dimension is single. The score mainly focuses on disease activity and does not integrate important factors such as complications, combined medication, and quality of life, requiring doctors to make comprehensive judgments based on other information, further increasing the dependence on experience. Fifth, the population applicability is questionable. SLE manifestations vary by race, and the applicability and accuracy of SLEDAI in different populations are still controversial.

[0032] To address the above evaluation challenges, researchers have begun to explore the application of artificial intelligence (AI) technology, especially machine learning, in SLE clinical decision-making. Existing related research mainly includes risk screening and prediction, multi-modal data classification, and image and pathology analysis. Among them, risk screening and prediction based on models such as logistic regression, support vector machine, and random forest for high-risk population screening and prediction; multi-modal data classification integrates multi-modal data (such as hematological, immunological, and biochemical indicators) to build auxiliary screening or subtype classification models; image and pathology analysis applies computer vision technology to analyze images such as dermoscopy images and kidney pathology sections to assist in evaluation.

[0033] Although AI shows certain potential, existing technologies still face multiple problems in clinical translation, limiting their application and promotion. First, the quality and representation of the data are insufficient, and there is a lack of high-quality, large-scale, multi-modal, and longitudinal follow-up SLE clinical data. The positive samples are rare and highly heterogeneous, leading to sample imbalance. Common methods such as random oversampling easily introduce noise, causing model overfitting and poor generalization ability, making it difficult to learn robust disease representations. Second, the model algorithm has limitations and information fragmentation. Existing model algorithms lack complex relationship modeling, and traditional statistics and shallow machine learning models cannot capture deep non-linear relationships between high-dimensional features. Moreover, the multi-modal information fusion of existing model algorithms is insufficient, and multi-modal data (such as blood, immunity, and images) are often simply concatenated as feature vectors, ignoring their potential biological associations and insufficient information fusion. In addition, existing model algorithms lack dynamic modeling processes, with most models being static, unable to effectively utilize longitudinal time series data, and difficult to predict disease activity changes and recurrence risk. Third, there is a lack of explainability and insufficient clinical trust. AI models, especially deep learning, are often black boxes, and the prediction results lack medical logic explanations, making it difficult for doctors to understand and judge the basis, hindering clinical acceptance. Fourth, it is difficult to integrate medical knowledge and system integration. Existing models are mostly data-driven and do not effectively incorporate prior knowledge such as pathophysiological mechanisms and treatment guidelines, making the prediction results potentially unreasonable from a medical logic perspective. In addition, AI systems are often independent of hospital electronic medical records or information systems, making it difficult to embed into clinical workflows and have low practicality.

[0034] Therefore, there is an urgent need for a new technical solution for SLE evaluation to address the lack of effective early screening tools in primary medical institutions, the strong subjectivity and difficulty in standardization of evaluation, and the limitations of existing AI-assisted screening technologies in data processing, algorithm modeling, explainability, and integration with clinical processes, among other technical problems, to meet the comprehensive needs of high precision, strong explainability, local adaptability, and implementability in primary hospitals for SLE evaluation.

[0035] To solve the above technical problems, the present application provides a systemic lupus erythematosus evaluation method based on multi-modal medical atlas and medical knowledge fusion. First, a multi-modal medical atlas and robust feature representation are constructed. The patient and his / her multi-dimensional medical indicators (such as hematology, immunology, liver and kidney function, and imaging features) are represented as nodes in a graph structure, and their internal correlations are represented as edges. A multi-modal medical knowledge graph is constructed. The data comes from electronic medical records, laboratory systems, and imaging systems, ensuring the structured integration of multi-source heterogeneous data. Based on the graph, a self-supervised learning strategy is used for pre-training to extract complex correlation features that are robust to noise, missing, and sample sparsity, significantly improving the generalization ability of the model in real clinical scenarios. Subsequently, a knowledge-guided heterogeneous model dynamic fusion mechanism is innovatively designed. Heterogeneous model ensembles are constructed for different modalities or indicators, and optimal sub-models are trained for each modality or sub-task, and prediction results and feature importance are outputted to ensure the optimal performance of single-modality models. A key innovation is the introduction of a dynamic fusion module driven by medical prior knowledge (such as pathological mechanisms and clinical guidelines) to adaptively and dynamically adjust the weights of each sub-model based on the specific case characteristics, making the final decision both data-driven and consistent with medical logic. In addition, an S+ scoring module is constructed to achieve scientific quantitative evaluation of the disease. The embedding vectors of patients and indicators are obtained based on graph neural networks, the cosine similarity is used to select the key indicators most relevant to the patient's state, and the probability distribution is constructed based on the frequency of the indicators to reflect the importance of the indicators. The S+ score for SLE disease activity evaluation is constructed by adding the expected value of the frequency probability and the correlation weight. This score covers 148 indicators, far exceeding the 24 indicators of SLEDAI-2000 score, and has a more comprehensive evaluation dimension. It is based on local data and is easy to localize and promote. The whole process is data-driven, and the results are objective, repeatable, and independent of the subjective experience of doctors. Based on the above technical solution, an interpretable decision and hierarchical treatment support are realized. The scoring method can clearly show the key indicators and their contribution, and provide screening and step-by-step treatment recommendations. The output design is user-friendly, especially suitable for primary care settings, and only requires routine and simple indicators to achieve high-precision auxiliary judgment.

[0036] The multi-modal medical atlas and medical knowledge fusion-based systemic lupus erythematosus evaluation method provided in this embodiment, as shown in Figure 1 The method comprises the following steps: Step S100, obtain the multi-dimensional medical indicators of the patient, and establish the correlation between the patient and the medical indicators as nodes in a graph structure to obtain a multi-modal medical knowledge graph.

[0037] In the embodiment, the patient is a specific evaluation object of the evaluation method, including diagnosed patients, suspected patients, and healthy people. By establishing a multi-modal medical knowledge graph, the unified modeling of patients and multi-modal indicators is realized, and the potential correlation between patients and indicators, and indicators and indicators can be explicitly captured. The graph structure used by the multi-modal medical knowledge graph can use a bipartite graph (Bipartite Graph) structure of patients and indicators, or can use a hypergraph (Hypergraph), a temporal graph (Temporal Graph), and a heterogeneous information network (HIN) for expression. Specifically, the hypergraph models the high-order collaborative relationship between indicators (for example, multiple indicators jointly pointing to a complication), to represent a simple bilateral relationship, that is, a hyperedge is used to simultaneously connect a patient and multiple indicators (or vice versa), and the multi-element relationship is more naturally modeled. The temporal graph jointly models the follow-up time sequence information of the patient and the dynamic evolution process of the medical indicators to replace the static graph model, that is, the static graph is expanded into a dynamic graph, the node increases with the increase of the time point of the visit, and the edge represents the time sequence evolution relationship, which is used to capture the disease progression dynamics. The heterogeneous information network introduces more types of nodes (such as symptoms, drugs, departments, etc.), constructs a more rich heterogeneous graph, and uses a meta-path (Meta-Path) to model the semantic relationship.

[0038] In an implementation manner, the medical indicators include hematology indicators, immunology indicators, kidney function indicators, liver function indicators, and imaging features. The multi-dimensional medical indicators of the patient are acquired, the patient and the medical indicators are taken as nodes of a graph structure respectively, the correlation between the patient and the medical indicators is established, and a multi-modal medical knowledge graph is obtained. Specifically, the following steps are included. In step S110, each patient is taken as an independent node, each medical indicator is taken as an independent node, each patient and each corresponding medical indicator is taken as an edge, and the standardized value of the measured value of the patient and the corresponding medical indicator is taken as an edge weight, to construct a patient and medical indicator bipartite graph.

[0039] In this embodiment, the multi-modal medical indicator set adopted is a comprehensive indicator pool highly related to the diagnosis and treatment of systemic lupus erythematosus, which is determined in cooperation with clinical experts of the Second People's Hospital of Shenzhen. The indicator set aims to comprehensively cover the multi-organ and multi-system involvement characteristics of SLE, mainly covering five modalities. Specifically, the hematological indicators include white blood cell count, lymphocyte count, hemoglobin, platelet count, etc.; the immunological indicators include anti-dsDNA antibody, complement C3, complement C4, immunoglobulin G (IgG), etc.; the renal function indicators include serum creatinine, urea nitrogen, urine protein quantification, etc.; the liver function indicators include alanine aminotransferase (ALT), aspartate aminotransferase (AST), etc.; the imaging features include quantitative or qualitative features related to SLE in ultrasound, CT or MRI reports (such as kidney volume, synovial membrane thickness, etc.).

[0040] In this embodiment, the graph structure used by the multi-modal medical knowledge graph is a bipartite graph structure of patients and indicators. Figure 2 The specific structure of the bipartite graph includes nodes and edges. The nodes include patient nodes and medical indicator nodes, where the patient nodes are defined as each SLE patient, and the medical indicator nodes are defined as covering the multi-modal medical indicators described above. The patient nodes and medical indicator nodes are connected to each other using edges, and the edge weights are set to the standardized values of the measured values of the patient and the corresponding connected medical indicators. When missing, the edge weight is set to 0. Unlike traditional methods that represent patients as isolated feature vectors, the present invention first constructs multi-modal medical data into a bipartite graph structure of patients and medical indicators, explicitly modeling the complex correlation between patients and indicators, indicators and indicators, and achieving unified modeling of patients and multi-modal indicators, which can explicitly capture the potential correlation between patients and indicators, indicators and indicators, such as the co-occurrence of elevated creatinine and urine protein.

[0041] Step S200, based on the multi-modal medical knowledge graph, a graph neural network model for self-supervised learning is constructed and pre-trained to obtain patient embedding vectors and medical indicator embedding vectors, wherein the patient embedding vector is a low-dimensional dense vector representing the information of all medical indicators corresponding to a patient, and the medical indicator embedding vector is a low-dimensional dense vector representing the information of all patients corresponding to a medical indicator.

[0042] In the embodiment, by constructing a graph neural network model, patient embedding vectors and medical index embedding vectors are obtained based on a multi-modal medical knowledge graph. Specifically, the graph neural network model can use GNN (Graph Neural Network) as the core of graph learning, or use graph Transformer, replace the message passing mechanism of GNN with the self-attention mechanism, better handle the long-range dependency in the graph, and enhance the interpretability of the model through attention weight. In addition, a graph autoencoder (GAE / VGAE) can also be used, which is a graph representation learning structure with encoder and decoder, and is also suitable for reconstruction tasks and embedding vector generation.

[0043] For the constructed graph neural network model, node masking can be used for self-supervised pre-training, or a contrastive learning method can be used to enhance the discriminability of node embedding by constructing different enhanced views of the same patient, or a multi-view reconstruction task can be used, such as masking the patient node and its associated index node at the same time, and then performing collaborative reconstruction to learn a more robust representation.

[0044] In an implementation manner, the graph neural network model of self-supervised learning is constructed and pre-trained based on the multi-modal medical knowledge graph to obtain patient embedding vectors and medical index embedding vectors, wherein the patient embedding vector is a low-dimensional dense vector representing information of all medical indexes corresponding to a patient, and the medical index embedding vector is a low-dimensional dense vector representing information of all patients corresponding to a medical index, and the method comprises the following steps: Step S210, randomly masking patient nodes or index nodes in the multi-modal medical knowledge graph with a preset probability; Step S220, constructing a graph neural network model of self-supervised learning, and predicting information of the masked nodes by the graph neural network model of self-supervised learning to reconstruct the information of the masked nodes as a target for training, wherein the information of the masked nodes includes the existence and global attributes of the masked nodes; Step S230, generating patient embedding vectors and medical index embedding vectors using the trained graph neural network model of self-supervised learning.

[0045] In this embodiment, first, the random node discarding method is used to randomly mask the patient or indicator nodes in the graph with a probability p = 0.3 to simulate the missing of clinical data. This value is used to prevent model overfitting and improve generalization ability. The recommended range of masking probability is [0.1, 0.5]. Then, a graph neural network (GNN) based on the GraphSAGE architecture is used to train the GNN model to predict the values (continuous values) or existence (classification values) of the masked nodes. Specifically, the network has 3 layers of GNN, 64 hidden units per layer, ReLU activation function, Mean pooling (mean aggregation) aggregation function, Adam optimizer, learning rate set to 0.001, training rounds set to 20,000 rounds. For the selection of the loss function, the mean square error (MSE) is used for continuous value prediction, the cross-entropy (Cross-Entropy) is used for classification value prediction, and the mean absolute error (MAE) is used for evaluation index. The training objective of this network model is to minimize the reconstruction error, and there is no preset threshold. The MAE of the validation set no longer decreases as the early stopping condition. Further, the message passing, node updating and edge updating process of GNN is as follows: Message passing (l-th layer):

[0046] Node update:

[0047] Edge update:

[0048] where, represents the node embedding vector in the l-th layer; represents the edge embedding vector in the l-th layer; , , represent trainable weight matrices, respectively; represents the ReLU activation function; represents the neighbor set of the node ; represents the vector concatenation operation; represents the mean aggregation function.

[0049] Finally, the output of the model is the patient embedding vector representing the overall appearance of the individual and the medical indicator embedding vector representing the association between the indicators.

[0050] The self-supervised learning graph neural network model solves the problems of sample scarcity and data missing through self-supervised learning, and the embedding vector contains multi-modal correlation information, overcoming the problem of different modalities being split in traditional methods, and improving the adaptability and robustness of the model to real clinical environments.

[0051] By explicitly modeling the correlation between indicators, high-precision filling of missing medical indicators is achieved in the pre-training stage. For example Figure 4 As shown in the experimental verification data, the filling error of the embodiment is significantly improved compared with the traditional method, and the multi-modal split bottleneck of the traditional method is broken, the utilization rate of cross-group indicators such as blood, kidney, and immunity is improved, and the AUC of the early screening task can reach 0.99. And, as shown in Figure 4 The random node masking strategy is used to strengthen the anti-interference ability of the model. In the extreme test scenario of deliberately deleting 30% of the indicators, the filling error of the embodiment is significantly improved compared with the traditional method, which significantly enhances the generalization ability of the model in the scenario of poor data quality and high missing rate in primary hospitals. Clinical verification shows that the AUC can reach 0.99. On the basis of the two-part graph structure of patients and medical indicators, a self-supervised pre-training mechanism of random node discarding and reconstruction is introduced. This mechanism simulates the data missing scenario in real clinical practice, forcing the model to learn the deep pathological and physiological correlation between indicators rather than the surface statistical rule, thereby obtaining a high-robustness and high-information-density embedding vector representation, which can still maintain high precision in the primary medical environment where data quality is poor and missing is common.

[0052] Step S300, based on medical knowledge, divide the multi-dimensional medical indicators into several modal groups, construct several feature subset combinations based on the modal groups, and for each feature subset combination, train several sub-models and select an optimal sub-model, wherein each feature subset combination contains several medical indicators, and the sub-model is used to predict the binary classification prediction probability of systemic lupus erythematosus through the corresponding feature subset combination.

[0053] In this embodiment, multi-dimensional medical indicators are grouped, and an optimal sub-model is provided for each group to predict the binary classification prediction probability of systemic lupus erythematosus based on the grouped medical indicators. In the selection of grouping and integration strategy, heterogeneous models can be trained according to medical feature grouping, or multi-task learning (MTL) can be used. Specifically, a multi-task learning shared backbone feature extraction network is used, such as a Transformer-based encoder, different output heads are used to perform prediction tasks for each group, and parameter hard sharing is used to reduce the risk of overfitting. In addition, a MoE (Mixture of Experts) model can also be selected, which specifically trains multiple expert sub-networks (Expert Networks) and a gating network (Gating Network), dynamically selects and combines expert outputs by the gating network, and realizes soft and data-driven grouping fusion.

[0054] In an implementation manner, the multi-dimensional medical indicators are divided into a plurality of modal groups based on medical knowledge, a plurality of feature subset combinations are constructed based on the modal groups, and for each feature subset combination, a plurality of sub-models are trained and an optimal sub-model is selected, wherein each feature subset combination includes a plurality of medical indicators, and the sub-model is used to predict the binary classification prediction probability of systemic lupus erythematosus through the corresponding feature subset combination, and specifically includes the following steps: Step S310, based on medical knowledge, the medical indicators are divided into five modal groups, wherein the five modal groups include a hematology group, an immunology group, a renal function group, a liver function group, and an imaging group; Step S320, a non-empty power set of the five modal groups is constructed to obtain a plurality of feature subset combinations; Step S330, using the self-supervised learning graph neural network model, the missing values of the medical indicators in each feature subset combination are predicted and filled to obtain the filled feature subset combination; Step S340, the filled feature subset combination is subjected to synthetic minority over-sampling, to obtain a plurality of synthetic samples for expanding the patient samples; Step S350, for each feature subset combination, a plurality of different types of binary classification models are trained, wherein the binary classification models include support vector machines, K-nearest neighbor algorithms, random forests, extreme gradient boosting algorithms, and multi-layer perceptrons; Step S360, using grid search and 5-fold cross-validation, the average AUC-ROC value of cross-validation is used as the evaluation criterion of model performance, and from the trained plurality of different types of binary classification models, an optimal model is selected for each feature subset combination.

[0055] In this embodiment, the first step in the clinical knowledge-based grouping modeling process is to execute subset partitioning rules. Based on comprehensive medical knowledge covering the multi-organ and multi-system involvement characteristics of SLE, all clinical indicators can be divided into the following five core modality groups: hematology, immunology, renal function, liver function, and imaging. Specifically, G1 (hematology group) includes indicators such as white blood cell count, lymphocyte count, hemoglobin, and platelet count; G2 (immunology group) includes indicators such as anti-dsDNA antibody, complement C3, complement C4, and immunoglobulin G (IgG); G3 (renal function group) includes indicators such as serum creatinine, blood urea nitrogen, and quantified urine protein; G4 (liver function group) includes indicators such as alanine aminotransferase (ALT) and aspartate aminotransferase (AST); and G5 (imaging group) includes quantitative or qualitative features related to SLE in ultrasound, CT, or MRI reports (such as kidney volume and synovial thickness).

[0056] To further explore the predictive power of different modality combinations, instead of using only the five complete groups, all possible non-empty subset combinations, i.e., non-empty power sets, were constructed. For k=5 groups, a total of 31 feature subset combinations were generated, namely G1, G2, ..., G1+G2, G1+G3, ..., G1+G2+G3+G4+G5, each combination corresponding to a unique dataset perspective.

[0057] Following this, data preprocessing and missing value imputation are necessary. Because real-world clinical data often contains widespread missing values ​​and is class imbalanced (e.g., the number of SLE-positive patients is far less than the negative controls), missing value imputation and oversampling operations need to be performed sequentially before model training to generate a high-quality, balanced training dataset for each feature subset combination, such as... Figure 3 As shown in (a). Specifically, in the missing value imputation step, the pre-trained self-supervised learning graph neural network model described above is first used to impute missing values ​​in the original clinical indicator data of all patients. Here, "all patients" refers to both SLE-positive patients and normal control samples. This GNN model can perform high-precision imputation using the graph structure relationship between features, outputting a complete, missing-free data matrix. Subsequently, oversampling is performed. To address the class imbalance problem, the Synthetic Minority Over-Sampling Technique (SMOTE) is applied to the imputed original data in the non-embedded space to expand the sample data volume.

[0058] After the above-mentioned preprocessing of the sample data, an optimal sub-model is selected for each subset. For each of the 31 feature subset combinations generated in the above steps, we use its corresponding, filled-in complete data to train a binary classification model for predicting whether a patient is SLE positive or negative. Based on this, a binary classification model needs to be selected. For each combination, a grid search (Grid Search) and 5-fold cross-validation (5-Fold Cross-Validation) are used to find the optimal model among the existing various machine learning algorithms, such as Figure 3 (c) shown. Among them, support vector machine (SVM), K nearest neighbor (KNN), random forest (Random Forest), extreme gradient boosting algorithm (XGBoost) and multilayer perceptron (MLP) can be selected. The selection criteria for the binary classification model is that the average AUC-ROC (Area Under Receiver Operating Characteristic Curve) value of cross-validation is used as the criterion for model performance. For each feature subset combination, the algorithm model with the highest AUC-ROC score is selected as the optimal sub-model for the group.

[0059] Through such a design of grouped heterogeneous sub-models, the module output conforms to the cognitive habits of specialists, and the rheumatology and immunology physician's trust score for disease classification results is improved to 4.2 / 5.0, while the control group is 2.8 / 5.0.

[0060] In an implementation manner, the filled-in feature subset combination is subjected to synthetic minority over-sampling, to obtain a plurality of synthetic samples for expanding the patient samples, specifically including the following steps: Step S341, in the minority class systemic lupus erythematosus positive patient sample, for each sample, a plurality of neighbors are randomly selected from its k neighbors; Step S342, in the feature space, for each selected minority class sample, a point is randomly selected on the connecting line between the sample and the randomly selected neighbor, to generate a new synthetic sample, wherein the synthetic sample is a linear combination of the original sample features; Step S343, a series of generated synthetic samples are added to the training data set of the binary classification model, wherein the synthetic samples statistically approach the feature vectors of the real patients.

[0061] In this embodiment, the synthetic minority oversampling technique first identifies the samples in the minority class SLE-positive patients. For each sample, a number of neighbors are randomly selected from its k-nearest neighbors (e.g., k = 5). Then, for each selected minority class sample, a new synthetic sample is generated by randomly selecting a point on the line between the selected sample and a randomly selected neighbor in the feature space. The feature values of the synthetic sample are linear combinations of the original sample features. Finally, the generated series of synthetic samples are added to the training dataset. These synthetic samples are virtual but statistically realistic patient feature vectors, which aim to expand the sparse SLE-positive patient data and balance the number of samples in the positive and negative classes, thereby avoiding bias in model training.

[0062] Applying the SMOTE algorithm and other generation algorithms in the pre-trained embedding space rather than the original data space can efficiently and effectively synthesize minority class samples, avoid the noise amplification problem caused by traditional oversampling methods, significantly improve the fitting ability of the model for long-tail distribution data, and further improve the recognition ability of rare cases and complications. As shown in Figure 4 The SMOTE technique in the embedding space performs oversampling in the patient embedding vector space, avoids noise amplification in the original data, and can achieve an F1-score of 0.97 for positive samples. Moreover, the recognition rate for rare complications (e.g., lupus encephalopathy) with extremely small sample sizes is improved from 51% to 89%, filling the clinical blind spots.

[0063] In step S400, a dynamic weighted fusion module is constructed based on the medical knowledge and the optimal sub-model corresponding to each feature subset combination. The binary classification prediction probabilities corresponding to all feature subset combinations are fused through the dynamic weighted fusion module to obtain the comprehensive prediction probability of systemic lupus erythematosus.

[0064] In this embodiment, the comprehensive prediction probability is essentially the probability that the patient is SLE-positive. Specifically, each feature subset combination corresponds to an optimal sub-model for predicting the binary classification prediction probability of systemic lupus erythematosus, i.e., the probability of being positive or negative. The dynamic weighted fusion module integrates the binary classification prediction probabilities of these sub-models to obtain the final comprehensive prediction probability, which quantifies the likelihood of the patient having SLE. Based on this, the comprehensive prediction probability can be used for rapid assessment of the likelihood of SLE positivity in primary medical care, which is suitable for primary medical care and supports hierarchical diagnosis and treatment. Specifically, primary medical resources are limited and data quality is easily affected. However, the comprehensive prediction probability integrates multi-modal indicator information and medical knowledge, so even based on conventional indicators that can be obtained at the primary level, such as basic hematology, liver and kidney function indicators, it can output reliable positive probabilities, helping primary medical personnel to quickly assess the risk of the patient's disease and provide a quantitative basis for further referral or intervention.

[0065] The expert knowledge fusion mechanism adopts an expert knowledge fusion method to fuse expert knowledge and multiple optimal sub-models to predict a comprehensive prediction probability of SLE. The comprehensive prediction probability represents the evaluation probability result of the method for the patient SLE being positive. In this embodiment, in addition to using the expert knowledge fusion method, a differentiable rule network can also be used to convert the expert priori rules into a differentiable network module, so as to realize end-to-end joint optimization with the deep learning model. Through the differentiable rule network, the expert rules are converted into a differentiable network layer, for example, a distance-based activation function is used, so that it can be optimized with the entire model end-to-end.

[0066] Alternatively, in the case of sufficient expert decision annotation data, an attention mechanism can be used to automatically learn the importance weight of different sub-model results to replace the frequency statistics or hard rule weighting method. The knowledge fusion based on the attention mechanism specifically uses a knowledge graph embedding or a medical ontology to generate a key-value pair, and uses a cross-attention mechanism to enable the model to dynamically retrieve and fuse related knowledge, replacing the static rule base. Using the attention mechanism of the annotation expert decision data to automatically learn the weight or using a Bayesian network to model the probability dependence between sub-models is also an optional solution.

[0067] In an implementation mode, the optimal sub-model corresponding to each feature subset combination based on medical knowledge is used to construct a dynamic weighted fusion module, and the binary classification prediction probability corresponding to each feature subset combination is fused through the dynamic weighted fusion module to obtain a comprehensive prediction probability of systemic lupus erythematosus, and the method comprises the following steps: Step S410, based on medical knowledge, the medical importance score of each modality group is marked, and the medical importance score is dynamically adjusted based on the clinical state of the patient; Step S420, based on the AUC-ROC value of each optimal sub-model, a performance weight is marked, wherein the performance weight is proportional to the AUC-ROC value of the optimal sub-model; Step S430, based on the number of medical indicators contained in the feature subset combination corresponding to each optimal sub-model, a complexity weight is marked, wherein the complexity weight is proportional to the number of medical indicators contained in the feature subset combination; Step S440, based on the average value of the medical importance score of the modality group contained in the feature subset combination corresponding to each optimal sub-model, a knowledge weight is marked, wherein the knowledge weight is proportional to the average value of the medical importance score; Step S450, integrating the performance weight, the complexity weight, and the knowledge weight to obtain a dynamic weight; Step S460, based on the dynamic weight, performing weighted average on the binary classification prediction probabilities of systemic lupus erythematosus output by all the optimal sub-models to obtain a comprehensive prediction probability of systemic lupus erythematosus.

[0068] In the embodiment, as shown in Figure 3 The step of expert knowledge driven weighted fusion is first to construct a rule base. Specifically, based on international medical guidelines (such as the SLE classification criteria of ACR and SLICC) and clinical expert experience, a fixed importance score is defined for each of the above defined clinical indicator modal group (G1-G5) to quantify the core position of the group of indicators in SLE evaluation. The scoring rules adopt a five-level system (I-V), and the higher the score represents the more important the group of indicators in the comprehensive evaluation. The detailed content of the modal group, the indicator name, and the medical importance score is shown in Table 1.

[0069] Table 1

[0070] When receiving the current clinical state information of the patient, it is necessary to dynamically adjust its model fusion strategy according to the pre-defined rules. The clinical state mainly includes two dimensions of disease course stage and chief complaint symptoms and suspected affected systems. Among them, the disease course stage is divided into acute onset period and stable period, and the chief complaint symptoms and suspected affected systems are the keywords input by the doctor or extracted from the electronic medical record, such as joint pain, rash, edema, etc.

[0071] Then, the specific weight of dynamic fusion is determined. For each of the above trained optimal sub-model corresponding to a feature subset combination, the weight in the final fusion is determined by the multiplication of three parts, including the performance weight, the complexity weight, and the knowledge weight, as shown in Figure 3 (b).

[0072] The performance weight (Performance Weight) is the best AUC-ROC value of the sub-model in cross-validation, denoted as The better the performance of the model, the higher the weight.

[0073] The complexity weight (Complexity Weight) is , wherein is the total number of original clinical indicators contained in the feature subset combination, and is not the number of modal groups. This item is a slight reward for the sub-models that use more features and have higher model complexity.

[0074] The Knowledge Weight is the average medical score of this feature subset combination. Specifically, it is calculated by averaging the scores of all modal groups (G1-G5) included in the combination as shown in Table 1. For groups whose scores were temporarily boosted, the boosted values ​​are used in the calculation.

[0075] The three weights mentioned above are integrated using the formula "submodel best performance (AUC-ROC) × log(number of subset metrics N+1) × average subset score" to obtain the dynamic weights of the submodel. The calculation formula is as follows:

[0076] Among them, combination Characterizing the first Combination of feature subsets Characteristic Combination The included modal group, Combination of feature subsets The included mode groups Medical importance score, For combination The total number of modal groups included.

[0077] After obtaining the dynamic weights, the fusion formula of the knowledge fusion mechanism uses a weighted average of the predicted probabilities of all sub-models, such as... Figure 3 As shown in (d), below:

[0078] in, For the final comprehensive prediction probability, It is the first The predicted probabilities given by each sub-model It is its corresponding dynamic weight.

[0079] By employing the aforementioned fusion method, prior medical knowledge is encoded into quantifiable rules, thus addressing the problem of black-box models. Simultaneously, dynamic weights enhance the model's adaptability to complex scenarios. For example... Figure 4 As shown, the fusion results meet the ACR / SLICC standard by 96%, and the model's decision path is consistent with medical logic and has been certified by the expert committee.

[0080] Step S500: Based on the patient embedding vector and the medical indicator embedding vector, select several key medical indicators and quantify the activity assessment contribution of the key medical indicators. Based on the patient's key medical indicators and the corresponding activity assessment contribution, obtain the comprehensive activity score of systemic lupus erythematosus.

[0081] In the present embodiment, the scoring mechanism of the comprehensive score of the activity of systemic lupus erythematosus, named S+ scoring mechanism in the present embodiment, is a comprehensive score for quantifying the activity of systemic lupus erythematosus generated based on multi-modal medical atlas and graph neural network embedding vectors. Specifically, it outputs an activity evaluation result of 0-100 points by screening key indicators and quantifying the contribution. The comprehensive score of the activity of systemic lupus erythematosus can specifically quantify the activity of the disease, replace the subjective SLEDAI score, output key indicators and contribution, and support primary classification and treatment. Specifically, if the comprehensive score of the activity of systemic lupus erythematosus of a patient is in a high segment (> 50 points), early screening needs to be prompted, a middle segment (20-50 points) prompts the start of the classification and treatment process, and is used in the scenario of primary medical care. In other words, the S+ scoring mechanism is completely based on data to automatically discover key indicators, and is free from the dependence on fixed scoring rules and expert subjective experience of, for example, SLEDAI, and has interpretability. For each patient, the S+ score is generated, and the Top-N key indicators can be output, directly informing the doctor of the basis for decision-making. In addition, the scoring mechanism has the advantages of scalability and localization. It is constructed based on training data, and by incorporating data from different regions, a more targeted S+ scoring model that fits the characteristics of the local population can be trained.

[0082] In an implementation manner, the screening of the several key medical indicators and the quantification of the activity evaluation contribution of the key medical indicators based on the patient embedding vector and the medical indicator embedding vector, and the obtaining of the comprehensive score of the activity of systemic lupus erythematosus based on the key medical indicators of the patient and the corresponding activity evaluation contribution, specifically include the following steps: Step S510, for each patient embedding vector, the cosine similarity between the patient embedding vector and all medical indicator embedding vectors is calculated, and the three medical indicator embedding vectors with the highest cosine similarity are selected as the medical indicator embedding vectors of the key medical indicators; Step S520, the medical indicator embedding vectors of the key medical indicators selected by each patient are recorded in a global list, and the occurrence frequencies of the medical indicator embedding vectors recorded in the global list are counted, wherein the occurrence frequencies are taken as the activity evaluation contribution of the medical indicators; Step S530, based on the occurrence frequencies, the probability distribution of each medical indicator in the patient space is calculated to obtain a global probability distribution; Step S540, for each patient, based on the cosine similarity between the patient embedding vector and all medical indicator embedding vectors and the global probability distribution, the sum of expected values is calculated, and linear normalization is performed to obtain the comprehensive score of the activity of systemic lupus erythematosus.

[0083] In the step of S+ score calculation and key indicator extraction in the present embodiment, firstly, similarity calculation and sorting based on embedding vectors. For the embedding vector of each patient, the cosine similarity between it and all indicator embedding vectors is calculated, and the calculated cosine similarities are sorted from high to low, and finally the embedding vectors of the top-3 indicators with the highest cosine similarity are selected as the top-3 key indicators. These vectors reflect the key clinical indicators most relevant to the current patient status.

[0084] After screening the top-3 key indicators, the top-3 key indicator embedding vectors of all patients are recorded in a global list, and the frequency of each specific indicator embedding vector in the list is counted, which directly reflects the relevance and importance of the indicator to the disease state in all patients. Based on the above frequency, the probability distribution P(indicator) of each indicator in the patient space is calculated. For each patient, an expected value is calculated according to its own similarity to each indicator and the global probability distribution of each indicator. The expected value is obtained by multiplying the frequency probability of each indicator with its similarity weight to the current patient and summing them up.

[0085] Through linear normalization, a comprehensive score that can be used to quantify and evaluate disease activity, i.e. S+ score, is finally obtained. The linear normalization formula used is as follows:

[0086] Wherein, represents the sum of the expected values calculated for the current patient, represents the minimum value of the sum of the expected values of all patient samples, represents the maximum value of the sum of the expected values of all patient samples, and are the upper and lower bounds of the score interval. In the present embodiment, they can be set as , The linear normalization method used can ensure the relative linear relationship of the score distribution, and the calculation is efficient and the result is stable, which is convenient for clinicians to understand and apply.

[0087] Through the above S+ score mechanism and key indicator extraction technology, an interpretable S+ disease activity score can be automatically generated, such as Figure 4The verification data shows that the Spearman correlation coefficient with SLEDAI-2000 score reaches 0.88, and the coverage index range is improved to 148. The top-3 key decision indicators are extracted for each patient, and the doctor survey shows that this function can improve the credibility of model decision by 50%. In addition, screening and grading functions are realized. S+ score high segment (S+ score greater than 50) is used for early screening, and middle segment (S+ score in the interval of 20 to 50) prompts to start the grading diagnosis process, guiding patients to different levels of hospitals, and effectively empowering the grassroots.

[0088] In addition, in the calculation of S+ abnormal score, the distance metric and the weight learning method, the cosine similarity can be replaced by the distance metric function, which can adopt Euclidean distance or Mahalanobis distance to adapt to different data distribution assumptions. The weight learning method can adopt attention mechanism to automatically learn, instead of manually defined frequency statistics method.

[0089] Further, the systemic lupus erythematosus evaluation result includes the comprehensive prediction probability of the patient and the S+ score of the patient. By combining the two, a double-channel intelligent decision-making model of global data-driven and local knowledge-guided is realized.

[0090] Specifically, the multi-channel hierarchical decision system includes data-driven and knowledge-guided. The data-driven part is based on the embedding vector described above, and the S+ score module is designed innovatively. The module generates an objective, quantitative and interpretable disease activity score by calculating the expected value of the patient's state and the importance of the global indicators. It has the advantages of scalability, localization and scientificity. The knowledge-guided part constructs a group heterogeneous sub-model and a dynamic fusion mechanism. According to the medical prior knowledge, the indicators are grouped and the optimal sub-model is trained for each group. A dynamic weight network based on expert rules is introduced innovatively, which can adjust the decision weight of each sub-model in real time according to the input patient's characteristics. A hierarchical decision system is proposed, which integrates the data-driven S+ score module and the knowledge-guided group heterogeneous sub-model and dynamic fusion mechanism. The dynamic weight network can adjust the weight of each sub-model in real time according to the medical prior knowledge. Such design not only excavates the statistical rules from global data, but also introduces constraints from local medical knowledge, so that the model decision meets the data optimization and medical rationality at the same time, greatly improving the evaluation accuracy and clinical acceptability.

[0091] Meanwhile, the clinically interpretable output and hierarchical diagnosis and treatment support mechanism bridges the final gap between technology and application. This method not only outputs evaluation conclusions but also provides a complete clinical decision support report. The contribution of key indicators can be automatically traced and clearly marked with the key clinical indicators that contribute the most to the current decision, along with their quantitative contributions. Building on this, it further provides hierarchical diagnosis and treatment recommendations based on the S+ score, transforming black-box decision-making into understandable white-box logic, significantly enhancing physician trust. Simultaneously, by simplifying input requirements, it greatly improves practicality and ease of use in primary healthcare institutions.

[0092] The technical solutions and core ideas covered by the above evaluation methods have profound significance and potential for expansion.

[0093] Firstly, this invention focuses on the medical concept mining and interpretability enhancement of embedded vectors. Specifically, based on graph structure modeling and a self-supervised learning mechanism, the learned indicator embedded vectors exhibit a non-random distribution in the vector space, and their geometric relationships contain rich medical semantics. Cluster analysis results show that the embedded vectors of indicators related to kidney function (such as creatinine and urinary protein) spontaneously cluster in the space, forming a cluster of concepts related to kidney function impairment. Similarly, immune-related indicators (such as complement C3, C4, and dsDNA) form a cluster of concepts related to immune activation. This phenomenon not only provides strong evidence for the interpretability of the model but also offers a new method for the automatic mining of disease biomarkers. Doctors can understand the potential connections between indicators and even discover new medical insights through visual exploration of the vector space.

[0094] Secondly, a cross-institutional federated learning and model transfer framework was constructed. Specifically, the pre-trained encoder (i.e., the graph self-supervised learning module) used in this invention has strong versatility and can be deployed as an independent medical representation extractor. Based on this, a federated learning system can be further constructed. Each collaborating hospital uses local data to fine-tune a copy of the pre-trained model, uploading only updated model parameters without sharing the original data. Collaborative optimization is achieved by aggregating parameters on a central server. This mechanism effectively breaks down medical data silos, integrates multi-center data resources to continuously improve model performance while strictly protecting patient privacy, and constructs a national-level intelligent assessment foundation model with stronger generalization capabilities.

[0095] Furthermore, it possesses the capability to cover the entire diagnosis and treatment process and provide personalized treatment recommendations. Specifically, although the current framework of this invention primarily focuses on disease assessment, its outputs (such as S+ scores, key indicators, patient embedding vectors, etc.) can naturally extend to the treatment stage. The system can retrieve historical similar cases in the embedding space based on the patient's current status and refer to their successful treatment cases to provide doctors with personalized treatment suggestions. Ultimately, it can realize a closed-loop intelligent auxiliary system covering the entire process from screening, assessment, treatment plan recommendation, prognosis, and recurrence prediction, promoting the leap of medical services from single assessment assistance to full-cycle health management.

[0096] Finally, this method also possesses the versatility of a technical platform and the ability to be applied to other diseases. The core technical framework of this invention exhibits a high degree of disease-agnosticness. Its graph structure modeling, self-supervised pre-training, interpretable scoring, and decision structure can be transferred and applied to other complex chronic diseases, such as rheumatoid arthritis (RA), inflammatory bowel disease (IBD), and diabetes. By replacing the corresponding medical indicator nodes and expert knowledge rules, specialized intelligent assessment devices for specific diseases can be quickly constructed, demonstrating powerful platform capabilities and scalability.

[0097] In summary, the evaluation method proposed in this invention significantly improves evaluation performance and generalization ability. Specifically, by deeply mining the intrinsic correlations and robust feature representations of multimodal data, the model exhibits higher accuracy, sensitivity, and specificity in tasks such as early SLE screening, assisted assessment, subtyping, activity assessment, and recurrence prediction. Especially when dealing with imbalanced samples, missing data, and complex cases, its robustness is significantly superior to existing methods. Furthermore, this evaluation method supports efficient early screening and tiered medical services: based on a knowledge-guided dynamic fusion mechanism, early disease screening can be achieved with only a few simple indicators, significantly reducing patients' time and economic costs, and providing clear guidance for tiered medical services, strongly supporting primary healthcare and the implementation of tiered medical services. In addition, it addresses the black-box problem. By providing clear and traceable decision-making basis, it effectively alleviates the AI ​​black-box problem and enhances clinicians' trust. Through the knowledge-guided dynamic fusion mechanism, it effectively achieves the organic integration of data-driven approaches and medical knowledge, ensuring that the prediction results are both data-optimal and medically reasonable, improving clinical practicality and physician acceptance. Finally, the S+ scoring system enhances clinical integration and practical value. The S+ scoring system boasts scalability (supporting indicator expansion), local adaptability (adaptable to data from different regions), and scientific rigor (data-driven, objective, and accurate). Its output closely aligns with clinical workflows, significantly enhancing its feasibility for implementation in healthcare institutions at all levels, especially primary care hospitals. This invention, through innovations in graph structure learning, knowledge fusion, and interpretability, provides a high-performance, highly reliable, and clinically friendly new solution for intelligent SLE assessment, geared towards primary care and supporting screening and tiered diagnosis and treatment, effectively addressing key challenges in existing technologies and practices.

[0098] like Figure 5 As shown in the figure, this embodiment of the invention provides a systemic lupus erythematosus assessment device based on the fusion of multimodal medical atlas and medical knowledge. The device includes: a multimodal medical knowledge graph construction module 10, a self-supervised learning graph neural network model construction and training module 20, an optimal sub-model construction module 30, a comprehensive prediction probability calculation module 40, an activity assessment contribution calculation module 50, and an assessment result calculation module 60.

[0099] Specifically, the multimodal medical knowledge graph construction module 10 is used to acquire multi-dimensional medical indicators of patients, and establish the association between patients and medical indicators as nodes of a graph structure, thereby obtaining a multimodal medical knowledge graph; the self-supervised learning graph neural network model construction and training module 20 is used to construct a self-supervised learning graph neural network model based on the multimodal medical knowledge graph and perform pre-training to obtain patient embedding vectors and medical indicator embedding vectors, wherein the patient embedding vector is a low-dimensional dense vector representing the information of all medical indicators corresponding to a patient, and the medical indicator embedding vector is a low-dimensional dense vector representing the information of all patients corresponding to a medical indicator; the optimal sub-model construction module 30 is used to divide multi-dimensional medical indicators into several modality groups based on medical knowledge, construct several feature subset combinations based on the modality groups, and target... For each feature subset combination, several sub-models are trained and an optimal sub-model is selected. Each feature subset combination contains several medical indicators. The sub-model is used to predict the binary prediction probability of systemic lupus erythematosus (SLE) based on the corresponding feature subset combination. The comprehensive prediction probability calculation module 40 is used to construct a dynamic weighted fusion module based on medical knowledge and the optimal sub-model corresponding to each feature subset combination. The dynamic weighted fusion module is used to fuse the binary prediction probabilities corresponding to all feature subset combinations to obtain the comprehensive prediction probability of SLE. The activity assessment contribution calculation module 50 is used to screen several key medical indicators based on the patient embedding vector and the medical indicator embedding vector, quantify the activity assessment contribution of the key medical indicators, and obtain the comprehensive activity score of SLE based on the patient's key medical indicators and the corresponding activity assessment contribution.

[0100] Based on the above embodiments, the present invention also provides a terminal device, the principle block diagram of which can be as follows: Figure 6 As shown, the terminal device includes a processor, memory, network interface, display screen, and temperature sensor connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a systemic lupus erythematosus assessment method based on the fusion of multimodal medical atlases and medical knowledge. The display screen can be an LCD screen or an e-ink screen. The temperature sensor is pre-installed inside the terminal device to detect the operating temperature of the internal components.

[0101] Those skilled in the art will understand that Figure 6 The schematic diagram shown is only a partial structural diagram related to the present invention and does not constitute a limitation on the terminal device to which the present invention is applied. The specific terminal device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0102] In one embodiment, a terminal device is provided, including a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors. The one or more programs include instructions for performing the following operations: The patient's multi-dimensional medical indicators are obtained, and the patient and the medical indicators are respectively used as nodes in a graph structure to establish the relationship between the patient and the medical indicators, thus obtaining a multimodal medical knowledge graph. Based on the multimodal medical knowledge graph, a self-supervised learning graph neural network model is constructed and pre-trained to obtain patient embedding vectors and medical indicator embedding vectors. The patient embedding vector is a low-dimensional dense vector representing the information of all medical indicators corresponding to a patient, and the medical indicator embedding vector is a low-dimensional dense vector representing the information of all patients corresponding to a medical indicator. Based on medical knowledge, multi-dimensional medical indicators are divided into several modal groups. Several feature subset combinations are constructed based on the modal groups. For each feature subset combination, several sub-models are trained and an optimal sub-model is selected. Each feature subset combination contains several medical indicators. The sub-model is used to predict the binary classification prediction probability of systemic lupus erythematosus through the corresponding feature subset combination. Based on medical knowledge and the optimal sub-model corresponding to each feature subset combination, a dynamic weighted fusion module is constructed. Through the dynamic weighted fusion module, the binary prediction probabilities corresponding to all feature subset combinations are fused to obtain the comprehensive prediction probability of systemic lupus erythematosus. Based on the patient embedding vector and the medical indicator embedding vector, several key medical indicators are selected and their activity assessment contribution is quantified. Based on the patient's key medical indicators and their corresponding activity assessment contribution, a comprehensive activity score for systemic lupus erythematosus is obtained.

[0103] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0104] In summary, this invention discloses a method, device, terminal equipment, and medium for assessing systemic lupus erythematosus (SLE) based on the fusion of multimodal medical atlas and medical knowledge. The method first acquires multidimensional medical indicators of the patient and establishes the association between the patient and the medical indicators as nodes in a graph structure, thereby obtaining a multimodal medical knowledge graph. Subsequently, based on the multimodal medical knowledge graph, a self-supervised learning graph neural network model is constructed and pre-trained to obtain patient embedding vectors and medical indicator embedding vectors. The patient embedding vector is a low-dimensional dense vector representing information about all medical indicators corresponding to a patient, and the medical indicator embedding vector is a low-dimensional dense vector representing information about all patients corresponding to a medical indicator. Next, based on medical knowledge, the multidimensional medical indicators are divided into several modal groups. Several feature subset combinations are constructed based on the modal groups, and for each feature subset combination, several sub-models are trained, and an optimal sub-model is selected. Each feature subset combination contains several medical indicators, and the sub-model is used to predict the binary classification probability of SLE using the corresponding feature subset combination. Then, based on medical knowledge and the optimal sub-model corresponding to each feature subset combination, a dynamic weighted fusion module is constructed. This module fuses the binary classification prediction probabilities corresponding to all feature subset combinations to obtain the comprehensive prediction probability of systemic lupus erythematosus (SLE). Further, based on the patient embedding vector and the medical indicator embedding vector, several key medical indicators are selected, and their activity assessment contribution is quantified. Based on the patient's key medical indicators and their corresponding activity assessment contributions, a comprehensive activity score for SLE is obtained. This invention improves the accuracy and robustness of SLE assessment, solves the problems of missing data and imbalanced samples, provides interpretable assessment results, is suitable for primary healthcare and supports hierarchical diagnosis and treatment, and enhances clinical applicability.

[0105] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0106] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for assessing systemic lupus erythematosus based on the fusion of multimodal medical atlas and medical knowledge, characterized in that, The method includes: The patient's multi-dimensional medical indicators are obtained, and the patient and the medical indicators are respectively used as nodes in a graph structure to establish the relationship between the patient and the medical indicators, thus obtaining a multimodal medical knowledge graph. Based on the multimodal medical knowledge graph, a self-supervised learning graph neural network model is constructed and pre-trained to obtain patient embedding vectors and medical indicator embedding vectors. The patient embedding vector is a low-dimensional dense vector representing the information of all medical indicators corresponding to a patient, and the medical indicator embedding vector is a low-dimensional dense vector representing the information of all patients corresponding to a medical indicator. Based on medical knowledge, multi-dimensional medical indicators are divided into several modal groups. Several feature subset combinations are constructed based on the modal groups. For each feature subset combination, several sub-models are trained and an optimal sub-model is selected. Each feature subset combination contains several medical indicators. The sub-model is used to predict the binary classification prediction probability of systemic lupus erythematosus through the corresponding feature subset combination. Based on medical knowledge and the optimal sub-model corresponding to each feature subset combination, a dynamic weighted fusion module is constructed. Through the dynamic weighted fusion module, the binary prediction probabilities corresponding to all feature subset combinations are fused to obtain the comprehensive prediction probability of systemic lupus erythematosus. Based on the patient embedding vector and the medical indicator embedding vector, several key medical indicators are selected and their activity assessment contribution is quantified. Based on the patient's key medical indicators and their corresponding activity assessment contribution, a comprehensive activity score for systemic lupus erythematosus is obtained.

2. The systemic lupus erythematosus assessment method based on the fusion of multimodal medical atlas and medical knowledge according to claim 1, characterized in that, The medical indicators include hematological indicators, immunological indicators, renal function indicators, liver function indicators, and imaging features. The process involves acquiring multi-dimensional medical indicators for the patient and using the patient and the medical indicators as nodes in a graph structure to establish a relationship between the patient and the medical indicators, resulting in a multimodal medical knowledge graph, including: Using each patient as an independent node, each medical indicator as an independent node, each patient and each corresponding medical indicator as an edge, and the standardized values ​​of the measured values ​​of the patient and the corresponding medical indicator as edge weights, a bipartite graph of patients and medical indicators is constructed.

3. The systemic lupus erythematosus assessment method based on the fusion of multimodal medical atlas and medical knowledge according to claim 1, characterized in that, Based on the multimodal medical knowledge graph, a self-supervised learning graph neural network model is constructed and pre-trained to obtain patient embedding vectors and medical indicator embedding vectors. The patient embedding vector is a low-dimensional dense vector representing information about all medical indicators corresponding to a patient, and the medical indicator embedding vector is a low-dimensional dense vector representing information about all patients corresponding to a medical indicator, including: Patient nodes or indicator nodes in the multimodal medical knowledge graph are randomly masked with a preset probability; A self-supervised learning graph neural network model is constructed, and the information of the occluded node is predicted through the self-supervised learning graph neural network model. The training is carried out with the goal of reconstructing the information of the occluded node, wherein the information of the occluded node includes the existence of the occluded node and global attributes. Using the trained self-supervised learning graph neural network model, patient embedding vectors and medical indicator embedding vectors are generated.

4. The systemic lupus erythematosus assessment method based on the fusion of multimodal medical atlas and medical knowledge according to claim 1, characterized in that, The method involves dividing multi-dimensional medical indicators into several modal groups based on medical knowledge, constructing several feature subset combinations based on the modal groups, and training several sub-models for each feature subset combination, then selecting an optimal sub-model. Each feature subset combination contains several medical indicators, and the sub-model is used to predict the binary classification probability of systemic lupus erythematosus using the corresponding feature subset combination, including: Based on medical knowledge, the medical indicators are divided into five modal groups, including hematology group, immunology group, renal function group, liver function group, and imaging group. Construct non-empty power sets of the five modal groups to obtain several feature subset combinations; Using the self-supervised learning graph neural network model, the missing values ​​of medical indicators in each feature subset combination are predicted and filled to obtain the filled feature subset combination. The combined imputed feature subsets are subjected to synthetic minority class oversampling to obtain several synthetic samples, which are used to augment the patient samples. For each of the aforementioned feature subset combinations, several different types of binary classification models are trained, including support vector machines, K-nearest neighbors, random forests, extreme gradient boosting, and multilayer perceptrons. Using grid search and 5-fold cross-validation, and taking the average AUC-ROC value of the cross-validation as the criterion for model performance, an optimal model is selected from several trained binary classification models of different types for each feature subset combination.

5. The systemic lupus erythematosus assessment method based on the fusion of multimodal medical atlas and medical knowledge according to claim 4, characterized in that, The process of performing synthetic minority class oversampling on the augmented feature subset combination to obtain several synthetic samples for expanding the patient sample includes: In a small number of systemic lupus erythematosus positive patient samples, for each sample, several neighbors are randomly selected from its k nearest neighbors; In the feature space, for each selected minority class sample, a point is randomly selected on the line connecting it to a randomly selected neighbor to generate a new synthetic sample, wherein the synthetic sample is a linear combination of the features of the original sample. The generated series of synthetic samples are added to the training dataset of the binary classification model, wherein the synthetic samples statistically approximate the feature vectors of real patients.

6. The systemic lupus erythematosus assessment method based on the fusion of multimodal medical atlas and medical knowledge according to claim 4, characterized in that, The optimal sub-model corresponding to each feature subset combination, based on medical knowledge, constructs a dynamic weighted fusion module. This module then fuses the binary classification prediction probabilities corresponding to all feature subset combinations to obtain the comprehensive prediction probability of systemic lupus erythematosus, including: Based on medical knowledge, a medical importance score is assigned to each modality group, and the medical importance score is dynamically adjusted based on the patient's clinical status. Based on the AUC-ROC value of each of the optimal sub-models, performance weights are calibrated, wherein the performance weights are proportional to the AUC-ROC value of the optimal sub-model; Based on the number of medical indicators contained in the feature subset combination corresponding to each optimal sub-model, a complexity weight is assigned, wherein the complexity weight is proportional to the number of medical indicators contained in the feature subset combination. Based on the average medical importance score of the modality group included in the feature subset combination corresponding to each optimal sub-model, the knowledge weight is calibrated, wherein the knowledge weight is proportional to the average medical importance score; By integrating the performance weight, the complexity weight, and the knowledge weight, a dynamic weight is obtained. Based on the dynamic weights, the binary classification prediction probabilities of systemic lupus erythematosus output by all the optimal sub-models are weighted and averaged to obtain the comprehensive prediction probability of systemic lupus erythematosus.

7. The systemic lupus erythematosus assessment method based on the fusion of multimodal medical atlas and medical knowledge according to claim 1, characterized in that, The process involves selecting several key medical indicators based on the patient embedding vector and the medical indicator embedding vector, quantifying the activity assessment contribution of these key medical indicators, and obtaining a comprehensive activity score for systemic lupus erythematosus based on the patient's key medical indicators and their corresponding activity assessment contributions. This score includes: For each patient embedding vector, the cosine similarity between the patient embedding vector and all medical indicator embedding vectors is calculated, and the three medical indicator embedding vectors with the highest cosine similarity are selected as the medical indicator embedding vectors of the key medical indicators. The medical indicator embedding vectors of the key medical indicators selected for each patient are recorded in a global list, and the frequency of occurrence of each medical indicator embedding vector recorded in the global list is counted. The frequency of occurrence is used as the contribution of the medical indicator to the activity assessment. Based on the frequency of occurrence, the probability distribution of each medical indicator in the patient space is calculated to obtain the global probability distribution; For each patient, based on the cosine similarity between the patient's embedding vector and all medical indicator embedding vectors, and the global probability distribution, the sum of expected values ​​is calculated and linearly normalized to obtain the comprehensive activity score of systemic lupus erythematosus.

8. A systemic lupus erythematosus assessment device based on the fusion of multimodal medical atlas and medical knowledge, characterized in that, The device includes: A multimodal medical knowledge graph construction module is used to acquire multi-dimensional medical indicators of patients, and to establish the relationship between patients and medical indicators as nodes of a graph structure, thereby obtaining a multimodal medical knowledge graph. The self-supervised learning graph neural network model construction and training module is used to construct a self-supervised learning graph neural network model based on the multimodal medical knowledge graph and perform pre-training to obtain patient embedding vectors and medical indicator embedding vectors. The patient embedding vector is a low-dimensional dense vector representing the information of all medical indicators corresponding to a patient, and the medical indicator embedding vector is a low-dimensional dense vector representing the information of all patients corresponding to a medical indicator. The optimal sub-model construction module is used to divide multi-dimensional medical indicators into several modal groups based on medical knowledge, construct several feature subset combinations based on the modal groups, train several sub-models for each feature subset combination, and select an optimal sub-model. Each feature subset combination contains several medical indicators, and the sub-model is used to predict the binary classification prediction probability of systemic lupus erythematosus through the corresponding feature subset combination. The comprehensive prediction probability calculation module is used to construct a dynamic weighted fusion module based on medical knowledge and the optimal sub-model corresponding to each feature subset combination, and through the dynamic weighted fusion module, fuse the binary prediction probabilities corresponding to all feature subset combinations to obtain the comprehensive prediction probability of systemic lupus erythematosus. The activity assessment contribution calculation module is used to screen several key medical indicators and quantify the activity assessment contribution of the key medical indicators based on the patient embedding vector and the medical indicator embedding vector, and obtain the comprehensive activity score of systemic lupus erythematosus based on the patient's key medical indicators and the corresponding activity assessment contribution.

9. A terminal device, characterized in that, The terminal device includes a memory, a processor, and a systemic lupus erythematosus (SLE) assessment program based on the fusion of multimodal medical atlas and medical knowledge, stored in the memory and executable on the processor. When the processor executes the systemic lupus erythematosus assessment program based on the fusion of multimodal medical atlas and medical knowledge, it implements the steps of the systemic lupus erythematosus assessment method based on the fusion of multimodal medical atlas and medical knowledge as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a systemic lupus erythematosus (SLE) assessment program based on the fusion of multimodal medical atlases and medical knowledge. When the SLE assessment program based on the fusion of multimodal medical atlases and medical knowledge is executed by a processor, it implements the steps of the systemic lupus erythematosus assessment method based on the fusion of multimodal medical atlases and medical knowledge as described in any one of claims 1-7.

Images