Rare disease diagnosis and treatment resource collaborative scheduling method and system based on multi-modal fusion

Abstract

The application discloses a kind of based on multimodal fusion's rare disease diagnosis and treatment resource collaborative scheduling method and system, the method includes: extracting multidimensional feature from multimodal data, based on multidimensional feature is adapted score calculation, based on comprehensive adaptation score from candidate diagnosis and treatment scheme Screening out recommended scheme, and matching required medical resources, through to patient state and resource state monitoring is adjusted and resource collaborative scheduling is carried out diagnosis and treatment path;The present application fully excavates the complex interaction between patients, diagnosis and treatment scheme and medical resources by multimodal fusion technology, improves the accuracy of comprehensive adaptation evaluation with the help of special model, combined with cross-agency collaborative scheduling and dynamic adjustment mechanism, effectively solves the problem of insufficient utilization of multimodal data, insufficient scheme adaptability, inefficient resource scheduling and lack of flexible adjustment of diagnosis and treatment path, significantly improves the individualization level of rare disease diagnosis and treatment, resource utilization efficiency and timeliness and rationality of diagnosis and treatment decision.

Description

Technical Field

[0001] This invention relates to the field of medical information technology, and in particular to a method and system for collaborative scheduling of rare disease diagnosis and treatment resources based on multimodal fusion. Background Technology

[0002] Currently, the planning of diagnosis and treatment pathways and the collaborative management of resources for rare diseases mainly rely on traditional manual decision-making models, which have the following technical bottlenecks:

[0003] Treatment pathway selection is highly subjective: existing technologies rely heavily on the experience of individual experts and lack a systematic decision-making framework. Studies show that the consistency of treatment plans for the same rare disease patient across different hospitals is only 53%, leading to frequent changes in treatment plans when patients move between different medical institutions. Traditional methods fail to transform multi-dimensional medical data into quantitative decision indicators, making the treatment plan selection process difficult to standardize and verify.

[0004] Multimodal medical data integration is challenging: Rare disease diagnosis and treatment involve multimodal data including genomics, clinical manifestations, imaging, and pathology. Existing electronic health record systems typically manage this information by storing text and structured data separately. Statistics show that a typical rare disease patient's file contains more than 20 different data files, resulting in a data utilization rate of less than 30%. Existing fusion methods often employ simple splicing or early fusion strategies, failing to fully capture the complex semantic relationships between modalities.

[0005] The efficiency of medical resource collaboration is low: Rare disease diagnosis and treatment resources are extremely unevenly distributed, with 85% of specialized resources concentrated in tertiary hospitals, while primary healthcare institutions lack the necessary diagnostic equipment and expertise. Existing resource allocation systems are mostly static, with resource utilization rates of only 45%-60%, and lack dynamic adjustment mechanisms based on the urgency of patient conditions and real-time resource status. Cross-institutional collaboration processes typically take 7-14 days to complete resource coordination, severely delaying timely diagnosis and treatment of rare diseases.

[0006] The disconnect between knowledge updates and clinical practice is a significant issue: The field of rare diseases is rapidly evolving, with over 3,000 related studies published annually. However, the average time for these latest advances to translate into clinical practice is 3-5 years. Existing decision support systems often rely on fixed rule bases, resulting in long knowledge update cycles and an inability to promptly integrate the latest research findings. The consistency between system-recommended protocols and the latest clinical guidelines is only 68%, leading to insufficient evidence-based support for these recommendations.

[0007] Lack of dynamic adjustment and feedback mechanisms: Changes in patient condition and treatment response are not effectively incorporated into pathway adjustment decisions. Existing systems mostly provide one-time static recommendations for pathway planning, lacking continuous learning and dynamic optimization capabilities. Studies show that approximately 65% ​​of rare disease patients require adjustments to their initial treatment plan, but existing systems cannot automatically trigger pathway reassessment based on treatment feedback, leading to unsatisfactory treatment outcomes or an increased incidence of adverse reactions.

[0008] Resource allocation lacks quantitative decision-making basis: Existing cross-institutional resource allocation is mostly based on administrative coordination rather than data-driven decision-making, lacking quantitative assessment of the match between patient needs and resource supply. The determination of resource allocation priorities is highly subjective, leading to low efficiency in the use of critical resources and excessively long waiting times for critically ill patients. Data shows that the average waiting time for rare equipment using traditional allocation methods is 14-21 days, while the urgency of actual clinical needs is insufficiently assessed. Summary of the Invention

[0009] The main objective of this invention is to provide a method and system for collaborative scheduling of rare disease diagnosis and treatment resources based on multimodal fusion, aiming to solve the technical problems existing in the prior art, such as difficulty in integrating multimodal medical data, lack of dynamic adjustment and feedback mechanisms, and lack of quantitative decision-making basis for resource scheduling.

[0010] To achieve the above objectives, this invention provides a method for collaborative scheduling of rare disease diagnosis and treatment resources based on multimodal fusion, the method comprising the following steps:

[0011] Multimodal data is collected, preprocessed, and features are extracted from the multimodal data to obtain multidimensional features. The multimodal data includes patient data, treatment plan data, and medical resource data. The multidimensional features include patient features, treatment plan features, and medical resource features. The treatment plan data is obtained based on candidate treatment plans, which are generated based on patient diagnosis results and a rare disease knowledge base.

[0012] The multidimensional features are input into a pre-trained adaptation score evaluation model, which outputs the patient's comprehensive adaptation score. The adaptation score evaluation model is a three-modal Transformer architecture, which includes a multimodal encoder, a cross-modal fusion layer, and an adaptation score calculation layer. The multimodal encoder includes a patient encoder, a protocol encoder, and a resource encoder. The cross-modal fusion layer is configured to fuse the encoding vectors of each modality output by the multimodal encoder through an attention mechanism. The adaptation score calculation layer is configured to calculate the comprehensive adaptation score based on the cross-modal interaction features output by the cross-modal fusion layer.

[0013] At least one recommended treatment plan is selected from the candidate treatment plans based on the comprehensive fit score;

[0014] Match the medical resources required for the recommended scheme, perform cross-institutional resource coordination and scheduling, and monitor patient data and medical resource data;

[0015] In response to the monitoring results meeting the trigger conditions, the treatment pathway is dynamically adjusted based on the treatment plan data, as well as the currently monitored patient data and medical resource data, and the treatment resources are coordinated and scheduled based on the dynamically adjusted treatment pathway.

[0016] Optionally, the feature extraction process for the patient features includes:

[0017] Historical text sequences are extracted from the patient's electronic medical record text, and the historical text sequences are input into a pre-trained text feature extraction model to output text feature sub-vectors. The text feature extraction model is a BioBERT model based on a multi-layer Transformer architecture.

[0018] Gene sequences are extracted from the patient's gene sequencing data and encoded using an attention-weighted one-hot encoder to output gene feature vectors.

[0019] Candidate time series are extracted from the patient's medical history time series data, and the candidate time series are input into a pre-trained bidirectional long short-term memory network model to output time series feature sub-vectors;

[0020] The text feature vectors, gene feature vectors, and temporal feature vectors are fused using an intramodal attention mechanism to obtain patient features.

[0021] Optionally, the feature extraction process for the diagnostic and treatment plan includes:

[0022] Entity sets and relation sets are extracted from the diagnostic knowledge graph, and the TransE model is used to learn and embed the entity sets and relation sets to obtain knowledge graph feature sub-vectors.

[0023] A set of research literature abstracts related to candidate treatment plans is obtained, and the set of research literature abstracts is input into a literature feature extraction model to output literature feature sub-vectors. The literature feature extraction model is a PubMedBERT model based on a multi-layer Transformer architecture. The literature feature extraction model is pre-trained based on biomedical text data.

[0024] Statistical feature vectors of efficacy distribution, efficacy uncertainty feature vectors, and adverse reaction risk feature vectors are extracted from clinical trial data of candidate treatment options, and effect feature sub-vectors are generated based on the statistical feature vectors, efficacy uncertainty feature vectors, and adverse reaction risk feature vectors.

[0025] The knowledge graph feature vector, the document feature vector, and the effect feature vector are fused using an intramodal attention mechanism to obtain the treatment plan features.

[0026] Optionally, the feature extraction process for the medical resource features includes:

[0027] Spatial feature subvectors are extracted from the geographic location data of medical institutions associated with candidate treatment plans. The geographic location data includes the latitude and longitude coordinates of the medical institutions and the straight-line distance between the patient's current location and the medical institutions.

[0028] Based on the equipment configuration vector, physician expertise vector, and departmental load vector of the medical institution, a sub-vector of diagnostic and treatment capabilities is generated. The equipment configuration vector is determined based on the presence or absence of key equipment in the medical institution. The physician expertise vector is constructed based on the topics of papers published by physicians in the medical institution and their target clinical fields. The departmental load vector is generated based on the current number of patients, the ratio of medical staff to nurses, and the bed occupancy rate of the medical institution.

[0029] Based on the waiting time feature vector and referral cost feature vector of the medical institution, a feature vector of accessibility of medical resources is generated.

[0030] The spatial feature sub-vector, the diagnostic and treatment capability feature sub-vector, and the accessibility feature sub-vector are fused using an intramodal attention mechanism to obtain medical resource features.

[0031] Optionally, the patient encoder is a BioBERT architecture, configured to encode patient features into a patient encoding vector;

[0032] The scheme encoder is a multi-layer Transformer architecture, configured to encode the diagnostic and treatment scheme features into a scheme encoding vector;

[0033] The resource encoder is a multi-layer Transformer and LSTM combined architecture, configured to perform spatial attention calculation and temporal modeling on medical resource features to encode the medical resource features into resource encoding vectors.

[0034] The cross-modal fusion layer includes a bimodal attention module and a trimodal attention module;

[0035] The bimodal attention module is used to perform cross-modal fusion of the patient encoding vector and the protocol encoding vector to output patient-protocol interaction features, as shown in the following formula:

[0036]

[0037] in, This represents the patient-treatment interaction characteristics obtained after the fusion of bimodal attentional interactions between the patient and the treatment plan. This represents the cross-modal attention computation function. Represents the patient coding vector. Represents the scheme encoding vector, Indicates the dimension of interaction features;

[0038] The bimodal attention module is also used to perform cross-modal fusion of the scheme encoding vector and the resource encoding vector to output the scheme-resource interaction feature, as shown in the following formula:

[0039]

[0040] in, This represents the solution-resource interaction characteristics obtained after the treatment plan and medical resources are fused through bimodal attention interaction. Represents a resource encoding vector;

[0041] The trimodal attention module is used to perform cross-modal fusion of patient encoding vectors, protocol encoding vectors, and resource encoding vectors to output trimodal interaction features, as shown in the following formula:

[0042]

[0043] in, This represents the trimodal interaction feature obtained after the trimodal attention interaction fusion of the patient encoding vector, the protocol encoding vector, and the resource encoding vector. This represents a trimodal attention computation function, used to realize the interaction between trimodal features.

[0044] Optionally, the adaptation score calculation layer is further configured to calculate the patient-plan adaptation score based on patient-plan interaction features, referring to the following formula:

[0045]

[0046] in, This indicates the patient-protocol fit score. This represents a multilayer perceptron module used to calculate patient-protocol fit scores, containing two hidden layers with ReLU activation function, and the output layer scaled to the range [1, 10] using the Sigmoid function;

[0047] The adaptation score calculation layer is further configured to calculate the scheme-resource adaptation score based on the scheme-resource interaction features, referring to the following formula:

[0048]

[0049] in, The solution-resource adaptation score is represented. This represents a multilayer perceptron module used to calculate scheme-resource adaptation scores. It contains two hidden layers, uses the ReLU activation function, and scales the output layer to the range [1, 10] using the Sigmoid function.

[0050] The adaptation score calculation layer is also configured to calculate the trimodal co-adaptation score based on trimodal interaction features, as shown in the following formula:

[0051]

[0052] in, This represents the trimodal co-adaptation score. This represents a multilayer perceptron module used to compute the trimodal co-adaptation score between patient modality, protocol modality, and resource modality. It contains three hidden layers, uses the GELU activation function, and scales the output layer to the range [1,10] using the Sigmoid function.

[0053] The adaptation score calculation layer is further configured to calculate a comprehensive adaptation score based on the patient-solution adaptation score, the solution-resource adaptation score, and the trimodal collaborative adaptation score, referring to the following formula:

[0054]

[0055] in, This indicates the overall fit score. This indicates the patient-treatment fit weight. Representation scheme - resource adaptation weight, This represents the weights for trimodal collaborative adaptation.

[0056] Optionally, the adaptation score evaluation model is trained using a hybrid loss function, the mathematical expression of which is as follows:

[0057]

[0058] in, Represents the mixed loss term. This represents the mean squared error loss term. This indicates the ranking loss item. This represents the modal alignment loss term. Represents the regularization loss term. This represents the weighting coefficient of the ranking loss term. This represents the weighting coefficients of the modal alignment loss term. Represents the weight coefficients of the regularization loss term;

[0059] The mathematical expression for the mean squared error loss term is as follows:

[0060]

[0061] in, The model represents the first The prediction fit score for each sample. Indicates the first The true fit score for each sample is determined based on the gold standard score annotated by experts. Indicates the number of training samples;

[0062] The ranking loss term is used to ensure that the model can correctly rank the suitability of different solutions. The mathematical expression of the ranking loss term is as follows:

[0063]

[0064] in, and They represent the model for the first... The first sample and the first The prediction fit score for each sample. and They represent the first The first sample and the first The true fit score of each sample Represents a symbolic function;

[0065] The modality alignment loss term is used to promote the alignment of feature spaces of different modalities, and the mathematical expression of the modality alignment loss term is as follows:

[0066]

[0067] in, Indicates the first Patient modal feature vectors of each sample Indicates the first Modal feature vectors of treatment plans for each sample. Indicates the first The modal feature vector of medical resources for each sample. This represents the Euclidean norm, used to calculate the length of a vector.

[0068] The regularization loss term is used to prevent the fit score evaluation model from overfitting. The mathematical expression of the regularization loss term is as follows:

[0069]

[0070] in, This represents a single learnable parameter in the model. This represents the set of all learnable parameters of the model.

[0071] Optionally, the response to the monitoring result meeting the trigger condition, dynamically adjusting the treatment pathway based on the treatment plan data and the currently monitored patient data and medical resource data, and coordinating the scheduling of treatment resources based on the dynamically adjusted treatment pathway, includes:

[0072] Based on treatment plan data, as well as currently monitored patient and medical resource data, the dynamic adjustment process of treatment pathways is modeled as a Markov decision process, and the value function is defined as follows:

[0073]

[0074]

[0075] in, express The state space at any given time includes the patient's state and the resource state. express Always in the zone The value function under, express Always in the zone Select action Instant rewards Indicates from action space The action of selecting a treatment plan express The action space at any given moment contains the set of currently available treatment options. Indicates the patient's condition. Indicates resource status. express The patient's status at any given time is Choosing a treatment plan Resource status is The corresponding comprehensive adaptation score at that time This represents a discount factor used to balance current rewards with future rewards. Indicates in Moment State After making a selection, Time-value function The expected value of the condition;

[0076] The triggering condition is that the change in the comprehensive adaptation score exceeds the adaptation score change threshold or the change in resource status exceeds the resource status change threshold.

[0077] The value function is solved, and the treatment pathway is dynamically adjusted based on the solution. The value function is solved using a deep Q-network optimal solution strategy, as shown in the following formula:

[0078]

[0079] in, Represents the state at time t Select action Q-value function, express A treatment plan selected from the action space at any given moment. express Moment State Select action Q-value function;

[0080] Resource scheduling priority is calculated based on the dynamically adjusted treatment pathway, referring to the following formula:

[0081]

[0082] in, Indicates the patient Resources Appointment priority, This represents the patient-resource fit score. This represents the scheduling cost, which includes time cost and economic cost. Indicates the current utilization rate of resources. This indicates the patient-resource matching weight. Indicates the scheduling cost weight. Indicates the weight of resource utilization rate;

[0083] Coordinated scheduling of medical resources based on resource scheduling priorities.

[0084] Optionally, the resource conflict resolution function is used to resolve resource conflicts during the collaborative scheduling process of medical resources. The mathematical expression of the target conflict resolution function is as follows:

[0085]

[0086] The constraints of the target conflict resolution function are:

[0087]

[0088] in, This indicates finding the minimum value of the set of assignable variables X. This represents the balance coefficient, used to balance scheduling costs and reservation priorities. This represents an assigned variable used to characterize the patient. Resources allocated ,like This means that resources will be allocated. Assigned to patients ,like This means not to allocate resources. Assigned to patients .

[0089] Furthermore, to achieve the above objectives, this invention also proposes a collaborative scheduling system for rare disease diagnosis and treatment resources based on multimodal fusion, wherein the collaborative scheduling system for rare disease diagnosis and treatment resources based on multimodal fusion includes:

[0090] A multimodal data processing module is used to collect multimodal data, preprocess the multimodal data and extract features to obtain multidimensional features. The multimodal data includes patient data, treatment plan data and medical resource data. The multidimensional features include patient features, treatment plan features and medical resource features. The treatment plan data is obtained based on candidate treatment plans, which are generated based on patient diagnosis results and a rare disease knowledge base.

[0091] The adaptation score evaluation module is used to input the multidimensional features into a pre-trained adaptation score evaluation model and output the patient's comprehensive adaptation score. The adaptation score evaluation model is a three-modal Transformer architecture, which includes a multimodal encoder, a cross-modal fusion layer, and an adaptation score calculation layer. The multimodal encoder includes a patient encoder, a protocol encoder, and a resource encoder. The cross-modal fusion layer is configured to perform cross-modal fusion of the encoding vectors of each modality output by the multimodal encoder through an attention mechanism. The adaptation score calculation layer is configured to calculate the comprehensive adaptation score based on the cross-modal interaction features output by the cross-modal fusion layer.

[0092] The treatment plan selection module is used to select at least one recommended treatment plan from the candidate treatment plans based on the comprehensive fit score.

[0093] The resource matching module is used to match the medical resources required by the recommended plan, perform cross-institutional resource collaborative scheduling, and monitor patient data and medical resource data.

[0094] The resource coordination and scheduling module is used to respond to the monitoring results meeting the trigger conditions, dynamically adjust the treatment path based on the treatment plan data and the currently monitored patient data and medical resource data, and coordinate the scheduling of treatment resources based on the dynamically adjusted treatment path.

[0095] This invention collects multimodal data including patient data, treatment plan data, and medical resource data. The multimodal data is preprocessed and feature extracted to obtain corresponding multidimensional features. The treatment plan data is obtained based on candidate treatment plans generated from patient diagnosis results and a rare disease knowledge base. These multidimensional features are input into a pre-trained three-modal Transformer architecture adaptation score evaluation model. The model uses a multimodal encoder to encode each modal feature, a cross-modal fusion layer to fuse the encoded vectors using an attention mechanism, and an adaptation score calculation layer to calculate a comprehensive adaptation score based on cross-modal interaction features. At least one recommended treatment plan is selected from the candidate plans based on the comprehensive adaptation score. The recommended treatment plan is then matched with the required medical resources. This invention enables cross-institutional resource coordination and scheduling, while continuously monitoring patient and medical resource data. When monitoring results meet trigger conditions, the treatment pathway is dynamically adjusted based on current monitoring data and treatment plan data, and treatment resources are coordinated and scheduled according to the adjusted treatment pathway. Because this invention fully explores the complex relationships between patients, treatment plans, and medical resources through multimodal fusion technology, improves the accuracy of comprehensive adaptation assessment with the help of a dedicated model, and effectively solves the problems of insufficient data utilization, inadequate plan adaptability, inefficient resource scheduling, and lack of flexible adjustment of treatment pathways in existing technologies, it significantly improves the personalization level, resource utilization efficiency, and timeliness and rationality of treatment decisions for rare diseases. Attached Figure Description

[0096] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0097] Figure 1 This is a schematic diagram of the structure of a rare disease diagnosis and treatment resource collaborative scheduling device based on multimodal fusion in the hardware operating environment of the embodiment of the present invention;

[0098] Figure 2 This is a flowchart illustrating the first embodiment of the rare disease diagnosis and treatment resource collaborative scheduling method based on multimodal fusion of the present invention;

[0099] Figure 3 This is a flowchart illustrating the second embodiment of the rare disease diagnosis and treatment resource collaborative scheduling method based on multimodal fusion of the present invention;

[0100] Figure 4 This is a flowchart illustrating the third embodiment of the rare disease diagnosis and treatment resource collaborative scheduling method based on multimodal fusion of the present invention;

[0101] Figure 5This is a structural block diagram of the first embodiment of the rare disease diagnosis and treatment resource collaborative scheduling system based on multimodal fusion of the present invention.

[0102] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0103] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0104] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of a rare disease diagnosis and treatment resource collaborative scheduling device based on multimodal fusion in the hardware operating environment of the embodiment of the present invention.

[0105] like Figure 1 As shown, the rare disease diagnosis and treatment resource collaborative scheduling device based on multimodal fusion may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to realize communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk storage device. Optionally, the memory 1005 may also be a storage system independent of the aforementioned processor 1001.

[0106] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the collaborative scheduling device for rare disease diagnosis and treatment resources based on multimodal fusion. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0107] like Figure 1 As shown, the memory 1005, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a rare disease diagnosis and treatment resource collaborative scheduling program based on multimodal fusion.

[0108] exist Figure 1In the multimodal fusion-based rare disease diagnosis and treatment resource collaborative scheduling device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and memory 1005 in the multimodal fusion-based rare disease diagnosis and treatment resource collaborative scheduling device of the present invention can be set in the multimodal fusion-based rare disease diagnosis and treatment resource collaborative scheduling device. The multimodal fusion-based rare disease diagnosis and treatment resource collaborative scheduling device calls the multimodal fusion-based rare disease diagnosis and treatment resource collaborative scheduling program stored in the memory 1005 through the processor 1001, and executes the multimodal fusion-based rare disease diagnosis and treatment resource collaborative scheduling method provided in the embodiment of the present invention.

[0109] This invention provides a method for collaborative scheduling of rare disease diagnosis and treatment resources based on multimodal fusion, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the rare disease diagnosis and treatment resource collaborative scheduling method based on multimodal fusion of the present invention.

[0110] In this embodiment, the method for collaborative scheduling of rare disease diagnosis and treatment resources based on multimodal fusion includes the following steps:

[0111] Step S10: Collect multimodal data, preprocess the multimodal data and extract features to obtain multidimensional features.

[0112] It should be noted that this embodiment is applied to the screening of rare disease treatment plans, dynamic adjustment of treatment pathways, and collaborative scheduling of treatment resources. To address the shortcomings of existing technologies, this invention aims to solve the following technical problems:

[0113] The problem of quantitative modeling of multimodal medical entity adaptability: How to formalize the complex rare disease diagnosis and treatment decision-making process into a computable mathematical problem, and achieve accurate quantification of the adaptability among the three-dimensional entities of "patient-treatment plan-medical resources". Existing technologies lack a unified adaptability assessment framework, failing to transform the qualitative judgments of clinical experts into quantitative indicators, making it difficult to standardize and automate the decision-making process. This invention aims to construct a scientific adaptability assessment system that quantifies the degree of matching among the three entities through multi-dimensional indicators, providing an objective basis for personalized treatment pathway planning.

[0114] Feature engineering and fusion of multi-source heterogeneous medical data: Rare disease diagnosis and treatment data are characterized by multimodality, high dimensionality, and strong heterogeneity, including gene sequence data, text medical records, imaging data, and time-series monitoring data. How to extract effective features from these heterogeneous data and perform deep fusion is a key challenge in building accurate diagnosis and treatment decision-making models. This invention requires designing feature extraction methods for different modalities and fusion strategies capable of capturing complex correlations between modalities, fully utilizing complementary information from multi-source data, and improving the model's ability to understand the complex clinical manifestations of rare diseases.

[0115] The mathematical modeling problem of dynamic treatment pathway planning: The diagnosis and treatment process of rare diseases is highly dynamic, with changes in patient condition, treatment response, and medical resource status all changing over time. How to construct a pathway planning model that can adapt to these dynamic changes and achieve real-time optimization and adjustment of treatment plans is a core problem that existing static planning methods have failed to solve. This invention requires establishing a dynamic programming model, incorporating time factors into the decision-making framework, and designing a pathway adjustment trigger mechanism based on state changes to ensure that the recommended plan always maintains an optimal match with the patient's current state and resource availability.

[0116] The optimization problem of cross-institutional medical resource collaborative scheduling: Rare disease diagnosis and treatment often requires collaboration among multiple institutions, but existing resource scheduling systems lack a global optimization perspective and dynamic coordination mechanism. How to achieve efficient collaboration and optimized allocation of cross-institutional resources while ensuring the quality of diagnosis and treatment is key to improving the accessibility of rare disease diagnosis and treatment. This invention requires the design of a resource optimization algorithm that considers multiple objectives such as the urgency of the patient's condition, resource utilization rate, and scheduling cost, as well as a collaborative mechanism capable of resolving resource conflicts, thereby improving the efficiency and fairness of cross-institutional resource sharing.

[0117] The integration of knowledge-enhanced learning with clinical practice: Decision-making in the diagnosis and treatment of rare diseases requires the integration of basic medical knowledge, clinical guidelines, the latest research progress, and practical experience. How to construct a learning framework that can effectively integrate this multi-source knowledge, enabling the model to continuously absorb new knowledge and apply it to clinical decision-making, is a significant challenge in improving the evidence-based level of recommendations. This invention requires designing a knowledge-enhanced model architecture to achieve deep integration of medical knowledge and data-driven models, while establishing a knowledge update mechanism to ensure that the system's recommendations remain synchronized with the latest medical advancements.

[0118] Uncertainty Quantification and Decision Risk Control: Decision-making in the diagnosis and treatment of rare diseases faces high uncertainty, including diagnostic uncertainty, treatment efficacy uncertainty, and prognostic uncertainty. Existing systems often provide deterministic recommendations, lacking the quantification of uncertainty and the assessment of decision risks. This invention aims to construct an uncertainty quantification framework to provide confidence assessments and risk warnings for recommended solutions, assisting clinicians in weighing treatment benefits against potential risks in complex decision-making environments and improving the robustness of decisions.

[0119] It should be understood that the executing entity of this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or a terminal electronic device capable of realizing the above functions. The following description uses a multimodal fusion-based rare disease diagnosis and treatment resource collaborative scheduling device (hereinafter referred to as the scheduling device) as an example to illustrate this embodiment and the following embodiments.

[0120] It should be noted that the multimodal data includes patient data, treatment plan data, and medical resource data, and the multidimensional features include patient features, treatment plan features, and medical resource features. The treatment plan data is obtained based on candidate treatment plans, which are generated based on patient diagnosis results and a rare disease knowledge base.

[0121] In a specific implementation, the scheduling device receives patient multimodal data, which may include:

[0122] Structured data: demographic information, laboratory test results, diagnostic codes, etc.;

[0123] Text data: electronic medical records, progress notes, discharge summaries, etc.;

[0124] Genetic data: gene sequencing reports, variant site annotations, etc.;

[0125] Imaging data: X-ray, CT, MRI and other imaging reports (text description section).

[0126] In some embodiments, data preprocessing of multimodal data may include:

[0127] Text preprocessing: word segmentation, stop word removal, and standardization of medical terminology (based on UMLS dictionary);

[0128] Structured data processing: missing value imputation (using multiple imputation based on the MICE algorithm), outlier detection and correction (IQR rule), feature standardization ( -score);

[0129] Time series data processing: timestamp alignment, interpolation of missing time points, and extraction of time series features (trends, periodicity, etc.).

[0130] Step S20: Input the multidimensional features into the pre-trained fitting score evaluation model and output the patient's comprehensive fitting score.

[0131] It should be noted that the adaptation score evaluation model is a three-modal Transformer architecture. The adaptation score evaluation model includes a multimodal encoder, a cross-modal fusion layer, and an adaptation score calculation layer. The multimodal encoder includes a patient encoder, a protocol encoder, and a resource encoder. The cross-modal fusion layer is configured to perform cross-modal fusion of the encoding vectors of each modality output by the multimodal encoder through an attention mechanism. The adaptation score calculation layer is configured to calculate a comprehensive adaptation score based on the cross-modal interaction features output by the cross-modal fusion layer.

[0132] In some embodiments, the scheduling device may use the Monte Carlo dropout method to calculate the model fit score prediction uncertainty:

[0133]

[0134]

[0135] in, Indicates the first The result of the dropout forward propagation This represents the total number of forward propagations in the Monte Carlo dropout method. . Let S(m) be the mean of the M forward propagations, serving as the core reference value for the overall fit score. The standard deviation of S(m) obtained from M forward propagations is used to measure the dispersion of multiple scoring results.

[0136] Final output fits the score range .

[0137] Step S30: Select at least one recommended treatment plan from the candidate treatment plans based on the comprehensive fit score.

[0138] In some embodiments, the generation of candidate treatment plans specifically includes: generating a set of candidate treatment plans based on the patient's diagnosis results and a rare disease knowledge base. ,generally .

[0139] In the specific implementation, the scheduling device calculates and ranks the adaptation score for each candidate solution using an adaptation score evaluation model:

[0140]

[0141] Determine the recommended solution based on the score threshold:

[0142] Main solution: Score There are usually 1-2 options;

[0143] Alternative solutions: Score There are usually 2-3 options;

[0144] Not recommended option: Score The solution.

[0145] Step S40: Match the medical resources required by the recommended plan, perform cross-institutional resource collaborative scheduling, and monitor patient data and medical resource data.

[0146] In practice, the scheduling equipment recommends a plan, and the system automatically queries and matches the required medical resources.

[0147]

[0148] in This indicates the recommended primary solution.

[0149] Step S50: In response to the monitoring results meeting the triggering conditions, dynamically adjust the treatment path based on the treatment plan data and the currently monitored patient data and medical resource data, and coordinate the scheduling of treatment resources based on the dynamically adjusted treatment path.

[0150] In practice, the scheduling equipment can update resource status information every 6 hours and monitor changes in the patient's condition: resource status updates include equipment availability, staff load, bed availability, etc. - patient status updates include new examination results, changes in symptoms, treatment response, etc.

[0151] When the patient's or resource status meets the triggering conditions, the path reassessment process is automatically initiated:

[0152] 1. Recalculate the patient-protocol fit score;

[0153] 2. Evaluate whether the current solution is still the optimal choice;

[0154] 3. If adjustments are needed, generate new solution recommendations and resource scheduling plans;

[0155] 4. Notify relevant medical institutions and healthcare personnel;

[0156] Treatment efficacy feedback: The scheduling device can record treatment efficacy and adverse reactions for continuous model optimization.

[0157]

[0158] in For actual treatment effect, To predict the effect, These are actual adverse reactions that occurred. This feedback data is used to periodically update model parameters and improve prediction accuracy.

[0159] The triggering condition mentioned above is that the change in the overall adaptation score exceeds the adaptation score change threshold or the change in resource status exceeds the resource status change threshold, as shown in the following formula:

[0160] or

[0161] in, This indicates the updated overall compatibility score. This indicates the overall compatibility score before the update. Indicates the threshold for changes in adaptation score, for example ; Indicates the threshold for resource status changes, for example ;

[0162] : Changes in resource status.

[0163] In some embodiments, when local resources are insufficient, an inter-agency coordination process is initiated:

[0164] The system queries available resources in the regional medical collaboration network; calculates patient-resource matching scores and scheduling costs; sorts resources according to scheduling priority; and generates resource scheduling instructions, including information such as time, location, and personnel allocation.

[0165] In some embodiments, the scheduling device outputs the following results:

[0166] 1. The recommended treatment plan list includes: main plan and alternative plans. Each plan includes: Plan description: specific examination items, treatment drugs, surgical methods, etc.; Fit score: overall score and subdivided scores for each dimension; Evidence basis: key research literature and guideline citations supporting the plan; Expected effect: expected efficacy and possible adverse reactions; Uncertainty interval: 95% confidence interval of the score.

[0167] 2. The resource allocation plan includes: Time arrangement: the specific time for each treatment step; Location allocation: the implementing agency, department and address; Personnel information: the responsible medical staff and their contact information; Transportation and accommodation suggestions (for patients seeking medical treatment across regions).

[0168] 3. The dynamic adjustment guidelines include: conditions and thresholds for triggering recalculation; patient self-monitoring indicators and recording methods; emergency contact information and handling procedures.

[0169] 4. Decision support information includes: comparative analysis of different options (tables and visualizations); weight distribution of key decision factors; treatment pathways and outcomes of similar cases (de-identified).

[0170] This embodiment fully explores the complex relationships between patients, treatment plans, and medical resources through multimodal fusion technology. It improves the accuracy of comprehensive adaptation assessment by using a dedicated model and combines cross-institutional collaborative scheduling and dynamic adjustment mechanisms. This effectively solves the problems of insufficient data utilization, inadequate plan adaptability, inefficient resource scheduling, and lack of flexible adjustment of treatment pathways in existing technologies. It significantly improves the personalization level of rare disease diagnosis and treatment, resource utilization efficiency, and the timeliness and rationality of treatment decisions.

[0171] refer to Figure 3 , Figure 3 This is a flowchart illustrating the second embodiment of the rare disease diagnosis and treatment resource collaborative scheduling method based on multimodal fusion of the present invention.

[0172] Based on the first embodiment described above, in this embodiment, the process of feature extraction from multimodal data specifically includes: extracting features from patient features, treatment plan features, and medical resource features.

[0173] The feature extraction process for the patient's characteristics includes:

[0174] Step S111: Extract historical text sequences from the patient's electronic medical record text, and input the historical text sequences into a pre-trained text feature extraction model to output text feature sub-vectors. The text feature extraction model is a BioBERT model based on a multi-layer Transformer architecture.

[0175] Step S112: Extract gene sequences from the patient's gene sequencing data and encode the gene sequences using an attention-weighted one-hot encoder to output gene feature vectors;

[0176] Step S113: Extract candidate time series from the patient's medical history time series data, and input the candidate time series into a pre-trained bidirectional long short-term memory network model to output time series feature sub-vectors;

[0177] Step S114: The text feature vector, gene feature vector and temporal feature vector are fused using an intramodal attention mechanism to obtain patient features.

[0178] It should be noted that the patient feature vector It consists of three sub-vectors: text features, genetic features, and temporal features.

[0179]

[0180] Text features ( Semantic features extracted from patient electronic medical record text are embedded using a BioBERT-based pre-trained model:

[0181]

[0182] in This represents the preprocessed sequence of medical record text, including chief complaint, present illness, past medical history, physical examination, etc. The model employs a 12-layer Transformer architecture, 768-dimensional hidden states, and a 12-head self-attention mechanism.

[0183] Genetic characteristics ( ): Variance features extracted from gene sequencing data are encoded using attention-weighted one-hot encoding:

[0184]

[0185] in:

[0186] Indicates length is The gene sequence, each ;

[0187] One-hot encoding representing a gene sequence;

[0188] The clinical significance weight matrix of variant sites is constructed based on the ClinVar database and expert knowledge.

[0189]

[0190] in , ;

[0191] Temporal characteristics ( Dynamic features extracted from medical history time series data are modeled using a bidirectional LSTM model.

[0192]

[0193] in:

[0194] Indicates length is Time-series data of medical history, each Includes timestamps and corresponding clinical indicators at the time points;

[0195] BiLSTM contains two hidden layers, each with 128 neurons, and uses dropout regularization. To prevent overfitting;

[0196] The output is the hidden state of the last time step, which serves as a comprehensive representation of the medical history timeline;

[0197] Patient Feature Fusion: Fusing three sub-features through an intramodal attention mechanism.

[0198]

[0199]

[0200]

[0201] in, represents a linear transformation layer used to map the 1536-dimensional Pintra to a 768-dimensional feature space; MultiHeadAttention represents a multi-head self-attention mechanism, which uses 8 attention heads, each with a dimension of 192 (1536 / 8).

[0202] Furthermore, the feature extraction process for the diagnostic and treatment plan includes:

[0203] Step S121: Extract entity sets and relation sets from the diagnostic knowledge graph, and use the TransE model to learn and embed the entity sets and relation sets to obtain knowledge graph feature sub-vectors;

[0204] Step S122: Obtain a set of research literature abstracts related to candidate treatment plans, and input the set of research literature abstracts into the literature feature extraction model to output literature feature sub-vectors. The literature feature extraction model is a PubMedBERT model based on a multi-layer Transformer architecture. The literature feature extraction model is pre-trained based on biomedical text data.

[0205] Step S123: Extract the statistical feature vector of efficacy distribution, the feature vector of efficacy uncertainty, and the feature vector of adverse reaction risk from the clinical trial data of the candidate treatment plan, and generate an effect feature sub-vector based on the statistical feature vector, the feature vector of efficacy uncertainty, and the feature vector of adverse reaction risk;

[0206] Step S124: The knowledge graph feature sub-vector, the document feature sub-vector, and the effect feature sub-vector are fused using an intramodal attention mechanism to obtain the treatment plan features.

[0207] It should be noted that the feature vector of the diagnosis and treatment plan It consists of three sub-vectors: knowledge graph feature sub-vector, document feature sub-vector, and effect feature sub-vector.

[0208]

[0209] in, Represents the feature vector of the treatment plan. Represents the feature sub-vectors of a knowledge graph. Represents the feature vector of the document. Represents the effect feature sub-vector;

[0210] Knowledge graph feature sub-vectors ( Structured features extracted from the diagnostic knowledge graph are used to learn embeddings using the TransE algorithm.

[0211]

[0212] in:

[0213] It represents a set of entities in a knowledge graph, including nodes such as diseases, drugs, examinations, and symptoms;

[0214] Represents a set of relationships between entities, such as "treatment", "cause", "diagnostic basis", etc.

[0215] The TransE model optimizes the objective function. Learning entity and relation embeddings, where , ;

[0216] Document feature sub-vectors ( Evidence-based features extracted from relevant research literature were analyzed using the PubMedBERT model.

[0217]

[0218] in, This represents a collection of research literature abstracts related to treatment protocols. The model employs a 12-layer Transformer architecture, is pre-trained on biomedical texts, and fine-tuned on abstract data from CochraneLibrary and PubMedCentral.

[0219] Effect feature subvectors ( Efficacy and risk characteristics extracted from clinical trial data:

[0220]

[0221] in:

[0222] A statistical feature vector representing the distribution of therapeutic effects, including mean, median, quartiles, etc.

[0223] This represents the feature vector indicating the uncertainty of therapeutic efficacy, including standard deviation, confidence interval, etc.

[0224] This represents the adverse reaction risk feature vector, based on the CTCAE adverse reaction grading standard coding.

[0225] Feature fusion scheme: Three sub-features are fused using an intra-modal attention mechanism.

[0226]

[0227]

[0228] in MultiHeadAttention uses 8 heads, each with 176 dimensions (1408 / 8).

[0229] Furthermore, the feature extraction process for the medical resource characteristics includes:

[0230] Step S131: Extract spatial feature sub-vectors from the geographic location data of medical institutions associated with candidate treatment plans, wherein the geographic location data includes the latitude and longitude coordinates of the medical institutions and the straight-line distance between the patient's current location and the medical institutions;

[0231] Step S132: Generate a sub-vector of diagnostic and treatment capabilities based on the equipment configuration vector, physician expertise vector, and departmental load vector of the medical institution. The equipment configuration vector is determined based on the presence or absence of key equipment in the medical institution. The physician expertise vector is constructed based on the topics of papers published by physicians in the medical institution and their target clinical fields. The departmental load vector is generated based on the current number of patients, the ratio of medical staff to nurses, and the bed occupancy rate of the medical institution.

[0232] Step S133: Generate a feature vector for the accessibility of medical resources based on the waiting time feature vector and referral cost feature vector of the medical institution;

[0233] Step S134: The spatial feature sub-vector, the diagnostic and treatment capability feature sub-vector, and the accessibility feature sub-vector are fused using an intramodal attention mechanism to obtain medical resource features.

[0234] Medical resource feature vector It consists of three sub-vectors: spatial features, capability features, and accessibility features.

[0235]

[0236] Spatial features ( Spatial features extracted from geographic location data are used for geographic embedding enhanced with a Gaussian kernel function.

[0237]

[0238] in:

[0239] Indicates the latitude and longitude coordinates of the medical institution;

[0240] To represent geographic location embeddings, a method similar to Word2Vec is used, which is trained on a national medical institution coordinate dataset.

[0241] This indicates the straight-line distance between the patient's current location and the medical institution;

[0242] ,in Adjusted adaptively based on the density of regional medical resources;

[0243] Ability characteristics ( ): represents the feature vector of a medical institution's diagnostic and treatment capabilities:

[0244]

[0245] in:

[0246] This represents the device configuration vector, using one-hot encoding to indicate the presence or absence of critical devices.

[0247] The vector representing physician expertise is constructed based on the topics of physicians' published papers and their areas of clinical expertise.

[0248] This represents the departmental load vector, which includes indicators such as the current number of patients, the ratio of medical staff to patients, and the bed occupancy rate.

[0249] Accessibility features ( ): represents a feature vector representing the accessibility of medical resources:

[0250]

[0251] in: This represents a feature vector of waiting time, including appointment waiting time, examination waiting time, and surgery waiting time;

[0252] This represents the feature vector of referral costs, which includes time costs, economic costs, transportation costs, etc.

[0253] Resource Feature Fusion: Three sub-features are fused using an intra-modal attention mechanism.

[0254]

[0255]

[0256] in,

[0257]

[0258] MultiHeadAttention uses 6 heads, each with 149 dimensions (896 / 6).

[0259] This embodiment extracts features from multiple dimensions, including patient, treatment plan, and medical resources. It designs dedicated feature extraction methods for each dimension based on the characteristics of multimodal data of rare diseases, effectively improving data utilization and feature extraction accuracy, and significantly reducing the misdiagnosis rate and diagnosis delay time of rare diseases.

[0260] Furthermore, in order to accurately capture the interaction relationships between multimodal features, in one embodiment, the patient encoder is a BioBERT architecture, configured to encode patient features into patient encoding vectors;

[0261] The scheme encoder is a multi-layer Transformer architecture, configured to encode the diagnostic and treatment scheme features into a scheme encoding vector;

[0262] The resource encoder is a multi-layer Transformer and LSTM combined architecture, configured to perform spatial attention calculation and temporal modeling on medical resource features to encode the medical resource features into resource encoding vectors.

[0263] The cross-modal fusion layer includes a bimodal attention module and a trimodal attention module;

[0264] The bimodal attention module is used to perform cross-modal fusion of patient encoding vectors and protocol encoding vectors to output patient-protocol interaction features;

[0265] The bimodal attention module is also used to perform cross-modal fusion of the scheme encoding vector and the resource encoding vector to output scheme-resource interaction features;

[0266] The trimodal attention module is used to perform cross-modal fusion of patient encoding vectors, protocol encoding vectors, and resource encoding vectors to output trimodal interaction features.

[0267] It should be noted that the patient encoder is coded according to the following formula:

[0268]

[0269] The BioBERT encoder contains 12 Transformer layers, each with the following structure:

[0270]

[0271]

[0272] The FeedForward network contains two linear transformations and the GELU activation function: .

[0273] The encoder uses the following formula for encoding:

[0274]

[0275] This encoder introduces a knowledge graph attention mechanism:

[0276]

[0277] in The knowledge graph entity embedding matrix is ​​represented by the attention weights calculated as follows:

[0278]

[0279]

[0280] The final scheme is coded as follows .

[0281] The resource encoder uses the following formula for encoding:

[0282]

[0283] The encoder consists of two sub-modules: spatial attention and temporal modeling.

[0284] Spatial attention: Location-based self-attention weights:

[0285]

[0286] in This represents a geographic location mask matrix, with penalty terms set based on the actual distance.

[0287] Temporal modeling: LSTM captures resource state changes:

[0288]

[0289] in This represents a time series of resource status.

[0290] It should be noted that the interaction attention analysis of patient-protocol interaction features is performed using the following formula:

[0291]

[0292]

[0293] in, This represents the patient-treatment interaction characteristics obtained after the fusion of bimodal attentional interactions between the patient and the treatment plan. This represents the cross-modal attention computation function. Represents the patient coding vector. Represents the scheme encoding vector, Represents the interaction feature dimension. , , .

[0294] The interaction attention analysis of the resource interaction features of the solution is based on the following formula:

[0295]

[0296]

[0297] in, This represents the solution-resource interaction characteristics obtained after the treatment plan and medical resources are fused through bimodal attention interaction. Represents a resource encoding vector. , .

[0298] Interaction attention analysis based on three-modal interaction features is performed using the following formula:

[0299]

[0300]

[0301] The trimodal attention module introduces modality type embedding. And calculate the interaction attention between the three modalities;

[0302] in, This represents a modal interaction gating mechanism that controls the contribution of different modalities to the fused features. This represents the trimodal interaction feature obtained after the trimodal attention interaction fusion of the patient encoding vector, the protocol encoding vector, and the resource encoding vector. This represents a trimodal attention computation function, used to realize the interaction between trimodal features.

[0303] Furthermore, to improve the accuracy of the overall fit score calculation, in one embodiment, the fit score calculation layer is further configured to calculate the patient-plan fit score based on patient-plan interaction features, referring to the following formula:

[0304]

[0305] in, This indicates the patient-protocol fit score. This represents a multilayer perceptron module used to calculate patient-protocol fit scores, containing two hidden layers with ReLU activation function, and the output layer scaled to the range [1, 10] using the Sigmoid function;

[0306] The adaptation score calculation layer is further configured to calculate the scheme-resource adaptation score based on the scheme-resource interaction features, referring to the following formula:

[0307]

[0308] in, The solution-resource adaptation score is represented. This represents a multilayer perceptron module used to calculate scheme-resource adaptation scores. It contains two hidden layers, uses the ReLU activation function, and scales the output layer to the range [1, 10] using the Sigmoid function.

[0309] The adaptation score calculation layer is also configured to calculate the trimodal co-adaptation score based on trimodal interaction features, as shown in the following formula:

[0310]

[0311] in, This represents the trimodal co-adaptation score. This represents a multilayer perceptron module used to compute the trimodal co-adaptation score between patient modality, protocol modality, and resource modality. It contains three hidden layers, uses the GELU activation function, and scales the output layer to the range [1,10] using the Sigmoid function.

[0312] The adaptation score calculation layer is further configured to calculate a comprehensive adaptation score based on the patient-solution adaptation score, the solution-resource adaptation score, and the trimodal collaborative adaptation score, referring to the following formula:

[0313]

[0314] in, This indicates the overall fit score. This indicates the patient-treatment fit weight. Representation scheme - resource adaptation weight, Represents the weights for trimodal cooperative adaptation, for example The value was determined through a combination of expert Delphi method and cross-validation.

[0315] It is understandable that this embodiment processes different interaction features through layer-by-layer adaptation score calculation. For patient-treatment and treatment-resource interaction features, a multilayer perceptron with corresponding structure is used. For trimodal interaction features, a multilayer perceptron with three hidden layers and GELU activation is used. Adaptation scores for each dimension are calculated and uniformly scaled to the [1,10] interval. Then, a comprehensive adaptation score is obtained by weighting based on the expert Delphi method and cross-validation. This not only accurately adapts to the representation requirements of different interaction features through differentiated model structures and improves the calculation accuracy of adaptation scores for each dimension, but also ensures the standardization and rationality of the comprehensive adaptation score through a unified interval and reasonable weights. This effectively improves the calculation accuracy of the comprehensive adaptation score and provides a more reliable evaluation basis for the subsequent screening of candidate treatment plans and the collaborative scheduling of medical resources.

[0316] Furthermore, to improve model accuracy, in one embodiment, the adaptation score evaluation model is trained using a hybrid loss function, the mathematical expression of which is as follows:

[0317]

[0318] in, Represents the mixed loss term. This represents the mean squared error loss term. This indicates the ranking loss item. This represents the modal alignment loss term. Represents the regularization loss term. This represents the weighting coefficient of the ranking loss term. This represents the weighting coefficients of the modal alignment loss term. These represent the weight coefficients of the regularization loss term; the weight coefficients in the loss function are determined through cross-validation as follows: .

[0319] The mathematical expression for the mean squared error loss term is as follows:

[0320]

[0321] in, The model represents the first The prediction fit score for each sample. Indicates the first The true fit score for each sample is determined based on the gold standard score annotated by experts. Indicates the number of training samples;

[0322] The ranking loss term is used to ensure that the model can correctly rank the suitability of different solutions. The mathematical expression of the ranking loss term is as follows:

[0323]

[0324] in, and They represent the model for the first... The first sample and the first The prediction fit score for each sample. and They represent the first The first sample and the first The true fit score of each sample Represents a symbolic function;

[0325] The modality alignment loss term is used to promote the alignment of feature spaces of different modalities, and the mathematical expression of the modality alignment loss term is as follows:

[0326]

[0327] in, Indicates the first Patient modal feature vectors of each sample Indicates the first Modal feature vectors of treatment plans for each sample. Indicates the first The modal feature vector of medical resources for each sample. This represents the Euclidean norm, used to calculate the length of a vector.

[0328] The regularization loss term is used to prevent the fit score evaluation model from overfitting. The mathematical expression of the regularization loss term is as follows:

[0329]

[0330] in, This represents a single learnable parameter in the model. This represents the set of all learnable parameters of the model.

[0331] Understandably, this embodiment trains the fit score evaluation model using a hybrid loss function that includes mean squared error, ranking, modality alignment, and regularization terms. This not only improves the model's fit score prediction accuracy against the expert gold standard through mean squared error loss, but also ensures the rationality of the fit ranking of different solutions through ranking loss. Simultaneously, modality alignment loss promotes compatibility and interaction among the modal feature spaces of patients, treatment plans, and medical resources, and regularization loss avoids model overfitting. After multi-dimensional collaborative optimization, the model's prediction accuracy, ranking reliability, multi-modal fusion effectiveness, and generalization performance are effectively improved, providing a better evaluation basis for the subsequent rational selection of treatment plans and efficient collaborative scheduling of medical resources.

[0332] In some embodiments, the present invention transforms the rare disease diagnosis and treatment pathway planning and resource coordination problem into a multimodal medical entity adaptability computation problem, formally defined as follows:

[0333] Given triples ,in:

[0334] : A multimodal feature set of patients, including text medical records, genetic data, and time-series medical history;

[0335] A set of diagnostic and treatment plan features, including examination items, treatment drugs, surgical plans, etc.

[0336] A collection of medical resource characteristics, including information on medical institutions, equipment, and medical personnel;

[0337] The objective function is to calculate the comprehensive fit score. :

[0338]

[0339] in:

[0340] Patient-treatment fit score: This indicates the suitability of a treatment plan for a specific patient.

[0341] The solution-resource matching score indicates the degree to which medical resources support the implementation of a specific treatment plan;

[0342] The score for the synergy and adaptation of the three elements represents the overall coordination among the patient, the treatment plan, and the resources.

[0343] Weighting coefficients were determined using the Delphi method by clinical experts.

[0344] For example, 20 experts covering the field of rare diseases were selected to conduct multiple rounds of Delphi method surveys: the first round was anonymous initial assignment, and after feedback on disagreements, a second round of iterative adjustments was made. After the experts' opinions converged (standard deviation of medical resources < medical resources 0.05), the weighting coefficients of medical resources α=0.45, β=0.35, and γ=0.20 were finally determined to meet the clinical and practical needs of rare disease diagnosis and treatment.

[0345] The overall fit score is composed of four dimensions, and the dimensions and their weights are as follows:

[0346]

[0347] in:

[0348] These represent four dimensions: patient fit score, treatment effectiveness, resource accessibility, and cost-effectiveness.

[0349] Let be the dimension weight vector, satisfying and ;

[0350] Experts on the first Dimensional rating;

[0351] The specific components of each dimension are as follows:

[0352] Patient fit score ( ):

[0353]

[0354] in:

[0355] Gene matching score (0-10 points) measures the degree of matching between a patient's gene mutation and the target disease;

[0356] Symptom similarity (0-10 points) measures the similarity between a patient's clinical manifestations and typical symptoms of the disease.

[0357] History fit (0-10 points): measures the degree of fit between the patient's medical history and the trajectory of disease development;

[0358] Indicates gene matching degree, Indicates the weight of symptom similarity. Indicates the degree of fit between medical history and clinical history, for example ;

[0359] Solution effectiveness ( ):

[0360]

[0361] in:

[0362] Evidence-based rating (0-10 points): The level of evidence-based evaluation of the proposed solution, based on the quality of evidence and the strength of recommendation.

[0363] Expected therapeutic effect (0-10 points): A quantitative score of the expected treatment effect;

[0364] : Adverse reaction risk (0-10 points, the lower the risk, the higher the score), assessment of the probability and severity of adverse reactions to the regimen;

[0365] Weights of each indicator;

[0366] Resource accessibility ( ):

[0367]

[0368] in, Institutional qualification matching (0-10 points): The degree to which the medical institution's qualifications meet the requirements for a specific treatment plan; Equipment availability (0-10 points): the current availability status of the required equipment and the waiting time for reservation; Staff workload (0-10 points): The current workload and availability of relevant medical staff; These represent the indicator weights;

[0369] Cost-benefit ratio ( ):

[0370]

[0371] in:

[0372] Treatment cost (0-10 points, the lower the cost, the higher the score), the overall economic cost of the treatment plan;

[0373] Medical insurance coverage (0-10 points): The extent to which medical insurance policies cover the treatment plan;

[0374] Prognostic quality of life (0-10 points): The expected improvement in the patient's quality of life after treatment;

[0375] Weights of each indicator.

[0376] In practical implementation, the methods for collecting expert-annotated gold standard scores may include:

[0377] The annotation expert team consists of 3 rare disease experts (with more than 10 years of clinical experience) and 2 medical resource management experts (with more than 5 years of work experience).

[0378] Scoring rules: A 1-10 point system is used (1 point is the lowest and 10 points is the highest). Each sample is scored independently by 5 experts and the average score is taken as the final gold standard score.

[0379] Consistency test: The intraclass correlation coefficient (ICC) is used to assess the consistency of the annotations.

[0380]

[0381] An ICC value of ≥0.85 is required to ensure labeling reliability.

[0382] Sample size requirements: Covering ≥50 rare disease types, with ≥30 clinical cases for each disease, forming 1500+ labeled sample pairs. Each sample includes patient multimodal data, candidate treatment plans, medical resource status, and corresponding comprehensive fit score.

[0383] In some embodiments, the optimization strategy and training method for the adaptive score evaluation model include:

[0384] Optimizer configuration: AdamW optimizer is used.

[0385]

[0386] in:

[0387] Indicates the initial learning rate;

[0388] This represents the weight decay coefficient (0.01).

[0389] and These represent the first and second moment estimates after bias correction;

[0390] It is the numerical stability constant;

[0391] Learning rate scheduling: A learning rate scheduling strategy combining linear warm-up and cosine decay is adopted.

[0392]

[0393] in Indicates the number of preheating steps. This represents the total number of training steps.

[0394] Layered fine-tuning strategy:

[0395] Phase 1 (1-10 epochs): Freeze the parameters of the pre-trained model and train only the fusion layer and the output layer. The learning rate is 1e-4.

[0396] Phase 2 (11-30 epochs): Unfreeze the parameters of the bottom 4 Transformer layers, with learning rates of 1e-5 (pre-trained layers) and 5e-5 (other layers).

[0397] Phase 3 (31-50 epochs): Unfreeze all parameters, learning rate = 5e-6 (pre-trained layer), 1e-5 (intermediate layer), and 5e-5 (output layer).

[0398] Data augmentation strategies: To improve the model's generalization ability, the following data augmentation methods are adopted:

[0399] Text enhancement: Randomly replace 3%-5% of non-critical terms in medical records with synonyms;

[0400] Temporal enhancement: Adding minor temporal perturbations (±5%) to the medical history time series;

[0401] Feature perturbation: Gaussian noise is added to the numerical features. );

[0402] Negative sample generation: generating negative samples using random replacement schemes or resources ( ).

[0403] refer to Figure 4 , Figure 4 This is a flowchart illustrating the third embodiment of the rare disease diagnosis and treatment resource collaborative scheduling method based on multimodal fusion of the present invention.

[0404] Based on the above embodiments, in this embodiment, step S50 may include:

[0405] Step S501: In response to the monitoring results meeting the triggering conditions, the dynamic adjustment process of the treatment pathway is modeled as a Markov decision process based on the treatment plan data and the currently monitored patient data and medical resource data, and a value function is defined.

[0406] It should be noted that this embodiment models the path planning problem as a Markov Decision Process (MDP), with a state space... Includes patient status and resource status Action space This represents the set of available treatment options. The value function is defined as:

[0407]

[0408]

[0409] in, express The state space at any given time includes the patient's state and the resource state. express Always in the zone The value function under, express Always in the zone Select action Instant rewards (adaptive score). Indicates from action space The action of selecting a treatment plan express The action space at any given moment contains the set of currently available treatment options. Indicates the patient's condition. Indicates resource status. express The patient's status at any given time is Choosing a treatment plan Resource status is The corresponding comprehensive adaptation score at that time This represents a discount factor used to balance current rewards with future rewards (e.g., ...). ), Indicates in Moment State After making a selection, Time-value function The expected condition.

[0410] It should be noted that the triggering condition is that the change in the comprehensive adaptation score exceeds the adaptation score change threshold or the change in resource status exceeds the resource status change threshold.

[0411] Step S502: Solve the value function, and dynamically adjust the treatment pathway based on the solution. The value function is solved using a deep Q-network optimal solution strategy, as shown in the following formula:

[0412]

[0413] in, Represents the state at time t Select action The Q-value function, using a neural network approximate, express A treatment plan selected from the action space at any given moment. express Moment State Select action Q-value function;

[0414] Step S503: Calculate resource scheduling priority based on the dynamically adjusted treatment pathway, referring to the following formula:

[0415]

[0416] in, Indicates the patient Resources Appointment priority, This represents the patient-resource fit score. The scheduling cost is represented by the time cost and the economic cost (normalized to [0,10]). Indicates the current utilization rate of resources. This indicates the patient-resource matching weight. Indicates the scheduling cost weight. Indicates the weight of resource utilization rate, for example ;

[0417] Step S504: Perform collaborative scheduling of medical resources based on resource scheduling priority.

[0418] Furthermore, in one embodiment, the resource conflict resolution function is used to resolve resource conflicts during the collaborative scheduling process of medical resources. The mathematical expression of the target conflict resolution function is as follows:

[0419]

[0420] in, This indicates finding the minimum value of the set of assignable variables X. This represents the balance coefficient, used to balance scheduling costs and reservation priorities. This represents an assigned variable used to characterize the patient. Resources allocated ,like This means that resources will be allocated. Assigned to patients ,like This means not to allocate resources. Assigned to patients .

[0421] Constraints: ,in Indicates the patient Resources allocated .

[0422] This embodiment models the dynamic adjustment of treatment pathways as a Markov decision process, combining a value function that considers both the immediate fit score of the current treatment plan and the long-term benefits of future treatments. A deep Q-network is then used to solve for the optimal strategy, improving the scientific rigor and optimality of the dynamic adjustment of treatment pathways. Simultaneously, resource scheduling priorities are calculated based on a weighted average of patient-resource fit scores, scheduling costs, and resource utilization rates, ensuring that the priorities better align with actual treatment needs. Finally, resource collaborative scheduling is carried out based on these priorities, effectively improving the accuracy and efficiency of cross-institutional medical resource matching and facilitating efficient resource scheduling after the dynamic adjustment of treatment pathways.

[0423] In one embodiment, the present invention has been validated by clinical data, and the experimental design and results are as follows:

[0424] Data sample details:

[0425] Data source: 10 rare disease diagnosis and treatment centers nationwide; Disease coverage: 50 rare diseases, including spinal muscular atrophy, Fabry disease, Huntington's disease, etc.; Sample size: 1500 complete rare disease cases, each case including patient multimodal data, 3-5 candidate treatment plans and corresponding expert scores; Data division: training set 1050 cases (70%), validation set 225 cases (15%), test set 225 cases (15%).

[0426] Experimental Design: Five-fold cross-validation was used to evaluate model performance and compared with the following baseline methods: 1. Traditional rule system: expert rule base based on clinical guidelines; 2. Unimodal model: using only electronic medical record text features; 3. Simple fusion model: multimodal features were directly concatenated and input into a multilayer perceptron; 4. Clinical expert panel: a decision-making panel composed of 3 rare disease experts.

[0427] Evaluation indicators:

[0428] Regression performance: Mean Absolute Error (MAE) and Root Mean Square Error (RMSE), as shown in the following formulas:

[0429]

[0430]

[0431] Ranking performance: Normalized depreciation cumulative gain (NDCG@k) and mean reciprocal rank (MRR):

[0432]

[0433]

[0434] in For the plan Real ratings Rank the best solutions This is the normalization factor.

[0435] Decision support performance: The percentage of recommended solutions that are among the top two choices by experts (Top-2 accuracy).

[0436] The experimental results are shown in Table 1 below:

[0437] Table 1. Comparison of Multiple Evaluation Indicators

[0438]

[0439] Resource collaboration effectiveness evaluation: The effectiveness of resource collaboration was evaluated in a regional medical collaboration network consisting of multiple tertiary hospitals and multiple community hospitals.

[0440] Resource utilization rate: increased from 45% to 68%, an increase of 23 percentage points;

[0441] Inter-agency collaboration time: reduced from an average of 9.5 days to 2.3 days;

[0442] Patient satisfaction: increased from 62% to 89%;

[0443] Treatment costs: reduced by an average of 18.7%;

[0444] Experimental results show that the performance of the present invention is close to that of the expert panel (MAE difference <0.16), significantly better than the existing technical solutions, and can provide effective support for rare disease diagnosis and treatment pathway planning and cross-institutional resource collaboration.

[0445] Furthermore, this embodiment of the invention also proposes a computer-readable storage medium storing a collaborative scheduling program for rare disease diagnosis and treatment resources based on multimodal fusion. When the collaborative scheduling program for rare disease diagnosis and treatment resources based on multimodal fusion is executed by a processor, it implements the steps of the collaborative scheduling method for rare disease diagnosis and treatment resources based on multimodal fusion as described above.

[0446] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0447] The aforementioned computer-readable storage medium may be included in a multimodal fusion-based rare disease diagnosis and treatment resource collaborative scheduling device; or it may exist independently and not be assembled into a multimodal fusion-based rare disease diagnosis and treatment resource collaborative scheduling device.

[0448] Furthermore, this invention also proposes a computer program product, including a rare disease diagnosis and treatment resource collaborative scheduling program based on multimodal fusion. When the rare disease diagnosis and treatment resource collaborative scheduling program based on multimodal fusion is executed by a processor, it implements the steps of the rare disease diagnosis and treatment resource collaborative scheduling method based on multimodal fusion as described above.

[0449] The specific implementation of the computer program product of the present invention is basically the same as the embodiments of the above-mentioned collaborative scheduling method for rare disease diagnosis and treatment resources based on multimodal fusion, and will not be repeated here.

[0450] Reference Figure 5 , Figure 5 This is a structural block diagram of the first embodiment of the rare disease diagnosis and treatment resource collaborative scheduling system based on multimodal fusion of the present invention.

[0451] like Figure 5 As shown in the embodiments of the present invention, the rare disease diagnosis and treatment resource collaborative scheduling system based on multimodal fusion includes:

[0452] The multimodal data processing module 10 is used to collect multimodal data, preprocess the multimodal data and extract features to obtain multidimensional features. The multimodal data includes patient data, treatment plan data and medical resource data. The multidimensional features include patient features, treatment plan features and medical resource features. The treatment plan data is obtained based on candidate treatment plans. The candidate treatment plans are generated based on patient diagnosis results and a rare disease knowledge base.

[0453] The adaptation score evaluation module 20 is used to input the multidimensional features into a pre-trained adaptation score evaluation model and output the patient's comprehensive adaptation score. The adaptation score evaluation model is a three-modal Transformer architecture, which includes a multimodal encoder, a cross-modal fusion layer, and an adaptation score calculation layer. The multimodal encoder includes a patient encoder, a protocol encoder, and a resource encoder. The cross-modal fusion layer is configured to perform cross-modal fusion of the encoding vectors of each modality output by the multimodal encoder through an attention mechanism. The adaptation score calculation layer is configured to calculate the comprehensive adaptation score based on the cross-modal interaction features output by the cross-modal fusion layer.

[0454] The solution screening module 30 is used to screen at least one recommended solution from the candidate treatment solutions based on the comprehensive fit score.

[0455] The resource matching module 40 is used to match the medical resources required by the recommended plan, perform cross-institutional resource collaborative scheduling, and monitor patient data and medical resource data.

[0456] The resource coordination and scheduling module 50 is used to respond to the monitoring results meeting the trigger conditions, dynamically adjust the treatment path based on the treatment plan data and the currently monitored patient data and medical resource data, and coordinate the scheduling of treatment resources based on the dynamically adjusted treatment path.

[0457] This embodiment fully explores the complex relationships between patients, treatment plans, and medical resources through multimodal fusion technology. It improves the accuracy of comprehensive adaptation assessment by using a dedicated model and combines cross-institutional collaborative scheduling and dynamic adjustment mechanisms. This effectively solves the problems of insufficient data utilization, inadequate plan adaptability, inefficient resource scheduling, and lack of flexible adjustment of treatment pathways in existing technologies. It significantly improves the personalization level of rare disease diagnosis and treatment, resource utilization efficiency, and the timeliness and rationality of treatment decisions.

[0458] The rare disease diagnosis and treatment resource collaborative scheduling system based on multimodal fusion provided in this application adopts the rare disease diagnosis and treatment resource collaborative scheduling method based on multimodal fusion in the above embodiments, and can solve the technical problem of rare disease diagnosis and treatment resource collaborative scheduling based on multimodal fusion. Compared with the prior art, the beneficial effects of the rare disease diagnosis and treatment resource collaborative scheduling system based on multimodal fusion provided in this application are the same as the beneficial effects of the rare disease diagnosis and treatment resource collaborative scheduling method based on multimodal fusion provided in the above embodiments, and other technical features of the rare disease diagnosis and treatment resource collaborative scheduling system based on multimodal fusion are the same as the features disclosed in the methods of the above embodiments, and will not be repeated here.

[0459] It should be understood that the above are merely illustrative examples and do not constitute any limitation on the technical solutions of the present invention. In specific applications, those skilled in the art can make settings as needed, and the present invention does not impose any restrictions on this.

[0460] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this invention. In practical applications, those skilled in the art can select some or all of the workflow to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0461] In addition, for technical details not described in detail in this embodiment, please refer to the method for collaborative scheduling of rare disease diagnosis and treatment resources based on multimodal fusion provided in any embodiment of the present invention, which will not be repeated here.

[0462] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0463] It should be noted that the user information (including but not limited to user device information, user personal information, user location information, user behavior information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0464] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0465] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0466] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for collaborative scheduling of rare disease diagnosis and treatment resources based on multimodal fusion, characterized in that, The method for collaborative scheduling of rare disease diagnosis and treatment resources based on multimodal fusion includes: Multimodal data is collected, preprocessed, and features are extracted from the multimodal data to obtain multidimensional features. The multimodal data includes patient data, treatment plan data, and medical resource data. The multidimensional features include patient features, treatment plan features, and medical resource features. The treatment plan data is obtained based on candidate treatment plans, which are generated based on patient diagnosis results and a rare disease knowledge base. The multidimensional features are input into a pre-trained adaptation score evaluation model, which outputs the patient's comprehensive adaptation score. The adaptation score evaluation model is a three-modal Transformer architecture, which includes a multimodal encoder, a cross-modal fusion layer, and an adaptation score calculation layer. The multimodal encoder includes a patient encoder, a protocol encoder, and a resource encoder. The cross-modal fusion layer is configured to fuse the encoding vectors of each modality output by the multimodal encoder through an attention mechanism. The adaptation score calculation layer is configured to calculate the comprehensive adaptation score based on the cross-modal interaction features output by the cross-modal fusion layer. At least one recommended treatment plan is selected from the candidate treatment plans based on the comprehensive fit score; Match the medical resources required for the recommended scheme, perform cross-institutional resource collaborative scheduling, and monitor patient data and medical resource data; In response to the monitoring results meeting the trigger conditions, the treatment pathway is dynamically adjusted based on the treatment plan data, the currently monitored patient data, and the medical resource data, and the treatment resources are coordinated and scheduled based on the dynamically adjusted treatment pathway. The response to monitoring results meeting trigger conditions involves dynamically adjusting the treatment pathway based on treatment plan data, currently monitored patient data, and medical resource data, and then coordinating the scheduling of treatment resources based on the dynamically adjusted treatment pathway, including: In response to the monitoring results meeting the trigger conditions, the dynamic adjustment process of the treatment pathway is modeled as a Markov decision process based on the treatment plan data, as well as the currently monitored patient data and medical resource data, and the value function is defined as: in, express The state space at any given time includes the patient's state and the resource state. express Always in the zone The value function under the following conditions express Always in the zone Select action Instant rewards Indicates from action space The action of selecting a treatment plan express The action space at any given moment contains the set of currently available treatment options. Indicates the patient's condition. Indicates resource status. express The patient's status at any given time is Choosing a treatment plan Resource status is The corresponding comprehensive adaptation score at that time This represents a discount factor used to balance current rewards with future rewards. Indicates in Moment State After making a selection, Time-value function The expected value of the condition; The triggering condition is that the change in the comprehensive adaptation score exceeds the adaptation score change threshold or the change in resource status exceeds the resource status change threshold. The value function is solved, and the treatment pathway is dynamically adjusted based on the solution. The value function is solved using a deep Q-network optimal strategy, as shown in the following formula: in, Represents the state at time t Select action Q-value function, express A treatment plan selected from the action space at any given moment. express Moment State Select action Q-value function; Resource scheduling priority is calculated based on the dynamically adjusted treatment pathway, referring to the following formula: in, Indicates the patient Resources Appointment priority, This represents the patient-resource fit score. This represents the scheduling cost, which includes time cost and economic cost. Indicates the current utilization rate of resources. This indicates the patient-resource matching weight. Indicates the scheduling cost weight. Indicates the weight of resource utilization rate; Coordinated scheduling of medical resources based on resource scheduling priorities; The collaborative scheduling of medical resources employs a target conflict resolution function to resolve resource conflicts. The mathematical expression of the target conflict resolution function is as follows: The constraints of the target conflict resolution function are: in, This means finding the minimum value of the set of assignable variables X. This represents the balance coefficient, used to balance scheduling costs and reservation priorities. This represents an assigned variable used to characterize the patient. Resources allocated ,like This means that resources will be used to... Assigned to patients ,like This means not to allocate resources. Assigned to patients .

2. The method for collaborative scheduling of rare disease diagnosis and treatment resources based on multimodal fusion as described in claim 1, characterized in that, The feature extraction process for the patient's characteristics includes: Historical text sequences are extracted from the patient's electronic medical record text, and the historical text sequences are input into a pre-trained text feature extraction model to output text feature sub-vectors. The text feature extraction model is a BioBERT model based on a multi-layer Transformer architecture. Gene sequences are extracted from the patient's gene sequencing data and encoded using an attention-weighted one-hot encoder to output gene feature vectors. Candidate time series are extracted from the patient's medical history time series data, and the candidate time series are input into a pre-trained bidirectional long short-term memory network model to output time series feature sub-vectors; The text feature vectors, gene feature vectors, and temporal feature vectors are fused using an intramodal attention mechanism to obtain patient features.

3. The method for collaborative scheduling of rare disease diagnosis and treatment resources based on multimodal fusion as described in claim 2, characterized in that, The feature extraction process of the treatment plan includes: Entity sets and relation sets are extracted from the diagnostic knowledge graph, and the TransE model is used to learn and embed the entity sets and relation sets to obtain knowledge graph feature sub-vectors. A set of research literature abstracts related to candidate treatment plans is obtained, and the set of research literature abstracts is input into a literature feature extraction model to output literature feature sub-vectors. The literature feature extraction model is a PubMedBERT model based on a multi-layer Transformer architecture. The literature feature extraction model is pre-trained based on biomedical text data. Statistical feature vectors of efficacy distribution, efficacy uncertainty feature vectors, and adverse reaction risk feature vectors are extracted from clinical trial data of candidate treatment options, and effect feature sub-vectors are generated based on the statistical feature vectors, efficacy uncertainty feature vectors, and adverse reaction risk feature vectors. The knowledge graph feature vector, the document feature vector, and the effect feature vector are fused using an intramodal attention mechanism to obtain the treatment plan features.

4. The method for collaborative scheduling of rare disease diagnosis and treatment resources based on multimodal fusion as described in claim 3, characterized in that, The feature extraction process for the medical resource characteristics includes: Spatial feature subvectors are extracted from the geographic location data of medical institutions associated with candidate treatment plans. The geographic location data includes the latitude and longitude coordinates of the medical institutions and the straight-line distance between the patient's current location and the medical institutions. Based on the equipment configuration vector, physician expertise vector, and departmental load vector of the medical institution, a sub-vector of diagnostic and treatment capabilities is generated. The equipment configuration vector is determined based on the presence or absence of key equipment in the medical institution. The physician expertise vector is constructed based on the topics of papers published by physicians in the medical institution and their target clinical fields. The departmental load vector is generated based on the current number of patients, the ratio of medical staff to nurses, and the bed occupancy rate of the medical institution. Based on the waiting time feature vector and referral cost feature vector of the medical institution, a feature vector of accessibility of medical resources is generated. The spatial feature sub-vector, the diagnostic and treatment capability feature sub-vector, and the accessibility feature sub-vector are fused using an intramodal attention mechanism to obtain medical resource features.

5. The method for collaborative scheduling of rare disease diagnosis and treatment resources based on multimodal fusion as described in any one of claims 1 to 4, characterized in that, The patient encoder is based on the BioBERT architecture and is configured to encode patient features into a patient encoding vector. The scheme encoder is a multi-layer Transformer architecture, configured to encode the diagnostic and treatment scheme features into a scheme encoding vector; The resource encoder is a multi-layer Transformer and LSTM combined architecture, configured to perform spatial attention calculation and temporal modeling on medical resource features to encode the medical resource features into resource encoding vectors. The cross-modal fusion layer includes a bimodal attention module and a trimodal attention module; The bimodal attention module is used to perform cross-modal fusion of the patient encoding vector and the protocol encoding vector to output patient-protocol interaction features, as shown in the following formula: in, This represents the patient-treatment interaction characteristics obtained after the fusion of bimodal attentional interactions between the patient and the treatment plan. This represents the cross-modal attention computation function. Represents the patient coding vector. Represents the scheme encoding vector, Indicates the dimension of interaction features; The bimodal attention module is also used to perform cross-modal fusion of the scheme encoding vector and the resource encoding vector to output the scheme-resource interaction feature, as shown in the following formula: in, This represents the solution-resource interaction characteristics obtained after the treatment plan and medical resources are fused through bimodal attention interaction. Represents a resource encoding vector; The trimodal attention module is used to perform cross-modal fusion of patient encoding vectors, protocol encoding vectors, and resource encoding vectors to output trimodal interaction features, as shown in the following formula: in, This represents the trimodal interaction feature obtained after the trimodal attention interaction fusion of the patient encoding vector, the protocol encoding vector, and the resource encoding vector. This represents a trimodal attention computation function, used to realize the interaction between trimodal features.

6. The method for collaborative scheduling of rare disease diagnosis and treatment resources based on multimodal fusion as described in claim 5, characterized in that, The adaptation score calculation layer is further configured to calculate the patient-plan adaptation score based on patient-plan interaction features, referring to the following formula: in, This indicates the patient-protocol fit score. This represents a multilayer perceptron module used to calculate patient-protocol fit scores, containing two hidden layers with ReLU activation function, and the output layer scaled to the range [1, 10] using the Sigmoid function; The adaptation score calculation layer is further configured to calculate the scheme-resource adaptation score based on the scheme-resource interaction features, referring to the following formula: in, The solution-resource adaptation score is represented. This represents a multilayer perceptron module used to calculate scheme-resource adaptation scores. It contains two hidden layers, uses the ReLU activation function, and scales the output layer to the range [1, 10] using the Sigmoid function. The adaptation score calculation layer is also configured to calculate the trimodal co-adaptation score based on trimodal interaction features, as shown in the following formula: in, This represents the trimodal co-adaptation score. This represents a multilayer perceptron module used to compute the trimodal co-adaptation score between patient modality, protocol modality, and resource modality. It contains three hidden layers, uses the GELU activation function, and scales the output layer to the range [1,10] using the Sigmoid function. The adaptation score calculation layer is further configured to calculate a comprehensive adaptation score based on the patient-solution adaptation score, the solution-resource adaptation score, and the trimodal collaborative adaptation score, referring to the following formula: in, This indicates the overall fit score. This indicates the patient-treatment fit weight. Representation scheme - resource adaptation weight, This represents the weights for trimodal collaborative adaptation.

7. The method for collaborative scheduling of rare disease diagnosis and treatment resources based on multimodal fusion as described in claim 6, characterized in that, The adaptation score evaluation model is trained using a hybrid loss function, the mathematical expression of which is as follows: in, Represents the mixed loss term. This represents the mean squared error loss term. This indicates the ranking loss item. This represents the modal alignment loss term. Represents the regularization loss term. This represents the weighting coefficient of the ranking loss term. This represents the weighting coefficients of the modal alignment loss term. Represents the weight coefficients of the regularization loss term; The mathematical expression for the mean squared error loss term is as follows: in, The model represents the first The prediction fit score for each sample. Indicates the first The true fit score for each sample is determined based on the gold standard score annotated by experts. Indicates the number of training samples; The ranking loss term is used to ensure that the model can correctly rank the suitability of different solutions. The mathematical expression of the ranking loss term is as follows: in, and They represent the model for the first... The first sample and the first The prediction fit score for each sample. and They represent the first The first sample and the first The true fit score of each sample Represents a symbolic function; The modality alignment loss term is used to promote the alignment of feature spaces of different modalities, and the mathematical expression of the modality alignment loss term is as follows: in, Indicates the first Patient modal feature vectors of each sample Indicates the first Modal feature vectors of treatment plans for each sample. Indicates the first The modal feature vector of medical resources for each sample. This represents the Euclidean norm, used to calculate the length of a vector. The regularization loss term is used to prevent the fit score evaluation model from overfitting. The mathematical expression of the regularization loss term is as follows: in, This represents a single learnable parameter in the model. This represents the set of all learnable parameters of the model.

8. A collaborative scheduling system for rare disease diagnosis and treatment resources based on multimodal fusion, characterized in that, The system employs the multimodal fusion-based collaborative scheduling method for rare disease diagnosis and treatment resources as described in any one of claims 1 to 7, wherein the multimodal fusion-based collaborative scheduling system for rare disease diagnosis and treatment resources comprises: A multimodal data processing module is used to collect multimodal data, preprocess the multimodal data and extract features to obtain multidimensional features. The multimodal data includes patient data, treatment plan data and medical resource data. The multidimensional features include patient features, treatment plan features and medical resource features. The treatment plan data is obtained based on candidate treatment plans, which are generated based on patient diagnosis results and a rare disease knowledge base. The adaptation score evaluation module is used to input the multidimensional features into a pre-trained adaptation score evaluation model and output the patient's comprehensive adaptation score. The adaptation score evaluation model is a three-modal Transformer architecture, which includes a multimodal encoder, a cross-modal fusion layer, and an adaptation score calculation layer. The multimodal encoder includes a patient encoder, a protocol encoder, and a resource encoder. The cross-modal fusion layer is configured to perform cross-modal fusion of the encoding vectors of each modality output by the multimodal encoder through an attention mechanism. The adaptation score calculation layer is configured to calculate the comprehensive adaptation score based on the cross-modal interaction features output by the cross-modal fusion layer. The treatment plan selection module is used to select at least one recommended treatment plan from the candidate treatment plans based on the comprehensive fit score. The resource matching module is used to match the medical resources required by the recommended plan, perform cross-institutional resource collaborative scheduling, and monitor patient data and medical resource data. The resource coordination and scheduling module is used to respond to the monitoring results meeting the trigger conditions, dynamically adjust the treatment path based on the treatment plan data and the currently monitored patient data and medical resource data, and coordinate the scheduling of treatment resources based on the dynamically adjusted treatment path.

Images