Method and system for optimizing dynamic diagnosis and treatment path of myasthenia gravis based on multi-modal fusion

Abstract

The application discloses a kind of myasthenia gravis dynamic diagnosis and treatment path optimization method and system based on multimodal fusion, the method includes: the multimodal data of myasthenia gravis patient is collected, and its feature extraction and enhancement are carried out to obtain multimodal feature;Based on multimodal feature, construct three-dimensional dynamic feature matrix and carry out cross-modal fusion, obtain target fusion feature;Target fusion feature is used to train multi-task depth network model, and the diagnosis and treatment path optimization model is obtained;According to preset interval, data is dynamically collected and input model, and diagnosis and treatment evaluation result is output to realize diagnosis and treatment path dynamic optimization.Due to the application, through multimodal fusion and dynamic optimization mechanism, break the limitation of traditional static path diagnosis and treatment, effectively excavate patient multidimensional disease characteristics, improve diagnosis and treatment evaluation accuracy, realize diagnosis and treatment path dynamic optimization, effectively reduce unreasonable diagnosis and treatment behavior, reduce diagnosis and treatment cost, improve efficiency, provide intelligent support for clinic, improve diagnosis and treatment effect.

Description

Technical Field

[0001] This invention relates to the field of medical and health information technology, and in particular to a method and system for optimizing dynamic diagnosis and treatment pathways for myasthenia gravis based on multimodal fusion. Background Technology

[0002] Myasthenia gravis (MG) is a neuromuscular junction transmission disorder mediated by autoantibodies. Its diagnosis and treatment involve multi-dimensional data, including clinical symptom assessment, laboratory testing, imaging examinations, and neurophysiological evaluation, and require dynamic adjustments to the treatment approach based on individual patient differences and the capabilities of medical institutions. Current technologies have the following limitations:

[0003] The treatment pathways are discrete and lack dynamic adaptation mechanisms: Most existing MG treatment pathways are fixed and static, without taking into account the dynamic changes in the patient's condition (such as symptom fluctuations and comorbidity progression) and the heterogeneity of medical institutions' capabilities.

[0004] Decision-making relies heavily on expert experience, is highly subjective, and has poor replicability: MG diagnosis and treatment decisions depend heavily on the personal experience of neurologists, especially at key points such as the diagnosis of antibody-negative patients, drug adjustments for refractory cases, and the timing of thymus surgery. However, there are significant differences in the treatment plans for the same MG case among different individuals. Therefore, the lack of objective and quantitative decision-making methods makes it difficult to standardize and promote high-quality treatment experience.

[0005] Multimodal data is fragmented, failing to realize the value of collaborative decision-making: MG diagnosis and treatment data covers multimodal information such as text-based electronic medical records (EHR), imaging data (thymus CT / MRI), neurophysiological signals (repetitive neural stimulation RNS), and structured laboratory indicators (antibody detection, muscle strength score), but existing systems mostly perform single-modal independent analysis, and the correlation between multimodal data has not been effectively explored, resulting in one-sided decision-making information.

[0006] The lack of clear division of labor in the tiered medical system and the mismatch between the capabilities of medical institutions and the needs of patients: The current MG tiered medical system lacks quantitative assessment standards for the capabilities of medical institutions, leading to resource misallocation.

[0007] The evaluation of treatment effectiveness is lagging and lacks a prospective prediction mechanism: Currently, the evaluation of MG treatment effectiveness is mostly based on retrospective indicators (such as changes in muscle strength scores after 3 months), which cannot predict the potential effects of intervention measures in real time at the treatment nodes. Summary of the Invention

[0008] The main objective of this invention is to provide a method and system for optimizing the dynamic diagnosis and treatment pathway for myasthenia gravis based on multimodal fusion, aiming to solve the technical problems of existing diagnosis and treatment pathways being static and rigid, relying on subjective experience, unable to effectively mine multimodal data, having an imbalance in hierarchical diagnosis and treatment, and lagging evaluation of treatment effects.

[0009] To achieve the above objectives, this invention provides a method for optimizing the dynamic diagnosis and treatment pathway for myasthenia gravis based on multimodal fusion, the method comprising the following steps:

[0010] Multimodal data of patients with myasthenia gravis were collected, including text modal data, image modal data, physiological signal modal data, and structured modal data.

[0011] Feature extraction and feature enhancement processing are performed on the multimodal data to obtain multimodal features, which include text modal features, image modal features, physiological signal temporal features, and structural temporal features.

[0012] A three-dimensional dynamic feature matrix is ​​constructed based on the multimodal features, and cross-modal feature fusion is performed to obtain the target fused features;

[0013] The multi-task deep network model is trained based on the target fusion features to obtain a treatment path optimization model. The treatment path optimization model is configured to perform adaptability score regression analysis, referral necessity analysis, and treatment effect prediction.

[0014] Multimodal data of myasthenia gravis patients are collected at preset time intervals and input into the treatment path optimization model. The model outputs the treatment assessment results of the myasthenia gravis patients and dynamically optimizes the treatment path of the myasthenia gravis patients based on the treatment assessment results. The treatment assessment results include the suitability score, the probability of referral, and the prediction results of treatment effect.

[0015] Optionally, the step of performing feature extraction and feature enhancement processing on the multimodal data to obtain multimodal features includes:

[0016] A pre-trained bidirectional encoder representation model is used to extract features from text modal data to obtain initial text features;

[0017] A residual network fusion feature pyramid network architecture is used to extract multi-scale features from image modal data to obtain initial image features;

[0018] A hybrid architecture combining a one-dimensional convolutional neural network and a long short-term memory network is used to dynamically extract features from physiological signal modal data to obtain initial physiological signal features.

[0019] Temporal convolutional networks are used to extract temporal features from structured modal data to obtain initial structural features;

[0020] Feature enhancement processing is performed on the initial text features, the initial image features, the initial physiological signal features, and the initial structural features respectively to obtain multimodal features.

[0021] Optionally, the step of performing feature enhancement processing on the initial text features, the initial image features, the initial physiological signal features, and the initial structural features to obtain multimodal features includes:

[0022] The extracted clinical entities are mapped to entity embedding vectors, and the entity embedding vectors are used to enhance the initial text features to obtain text modality features, as shown in the following formula:

[0023]

[0024] in, Representing text modal features, Represents entity weight, The entity embedding vector representing the clinical entity. Indicates initial text features;

[0025] The region of interest in the lesion image is pooled to extract lesion features, and the initial image features are then enhanced based on these lesion features to obtain image modal features, as shown in the following formula:

[0026]

[0027] in, Represents image modal features, Indicates the weight of the lesion. This represents the lesion features extracted by region-of-interest pooling in lesion images. Indicates initial image features;

[0028] The RNS decay rate feature is extracted by the convolution kernel of the temporal convolutional network, and the initial physiological signal features are then enhanced based on the RNS decay rate feature to obtain the temporal features of the physiological signal, as shown in the following formula:

[0029]

[0030] in, This represents the RNS decay rate feature extracted by the convolution kernel. Indicates the decay rate weight. Indicates the temporal characteristics of physiological signals. Indicates the characteristics of the initial physiological signal;

[0031] The QMG score time series of myasthenia gravis patients is input into a pre-trained long short-term memory network for time series analysis to obtain trend features. The QMG score time series is extracted based on structured modal data.

[0032] Based on the trend characteristics, feature enhancement processing is performed on the initial structural features to obtain structural temporal features, as shown in the following formula:

[0033]

[0034] in, Represents the temporal characteristics of the structure. Indicates the QMG score weight. This indicates the trend characteristics of the QMG score over time. This represents the initial structural features.

[0035] Optionally, the step of constructing a three-dimensional dynamic feature matrix based on the multimodal features and performing cross-modal feature fusion to obtain the target fused features includes:

[0036] A patient state feature matrix is ​​constructed based on the aforementioned structural temporal features, and a medical institution capability feature vector is constructed based on the medical institution capability indicators.

[0037] The patient state feature matrix is ​​multiplied by the medical institution capability feature vector to obtain a three-dimensional dynamic feature matrix, as shown in the following formula:

[0038]

[0039] in, Represents a three-dimensional dynamic feature matrix. Represents the patient state feature matrix. This represents the feature vector of a medical institution's capabilities. This represents the outer product operation, used to broadcast the capability feature vector of a medical institution to the time dimension. The dimension of the three-dimensional dynamic feature matrix is ​​represented. Represents the patient's state dimension. Indicates the time dimension. This indicates the capability dimension of the medical institution;

[0040] Dynamic tensor features are extracted based on the aforementioned three-dimensional dynamic feature matrix;

[0041] The multimodal weights are calculated using an attention mechanism, and the multimodal features are then weighted and fused based on these weights to obtain the multimodal fused features, as shown in the following formula:

[0042]

[0043]

[0044] in, Indicates multimodal fusion features, and They represent the first The and the first Modal features, This represents the multimodal weights of the i-th mode. This represents a multilayer perceptron;

[0045] The patient state sequence is modeled based on the patient state feature matrix, and the patient treatment trajectory features are extracted based on the patient treatment trajectory.

[0046] The dynamic tensor features, the multimodal fusion features, and the patient's diagnostic trajectory features are concatenated to obtain the target fusion features, as shown in the following formula:

[0047]

[0048] in, Indicates target fusion features, Represents the characteristics of a dynamic tensor. Indicates multimodal fusion features, This indicates the characteristics of the patient's medical treatment trajectory. This indicates a splicing operation.

[0049] Optionally, the step of modeling the patient state sequence based on the patient state feature matrix and extracting patient treatment trajectory features based on the patient treatment trajectory includes:

[0050] Based on the patient state feature matrix and intervention actions, a patient state sequence is modeled, referring to the following formula:

[0051]

[0052]

[0053] in, Represents a sequence of patient states. This represents the patient's state at time n. Indicates the first Intervention actions at any time, and This represents the patient's state at time t and t+1. Represents the state transition function. Let represent the random perturbation at time t. Indicates the first Intervention actions at specific moments;

[0054] The patient state sequence is input into the Transformer encoder to extract the patient's treatment trajectory features, as shown in the following formula:

[0055]

[0056] in, This indicates the characteristics of the patient's medical treatment trajectory. This indicates feature splicing.

[0057] Optionally, the diagnosis and treatment path optimization model includes an input layer, a feature extraction layer, a fusion layer, and a multi-task output layer;

[0058] The input layer is configured to receive multimodal data structured as time series, including text modal data, image modal data, physiological signal modal data, and structured modal data.

[0059] The feature extraction layer is configured to perform feature extraction and feature enhancement processing on the multimodal data to obtain multimodal features. The multimodal features include text modality features, image modality features, physiological signal temporal features, and structural temporal features. The feature extraction layer includes a pre-trained bidirectional encoder representation module, a residual network fusion feature pyramid network architecture module, a one-dimensional convolutional neural network fusion long short-term memory network hybrid architecture module, and a temporal convolutional network module.

[0060] The fusion layer is configured to construct a three-dimensional dynamic feature matrix based on the multimodal features and perform cross-modal feature fusion to obtain the target fused features;

[0061] The multi-task output layer is configured for adaptability score regression analysis, referral necessity analysis, and treatment effect prediction.

[0062] Optionally, the total loss function of the multi-task output layer is:

[0063]

[0064] in, Represents the total loss item. This represents the regression loss term for the fit score. This indicates the loss item categorized by the necessity of referral. This represents the loss term in the prediction of treatment effectiveness. and These represent the weights of the loss terms;

[0065] The loss function for the fitness score regression loss term is:

[0066]

[0067] in, Indicates the number of samples. This represents the regression loss weight for the i-th sample. This represents the fitness score predicted by the model for the i-th sample. The expert annotation fit score of the i-th sample;

[0068] The loss function for the referral necessity classification loss term is:

[0069]

[0070] in, Indicates the weight of positive examples. This represents a hard-case mining factor, used to enhance attention to hard-to-classify samples. This represents the referral necessity label for the i-th sample. This represents the referral probability of the i-th sample output by the model;

[0071] The loss function for the treatment effect prediction loss term is:

[0072]

[0073]

[0074] in, This represents the change in the QMG score predicted by the model for the i-th sample. This represents the actual change in QMG score for the i-th sample. Indicates the baseline QMG score, The QMG score represents the QMG score after treatment. The QMG score is a quantitative myasthenia gravis score.

[0075] Optionally, the dynamic optimization of the treatment pathway for the myasthenia gravis patient based on the diagnostic and treatment assessment results includes:

[0076] The suitability score is compared with the first threshold and the second threshold, and the referral determination probability is compared with the third threshold to obtain the comparison result. The first threshold is greater than the second threshold.

[0077] If the suitability score is not lower than the first threshold and the referral determination probability is lower than the third threshold, the current institution is determined to be suitable for myasthenia gravis patients, and the current treatment path is maintained.

[0078] If the suitability score is lower than the first threshold and not lower than the second threshold, the diagnosis and treatment nodes in the diagnosis and treatment path of the myasthenia gravis patient user will be optimized.

[0079] If the fit score is lower than the second threshold, or if the fit score decreases by more than a preset decrease threshold multiple times, a referral warning is triggered. An optimal list of referral target institutions is generated based on the medical institution capability feature vector, and the treatment path of the myasthenia gravis patient is adjusted based on the list of target institutions.

[0080] Repeat the steps of collecting multimodal data of myasthenia gravis patients at preset time intervals and inputting the multimodal data into the treatment pathway optimization model to achieve dynamic optimization of the treatment pathway.

[0081] Furthermore, to achieve the above objectives, this invention also proposes a dynamic diagnosis and treatment pathway optimization system for myasthenia gravis based on multimodal fusion, wherein the dynamic diagnosis and treatment pathway optimization system for myasthenia gravis based on multimodal fusion includes:

[0082] The data acquisition module is used to collect multimodal data from patients with myasthenia gravis. The multimodal data includes text modal data, image modal data, physiological signal modal data, and structured modal data.

[0083] The feature extraction module is used to extract features and enhance features from the multimodal data to obtain multimodal features, including text modal features, image modal features, physiological signal time-series features, and structural time-series features.

[0084] The feature fusion module is used to construct a three-dimensional dynamic feature matrix based on the multimodal features and perform cross-modal feature fusion to obtain the target fused features;

[0085] The model training module is used to train a multi-task deep network model based on the target fusion features to obtain a treatment path optimization model. The treatment path optimization model is configured to perform adaptive score regression analysis, referral necessity analysis, and treatment effect prediction.

[0086] The dynamic optimization module for treatment pathways is used to collect multimodal data of myasthenia gravis patients based on preset time intervals, input the multimodal data into the treatment pathway optimization model, output the treatment assessment results of the myasthenia gravis patients, and dynamically optimize the treatment pathway of the myasthenia gravis patients based on the treatment assessment results. The treatment assessment results include suitability score, referral determination probability, and treatment effect prediction results.

[0087] This invention collects multimodal data from patients with myasthenia gravis, extracts and enhances these data to obtain multimodal features; constructs a three-dimensional dynamic feature matrix based on these multimodal features and performs cross-modal fusion to obtain target fusion features; trains a multi-task deep network model using these target fusion features to obtain a treatment pathway optimization model; dynamically collects data at preset intervals and inputs it into the model, outputting treatment evaluation results to achieve dynamic optimization of the treatment pathway, transforming discrete treatment pathways into quantifiable dynamic adaptation problems. Multimodal fusion overcomes the limitations of single data dimensions, improving decision-making accuracy; and the dynamic feature matrix captures spatiotemporal correlations, enabling real-time updates of the treatment pathway. Because this invention performs multimodal analysis based on comprehensive multimodal data collection… Feature extraction and feature enhancement processing enable the effective mining, integration, and utilization of disease characteristics. Through precise evaluation using a multi-task model and dynamic optimization of treatment pathways, it breaks through the limitations of traditional myasthenia gravis diagnosis and treatment, such as reliance on single data, rigid static pathways, and subjective human decision-making. It effectively mines multi-dimensional disease characteristics of patients, improving the accuracy and scientific nature of diagnosis and treatment assessment; it enables personalized and dynamic adaptation of treatment pathways, improving the rationality and timeliness of diagnosis and treatment decisions; it reduces irrational treatment behaviors, lowers treatment costs, and improves treatment efficiency; it provides reliable intelligent technical support for clinical diagnosis and treatment, promoting the transformation of myasthenia gravis diagnosis and treatment models from traditional experience-based to precise and intelligent, ultimately achieving the core goal of improving patient treatment outcomes and enhancing treatment quality. Attached Figure Description

[0088] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0089] Figure 1 This is a schematic diagram of the structure of a multimodal fusion-based dynamic diagnosis and treatment path optimization device for myasthenia gravis, which is part of the hardware operating environment of the embodiment of the present invention.

[0090] Figure 2 This is a flowchart illustrating the first embodiment of the dynamic diagnosis and treatment path optimization method for myasthenia gravis based on multimodal fusion of the present invention.

[0091] Figure 3 This is a schematic diagram of the architecture of the treatment path optimization model in one embodiment of the dynamic treatment path optimization method for myasthenia gravis based on multimodal fusion of the present invention.

[0092] Figure 4 This is a flowchart illustrating the second embodiment of the dynamic diagnosis and treatment path optimization method for myasthenia gravis based on multimodal fusion of the present invention.

[0093] Figure 5 This is a flowchart illustrating the third embodiment of the dynamic diagnosis and treatment path optimization method for myasthenia gravis based on multimodal fusion of the present invention.

[0094] Figure 6 This is a structural block diagram of the first embodiment of the dynamic diagnosis and treatment path optimization system for myasthenia gravis based on multimodal fusion of the present invention.

[0095] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0096] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0097] Reference Figure 1 , Figure 1 This is a schematic diagram of the hardware operating environment of the dynamic diagnosis and treatment path optimization device for myasthenia gravis based on multimodal fusion, which is involved in the embodiment of the present invention.

[0098] like Figure 1 As shown, the multimodal fusion-based dynamic diagnosis and treatment pathway optimization device for myasthenia gravis may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk storage device. The memory 1005 may also optionally be a storage system independent of the aforementioned processor 1001.

[0099] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the dynamic diagnosis and treatment pathway optimization device for myasthenia gravis based on multimodal fusion, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0100] like Figure 1As shown, the memory 1005, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a dynamic diagnosis and treatment path optimization program for myasthenia gravis based on multimodal fusion.

[0101] exist Figure 1 In the multimodal fusion-based dynamic diagnosis and treatment path optimization device for myasthenia gravis shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and memory 1005 in the multimodal fusion-based dynamic diagnosis and treatment path optimization device for myasthenia gravis can be set in the multimodal fusion-based dynamic diagnosis and treatment path optimization device for myasthenia gravis. The multimodal fusion-based dynamic diagnosis and treatment path optimization device for myasthenia gravis calls the multimodal fusion-based dynamic diagnosis and treatment path optimization program for myasthenia gravis stored in the memory 1005 through the processor 1001, and executes the multimodal fusion-based dynamic diagnosis and treatment path optimization method for myasthenia gravis provided in the embodiment of the present invention.

[0102] This invention provides a method for optimizing the dynamic diagnosis and treatment pathway for myasthenia gravis based on multimodal fusion, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the dynamic diagnosis and treatment path optimization method for myasthenia gravis based on multimodal fusion of the present invention.

[0103] In this embodiment, the method for optimizing the dynamic diagnosis and treatment pathway for myasthenia gravis based on multimodal fusion includes the following steps:

[0104] Step S10: Collect multimodal data from patients with myasthenia gravis.

[0105] It should be understood that the executing entity of this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or a terminal electronic device capable of realizing the above functions. The following description uses a multimodal fusion-based dynamic diagnosis and treatment pathway optimization device for myasthenia gravis (hereinafter referred to as the optimization device) as an example to illustrate this embodiment and the following embodiments.

[0106] It should be noted that multimodal data refers to various data sets that originate from different information channels, exist in different forms, and can comprehensively reflect the condition of patients with myasthenia gravis. These data can characterize the patient's physiological, pathological, and diagnostic information from different dimensions. The multimodal data includes text modal data, image modal data, physiological signal modal data, and structured modal data.

[0107] It should be noted that text modal data can be data related to myasthenia gravis patients existing in the form of natural language text, with the core being unstructured textual information carried during the patient's diagnosis and treatment process. Imaging modal data can be visualized data generated by medical imaging equipment, reflecting the patient's body structure, lesion location, and severity. Physiological signal modal data can be dynamic signal data collected by physiological monitoring equipment, reflecting the patient's real-time physiological state and possessing temporal characteristics. Structured modal data can be data related to myasthenia gravis patients recorded according to a fixed format and standardized fields, also possessing temporal characteristics.

[0108] Understandably, the optimized equipment can extract text modal data from the hospital's electronic medical record system, including patient complaints, present medical history, past medical history, medical orders, progress notes, and laboratory test reports; collect image modal data from medical imaging equipment (such as CT, MRI, electromyography, etc.), including images of the patient's brain, neck, and chest, as well as muscle electrical activity images; use wearable physiological monitoring devices and the hospital's intensive care monitoring system to collect real-time physiological signals such as electrocardiograms, electromyograms, respiratory rate, heart rate, blood pressure, and blood oxygen saturation, record them at a fixed sampling frequency, and form time-series physiological signal modal data; and extract patient basic information (age, gender, disease course), laboratory test indicators (antibody levels, complete blood count, liver and kidney function, etc.), and treatment operation records (drug types, dosages, surgical methods, etc.) from the hospital information system (HIS) and laboratory information system (LIS), organizing them into structured tabular data according to preset fields.

[0109] In some embodiments, the optimized device collects the patient's text modal data, which includes medical records, examination reports, and medication records from electronic medical records; collects the patient's image modal data, which includes thymic computed tomography images, thymic magnetic resonance imaging images, and muscle ultrasound images; collects the patient's physiological signal modal data, which includes repetitive nerve stimulation signals and electromyography signals; collects the patient's structured modal data, which includes laboratory test indicators, medication records, and time-series data of quantitative myasthenia gravis scores; and performs format standardization processing on the collected data to obtain standardized multimodal medical data.

[0110] In the specific implementation, multimodal data is defined as follows:

[0111] Text modal data :

[0112] (1) Data type: Electronic medical records (medical progress notes, examination reports, medication records);

[0113] Structural representation: ,in ( For text length, (for vocabulary size);

[0114] Core characteristics: physical characteristics (symptoms / medication / examination results) and emotional characteristics (the physician's tendency to describe the stability of the condition).

[0115] (2) Image modal data :

[0116] Data types: Thymus CT / MRI (DICOM format, 512×512×30 voxels), muscle ultrasound images;

[0117] Structural representation: ,in ( voxels);

[0118] Core features: Quantitative features of lesions (volume, mean CT value), texture features (gray-level co-occurrence matrix, wavelet transform coefficients).

[0119] (3) Physiological signal modal data :

[0120] Data types: Repetitive neural stimulation (RNS) signal (sampling rate 1kHz, duration 10 seconds), electromyography;

[0121] Structural representation: ,in ( Sampling points aisle);

[0122] Key features: RNS amplitude decay rate, electromyography frequency / amplitude characteristics.

[0123] (4) Structured modal data :

[0124] Data types: Laboratory test indicators (antibody titer, complete blood count), medication records, QMG score time series;

[0125] Structural representation: ,in ( For the number of indicators, (for a specific time point)

[0126] Key features: time-series change rate of indicators, medication dosage adjustment sequence, and QMG score trend.

[0127] Step S20: Perform feature extraction and feature enhancement processing on the multimodal data to obtain multimodal features.

[0128] It should be noted that multimodal features refer to the set of features that can represent the core information of each modality after feature extraction and enhancement of the original data of different modalities. They correspond to the four types of original multimodal data, including text modal features, image modal features, physiological signal time series features, and structural time series features.

[0129] Text modal features can be extracted from text modal data and can reflect key information such as patient condition descriptions and treatment processes, such as semantic features and keyword features.

[0130] Image modal features can be extracted from image modal data and can reflect key information such as the location, shape, and structure of the patient's lesion, such as texture features, contour features, and pixel distribution features.

[0131] Physiological signal temporal characteristics can be extracted from physiological signal modal data and can reflect the temporal correlation characteristics that reflect the changes in the patient's physiological state over time, such as signal peak value, period, trend characteristics, etc.

[0132] Structural temporal features can be extracted from structured modal data and can reflect the temporal correlation of changes in patient condition indicators and treatment procedures over time, such as the rate of change of indicators and the characteristics of treatment plan adjustments.

[0133] In some embodiments, the optimization device may employ differentiated feature extraction methods tailored to the characteristics of different modal data, combined with unified feature enhancement techniques, to obtain high-quality multimodal features, specifically including:

[0134] Feature extraction: Text modality data uses natural language processing techniques (such as BERT model, TF-IDF algorithm) to extract semantic features and keyword features, and transforms them into text feature vectors; image modality data uses deep learning networks (such as CNN, ResNet) to extract deep features such as image texture and contour, and generate image feature vectors; physiological signal modality data uses time series analysis methods (such as LSTM, temporal convolutional networks) to extract signal temporal features, and transforms them into physiological signal temporal feature vectors; structured modality data uses methods such as sliding window and feature encoding to extract the temporal change features of indicators, and generate structured temporal feature vectors.

[0135] Feature enhancement: Data normalization and standardization methods are used to unify the dimensions of features across modalities and avoid the impact of feature scale differences on subsequent processing; attention mechanisms are used to strengthen the weights of key features related to myasthenia gravis and suppress noisy features; for scarce modal features, data augmentation techniques (such as image rotation, text synonym replacement, and physiological signal interpolation) are used to supplement feature samples and improve feature stability.

[0136] It is understood that this embodiment effectively mines the core disease information in each modality of data through multimodal feature extraction, transforms the original messy multimodal data into structured and usable multimodal features, and removes redundant and noisy information; through feature enhancement processing, it improves the discriminativeness and stability of features, makes up for the limitations of single modality features, lays a solid foundation for subsequent cross-modal fusion and model training, and improves the model's ability to recognize and utilize disease features.

[0137] Step S30: Construct a three-dimensional dynamic feature matrix based on the multimodal features, and perform cross-modal feature fusion to obtain the target fused features.

[0138] It should be noted that the three-dimensional dynamic feature matrix refers to a three-dimensional matrix structure that integrates multimodal features in chronological order, using time, modality, and feature dimensions as three dimensions, and can reflect the changes in the patient's condition over time and the synergistic effect of multimodal features.

[0139] In some embodiments, the optimization device may first integrate multimodal temporal features to construct a three-dimensional matrix, and then use a fusion algorithm to eliminate modal differences to obtain the target fused features, which may specifically include:

[0140] Three-dimensional dynamic feature matrix construction: Time is the first dimension (sorted by data acquisition time node), modality type is the second dimension (text, image, physiological signal, and structured four modalities), and features extracted from each modality are the third dimension. Multimodal features at each time node are sequentially filled into the matrix to form a three-dimensional dynamic feature matrix, realizing the temporal and structured integration of multimodal features, and intuitively reflecting the dynamic changes of the patient's condition over time.

[0141] Cross-modal feature fusion: A cross-modal attention fusion network is adopted. First, the modal features in the three-dimensional dynamic feature matrix are aligned to eliminate the scale and distribution differences of different modal features. Then, the weights of each modal feature are adaptively allocated through the attention mechanism to highlight the modal features that are more valuable for the diagnosis and treatment decision of myasthenia gravis (such as myocardial imaging features and antibody-level structured features). Finally, the multimodal features are integrated into a unified target fusion feature through feature splicing, convolution fusion and other methods.

[0142] It is understood that this embodiment transforms dispersed multimodal features into a structured, temporally sequenced three-dimensional dynamic feature matrix, clearly capturing the dynamic changes in the patient's condition over time; by eliminating modal differences through cross-modal fusion, it integrates the advantageous features of each modality, avoids the one-sidedness of single modal features, and forms a more representative and discriminative target fusion feature, effectively improving the subsequent model's ability to identify and analyze the condition.

[0143] Step S40: Train the multi-task deep network model based on the target fusion features to obtain the diagnosis and treatment path optimization model.

[0144] It should be noted that the treatment pathway optimization model is configured to handle multiple related treatment tasks, achieve multi-task collaborative training, and be configured to perform adaptive score regression analysis, referral necessity analysis, and treatment effect prediction.

[0145] Understandably, fit score regression analysis refers to a model using a regression algorithm to output a score indicating the fit between the patient's current treatment plan and their condition, based on target fusion features, in order to determine the rationality of the treatment plan.

[0146] Referral necessity analysis refers to a model that, based on target fusion features, uses a classification algorithm to output the probability of whether a patient needs to be referred to a higher-level medical institution or specialist for treatment, thus providing a basis for referral decisions.

[0147] Treatment outcome prediction refers to the model using time-series prediction algorithms to output the trend of patient improvement and treatment outcome prediction after receiving the current or adjusted treatment plan, based on target fusion features, providing a reference for adjusting the treatment plan.

[0148] In some embodiments, the treatment pathway optimization model can adopt a "backbone network + multi-task branch" model structure. The backbone network adopts a Transformer or CNN-LSTM hybrid network to receive target fusion features and perform deep feature mining. The three task branches correspond to adaptability score regression, referral necessity analysis, and treatment effect prediction, respectively. The regression branch adopts a fully connected layer + regression activation function, the classification branch adopts a fully connected layer + classification activation function, and the prediction branch adopts a temporal prediction layer to achieve multi-task collaborative processing.

[0149] It is understood that this embodiment enables the model to simultaneously possess three core capabilities—adaptability score regression, referral necessity analysis, and treatment effect prediction—through multi-task collaborative training. This allows for the synchronous analysis of multiple treatment tasks, enhancing the model's comprehensive treatment decision-making ability. Furthermore, by utilizing deep networks to mine deep correlation information in the target fusion features, the accuracy and reliability of the model's evaluation results are improved. This provides efficient and accurate model support for the dynamic optimization of subsequent treatment pathways, replacing the subjectivity and limitations of traditional manual treatment evaluation.

[0150] Furthermore, it improves the efficiency of diagnosis and treatment assessment and avoids the limitations of single-task models. In one embodiment, the diagnosis and treatment path optimization model includes an input layer, a feature extraction layer, a fusion layer, and a multi-task output layer.

[0151] The input layer is configured to receive multimodal data structured as time series, including text modal data, image modal data, physiological signal modal data, and structured modal data.

[0152] The feature extraction layer is configured to perform feature extraction and feature enhancement processing on the multimodal data to obtain multimodal features. The multimodal features include text modality features, image modality features, physiological signal temporal features, and structural temporal features. The feature extraction layer includes a pre-trained bidirectional encoder representation module, a residual network fusion feature pyramid network architecture module, a one-dimensional convolutional neural network fusion long short-term memory network hybrid architecture module, and a temporal convolutional network module.

[0153] The fusion layer is configured to construct a three-dimensional dynamic feature matrix based on the multimodal features and perform cross-modal feature fusion to obtain the target fused features;

[0154] The multi-task output layer is configured for adaptability score regression analysis, referral necessity analysis, and treatment effect prediction.

[0155] In the specific implementation, refer to Figure 3 , Figure 3 The diagram below illustrates the architecture of the treatment pathway optimization model.

[0156] Input layer: Multimodal feature sequences, including text modality features, image modality features, physiological signal temporal features, and structural temporal features. );

[0157] Feature Extraction Layer: A feature extraction layer is built, integrating four dedicated sub-modules, corresponding to feature extraction and enhancement of four modalities of data, respectively. The specific configuration is as follows:

[0158] The pre-trained bidirectional encoder representation module adopts the BERT-base pre-trained model, with a word segmentation length of 128, a hidden layer dimension of 768, and a dropout probability of 0.3. It receives text modal data from the input layer, outputs initial text features, and performs L2 normalization enhancement.

[0159] The residual network fusion feature pyramid network architecture module adopts a fusion architecture of ResNet50 and FPN. The ResNet50 is configured with 16 residual blocks and the FPN is configured with 4 feature levels. It receives image modal data from the input layer, outputs multi-scale initial image features and enhances them through an attention mechanism.

[0160] The hybrid architecture module of one-dimensional convolutional neural network and long short-term memory network is as follows: 1D-CNN is configured with 3 convolutional layers (3 kernels and 1 stride), and LSTM is configured with 1 hidden layer (128 units). It receives physiological signal modal data from the input layer, first extracts local features through 1D-CNN, then captures long-term dependencies through LSTM, outputs the initial physiological signal temporal features and performs interpolation augmentation.

[0161] 4. Temporal Convolutional Network Module: The TCN is configured with four dilated convolutional layers (dilation coefficients of 1, 2, 4, and 8 respectively). It receives structured modal data from the input layer, expands the receptive field through dilation convolution, extracts initial structural features and normalizes them. Finally, the feature extraction layer integrates the outputs of the four sub-modules to obtain enhanced multimodal features, which are then fed into the fusion layer.

[0162] The fusion layer, configured to integrate and fuse multimodal features, specifically includes:

[0163] First, based on the structural temporal features output by the feature extraction layer, a two-dimensional patient state feature matrix is ​​constructed, and a one-dimensional medical institution capability feature vector is constructed based on the medical institution capability indicators. The two are then multiplied by an outer product to generate a three-dimensional dynamic feature matrix. The dynamic tensor features are extracted using a tensor decomposition algorithm, and the multimodal weights are calculated using a self-attention mechanism. The multimodal features are then weighted and fused to obtain the multimodal fusion features. Finally, the dynamic tensor features and multimodal fusion features extracted from the three-dimensional dynamic feature matrix are concatenated with the patient's diagnosis and treatment trajectory features. After unifying the feature scale, the target fusion features are output and fed into the multi-task output layer.

[0164] Multi-task output layer: Configured with three parallel task branches, corresponding to three types of diagnostic and assessment tasks, as follows:

[0165] Adaptability score regression analysis branch: A fully connected layer (output dimension 1) + linear activation function is used to receive the target fusion features output by the fusion layer, and the adaptation score between the patient's current treatment plan and condition is output through regression calculation (value range 0-10).

[0166] Referral necessity analysis branch: It adopts a fully connected layer (output dimension 2) + Softmax activation function, receives target fusion features, and calculates the probability of patients needing referral and not needing referral through binary classification. The referral determination probability is used as the core output indicator.

[0167] Treatment outcome prediction branch: LSTM temporal prediction layer (64 hidden units) + Sigmoid activation function is used to receive target fusion features, predict the patient's condition improvement trend in the future (e.g., 1 month), and output treatment outcome prediction results (improvement, stabilization, deterioration); the three branches are operated in parallel and output three types of treatment evaluation results simultaneously, which serve as the core basis for dynamic optimization of the treatment path.

[0168] Furthermore, the total loss function of the multi-task output layer is:

[0169]

[0170] in, Represents the total loss item. This represents the regression loss term for the fit score. This indicates the loss item categorized by the necessity of referral. This represents the loss term in the prediction of treatment effectiveness. and These represent the weights of the loss term, (Determined through validation set optimization, balancing the importance of primary and secondary tasks);

[0171] The target value of the fit score regression loss term is the fit score annotated by the expert. We use weighted MSE loss, and the loss function is:

[0172]

[0173] in, Indicates the number of samples. This represents the regression loss weight for the i-th sample. This represents the fitness score predicted by the model for the i-th sample. The expert annotation fit score of the i-th sample; weights (for extreme values ​​( or (Weighting enhances the model's ability to fit boundary cases).

[0174] The target value for the referral necessity classification loss item is a binary classification label. (0 = no referral needed, 1 = referral required), Focal Loss is used to address class imbalance:

[0175]

[0176] in, Represents the weight of positive examples (e.g.) ), Representing the hard case mining factors (e.g.) ), used to enhance attention to difficult-to-classify samples, This represents the referral necessity label for the i-th sample. This represents the referral probability of the i-th sample output by the model;

[0177] The target value for the treatment outcome prediction loss term is the change in QMG score after treatment. MAE loss is used:

[0178]

[0179]

[0180] in, This represents the change in the QMG score predicted by the model for the i-th sample. This represents the actual change in QMG score for the i-th sample. Indicates the baseline QMG score, This indicates the QMG score after treatment.

[0181] In some embodiments, the training strategy for the treatment pathway optimization model includes:

[0182] 1. Dataset partitioning:

[0183] Training set: 210 cases (70%), covering all subtypes of MG;

[0184] Validation set: 45 cases (15%), used for hyperparameter tuning;

[0185] Test set: 45 cases (15%), used for independent evaluation.

[0186] 2. Data Augmentation:

[0187] Text: Randomly replace synonyms (e.g., myasthenia gravis → MG) and insert irrelevant sentences (preserving semantics);

[0188] Image: Random rotation (±15°), scaling (0.8-1.2x), Gaussian noise (σ=0.01);

[0189] Physiological signals: time axis shift (±10%), amplitude perturbation (±5%);

[0190] Structured modal data: temporal interpolation (simulating missing values), index jitter (±3%).

[0191] 3. Optimizer configuration, using AdamW optimizer:

[0192]

[0193] in, and Let these represent the model parameters at the t-th and t+1-th iterations, respectively. This represents the learning rate in the t-th iteration. For the loss function with respect to the parameters The gradient (representing the direction and magnitude of parameter updates). This represents the weight decay coefficient. This represents the second-order moment estimate of the gradient. This represents a numerically stable term.

[0194] For example: initial learning rate (Main network) (Category Header);

[0195] Learning rate scheduling: cosine annealing, 100 epochs period, minimum ;

[0196] Weight decay , .

[0197] 4. Training process:

[0198] S1: Pre-training phase (50 epochs):

[0199] Freeze the fusion layer and output layer, and train only the feature extraction network;

[0200] Initialize weights: PubMedBERT is used for the text branch, and ResNet50 pre-trained for the image branch;

[0201] Batch size: Text 16, Image 8, Signal 32.

[0202] S2: Joint Training Phase (200 epochs):

[0203] Unfreeze all layers and train end-to-end; Batch size: 4 (comprehensive batch, mixed modality).

[0204] Gradient accumulation: Parameters are updated every 4 steps (equivalent batch 16).

[0205] 3. Fine-tuning phase (50 epochs):

[0206] The learning rate dropped to 1e-5;

[0207] Application of Label Smoothing ( );

[0208] Dropout rate = 0.3 (to prevent overfitting).

[0209] 5. Early shutdown mechanism:

[0210] When the validation set Training will be terminated if there is no improvement after 5 consecutive rounds:

[0211]

[0212] Step S50: Collect multimodal data of myasthenia gravis patients based on a preset time interval, input the multimodal data into the treatment path optimization model, output the treatment assessment results of the myasthenia gravis patients, and dynamically optimize the treatment path of the myasthenia gravis patients based on the treatment assessment results.

[0213] It should be noted that the preset time interval refers to the time period for regularly collecting multimodal data from patients with myasthenia gravis based on the severity of their condition (mild, moderate, severe) to monitor changes in their condition in real time.

[0214] It should be noted that the diagnosis and treatment assessment results refer to the comprehensive results output by the diagnosis and treatment pathway optimization model based on real-time collected multimodal data, used to assess the patient's current diagnosis and treatment status, including three core indicators: suitability score, referral determination probability, and diagnosis and treatment effect prediction results.

[0215] Among them, the suitability score is a quantitative score used to measure the degree of suitability between the current treatment plan and the patient's real-time condition. The higher the score, the more suitable the treatment plan is for the patient's condition.

[0216] The referral determination probability is the probability value that a patient needs to be referred for treatment. The higher the probability, the more likely the patient's current condition requires referral to a more specialized medical institution.

[0217] Treatment outcome prediction results can be the predicted trend of whether a patient's condition improves, stabilizes, or worsens after receiving the current treatment plan, and are used to judge the effectiveness of the treatment plan.

[0218] It should be noted that dynamic optimization of the treatment pathway can be based on the treatment evaluation results output by the model, combined with clinical treatment guidelines, to adjust the patient's treatment plan in real time (such as adjusting medication, adding or removing examination items, arranging referrals, changing treatment methods, etc.), so that the treatment pathway always adapts to the dynamic changes in the patient's condition.

[0219] In some embodiments, the optimization device can preset time intervals according to the severity of the patient's condition (once a month for mild patients, once every two weeks for moderate patients, and once a week for severe patients), and use the acquisition method in step S10 to periodically collect the patient's text, images, physiological signals, and structured multimodal data to ensure the real-time and timely nature of the data; the periodically collected multimodal data is then subjected to feature extraction, enhancement, and fusion according to the methods in steps S20 and S30 to obtain real-time target fusion features; the real-time target fusion features are input into the trained diagnosis and treatment path optimization model, and the patient's suitability is output. Scores, referral probability, and treatment outcome predictions form a complete treatment assessment. Based on the assessment results and clinical guidelines, personalized optimization strategies are developed: if the suitability score is low, the current medication dosage, examination items, or treatment methods are adjusted; if the referral probability is high, a referral recommendation is issued promptly, connecting with a higher-level specialist medical institution; if the treatment outcome prediction is unfavorable, the treatment plan is optimized in advance to avoid the risk of disease deterioration; after optimization, data is collected again at preset time intervals to enter the next round of monitoring-assessment-optimization closed loop, achieving dynamic adaptation of the treatment pathway.

[0220] Understandably, this embodiment enables real-time dynamic monitoring of the condition of patients with myasthenia gravis, promptly capturing changes in the patient's condition and avoiding the lag of traditional static treatment pathways; it accurately outputs treatment evaluation results through models, providing a scientific basis for adjusting treatment pathways and reducing the subjectivity and error of human decision-making; and it dynamically optimizes treatment pathways to ensure that treatment plans always adapt to changes in the patient's condition, improving the personalization and targeting of treatment, effectively avoiding unreasonable treatment behaviors, and improving the patient's treatment experience and treatment outcomes.

[0221] Furthermore, to improve the targeting and efficiency of treatment pathway optimization, in one embodiment, step S50 may include:

[0222] Step S501: Compare the suitability score with the first threshold and the second threshold, and compare the referral determination probability with the third threshold to obtain the comparison result, wherein the first threshold is greater than the second threshold;

[0223] Step S502: If the suitability score is not lower than the first threshold and the referral determination probability is lower than the third threshold, determine that the current institution is suitable for the myasthenia gravis patient user and maintain the current treatment path;

[0224] Step S503: If the suitability score is lower than the first threshold and not lower than the second threshold, optimize the diagnosis and treatment nodes in the diagnosis and treatment path of the myasthenia gravis patient user;

[0225] Step S504: If the fit score is lower than the second threshold, or the fit score decreases by more than a preset decrease threshold multiple times, a referral warning is triggered. An optimal referral target institution list is generated based on the medical institution capability feature vector, and the treatment path of the myasthenia gravis patient is adjusted based on the target institution list.

[0226] Step S505: Repeat the step of collecting multimodal data of myasthenia gravis patients based on a preset time interval and inputting the multimodal data into the treatment pathway optimization model to achieve dynamic optimization of the treatment pathway.

[0227] In practice, the trained model is deployed in the medical institution's information system to achieve real-time optimization of the diagnosis and treatment pathway. The process is as follows:

[0228] First, real-time adaptability assessment:

[0229] Input: Patient multimodal data (real-time updated electronic medical records, latest imaging / signal examinations, structured indicators);

[0230] Feature extraction: Extract multimodal features and perform enhancement processing;

[0231] Feature fusion computation: computation through a trained attention mechanism spliced ​​together ;

[0232] Model prediction: Input At most a task output layer, we get:

[0233] Adaptability score (Main task output);

[0234] Referral determination probability (Auxiliary task 1 output);

[0235] Expected QMG improvement value (Auxiliary task 2 output).

[0236] Second, dynamic path decision rules:

[0237] Based on fit score Make the following decisions:

[0238] 1. If the adaptability score is not lower than the first threshold (e.g.) (score), and the referral determination probability is lower than the third threshold (e.g. If ), then maintain the current path, for example: if Divide, and The system indicates that the current institution and the patient's condition are a good match, and recommends continuing the current treatment.

[0239] 2. If the adaptability score is lower than the first threshold (e.g.) (points), and not lower than the second threshold (e.g.) If (points), then optimize the diagnosis and treatment nodes, for example: if The report provides specific improvement suggestions (e.g., "It is recommended to complete a thymus CT assessment within 3 days").

[0240] 3. If the suitability score is lower than the second threshold (e.g., 5 points), or if the suitability score decreases by more than a preset decrease threshold (e.g., 2 points) multiple times consecutively, a referral warning is triggered. For example: if A score, or a decrease in score for three consecutive assessments. The score triggers a referral alert.

[0241] Target institution matching: based on real-time capability scores of medical institutions The formula for generating the top 3 referral candidate hospitals is as follows:

[0242]

[0243] ( For the collection of regional medical institutions, For patients With institutions (fitness prediction score).

[0244] Third, output and presentation:

[0245] Output via a visual dashboard:

[0246] Adaptability score trend chart (last 12 weeks);

[0247] Referral decision-making criteria (radar chart of scores for each dimension);

[0248] Expected Results Prediction ( (and 95% confidence interval);

[0249] A list of matching institutions (including referral distance, response time, and reasons for recommendation).

[0250] Fourth, closed-loop update mechanism:

[0251] During the diagnosis and treatment process, the model is reassessed periodically (every 2 weeks):

[0252] Enter newly generated medical data (such as the latest QMG score and response after medication adjustment);

[0253] Update patient status characteristics ;

[0254] Recalculate the fit score and dynamically adjust the path recommendation.

[0255] In some embodiments, the optimization device quantifies the MG diagnostic suitability into a suitability score. The suitability score is obtained by weighted fusion of three dimensions: patient status, medical institution capabilities, and quality of diagnosis and treatment nodes. The mathematical definition is:

[0256]

[0257] in, is the Sigmoid normalization function (maps the score to [0,10]). Indicates the fit score. (Patient status score) (Score for medical institutions) (The scores for each diagnosis and treatment node) account for 40%, 30%, and 30% respectively, and the specific calculations are as follows:

[0258] (1) Patient status dimension score Taking into account clinical indicators, disease stability, and social factors, the mathematical model is as follows:

[0259]

[0260] in:

[0261] (Quantitative myasthenia gravis score, standardized to [0,1]) );

[0262] Antibody type indicator variable (AChR antibody positive = 1, MuSK = 0.7, negative = 0.3);

[0263] (Standardization of the number of comorbidities) (Number of comorbidities)

[0264] (Standardized weekly mean of symptom fluctuation frequency) (Number of symptom exacerbations per week);

[0265] (Standardization of crisis risk levels) );

[0266] (Standardization of family care competence scoring) );

[0267] (Standardization of medical access convenience scoring) );

[0268] Weights satisfy It was determined by expert consensus to be .

[0269] (2) Score of medical institution dimension The mathematical model for assessing diagnostic and treatment capabilities, technology configuration, and collaborative networks is as follows:

[0270]

[0271] in:

[0272] (Standardization of the number of MG specialist physicians) (Number of physicians)

[0273] (Standardization of annual number of outpatient cases) (Number of cases per year)

[0274] Completeness of crisis management equipment (0-1, 1 = equipped with ventilator + ICU bed + plasma exchange equipment);

[0275] : Thymus imaging examination capability (0-1, 1 = can independently complete CT+MRI+radiomics analysis);

[0276] : Neurophysiological testing level (0-1, 1 = possessing RNS + single-fiber electromyography + repetitive nerve stimulation testing);

[0277] (Standardization of referral response time) (for response hours).

[0278] (Standardization of multidisciplinary consultation efficiency) (For consultation response hours)

[0279] Weights satisfy It was determined by expert consensus to be .

[0280] (3) Scoring of diagnosis and treatment nodes The mathematical model for quantifying the timeliness of intervention, the accuracy of decision-making, and the degree of improvement in prognosis is as follows:

[0281]

[0282] in:

[0283] (Timeliness of intervention) This is the actual delay time. For standard time windows, such as crisis response Hour);

[0284] (Decision accuracy, (Probability of misdiagnosis);

[0285] (Prognostic improvement rate);

[0286] Weights satisfy It was determined by expert consensus to be .

[0287] In some embodiments, the threshold corresponding to the fit score used for treatment pathway optimization is used to construct a gold standard for fit score through annotation by an expert committee:

[0288] 1. Expert Committee Composition: 15 members (8 neurologists with an annual MG treatment volume of >200 cases; 7 health management experts with backgrounds in hierarchical medical system research).

[0289] 2. Case selection: 300 complete MG diagnosis and treatment cases (covering mild / moderate / severe, AChR / MuSK / negative antibody typing, treatment-naïve / refractory cases);

[0290] 3. Labeling process: Experts independently score the suitability of each diagnosis and treatment node (such as diagnosis, medication adjustment, and crisis management) in the case (Likert 10-point scale).

[0291] 4. Consistency test: Calculate the within-group correlation coefficient (ICC), the formula is:

[0292]

[0293] in, The variance between cases, The variance between raters The variance represents random error. Target ICC > 0.85 (excellent labeling consistency).

[0294] This embodiment collects multimodal data from patients with myasthenia gravis, extracts and enhances these data to obtain multimodal features. Based on these multimodal features, a three-dimensional dynamic feature matrix is ​​constructed and cross-modal fusion is performed to obtain target fusion features. A multi-task deep network model is trained using these target fusion features to obtain a treatment pathway optimization model. Data is dynamically collected at preset intervals and input into the model, outputting treatment evaluation results to achieve dynamic optimization of the treatment pathway. This transforms discrete treatment pathways into quantifiable dynamic adaptation problems. Multimodal fusion overcomes the limitations of single data dimensions, improving decision-making accuracy. The dynamic feature matrix captures spatiotemporal correlations, enabling real-time updates of the treatment pathway. Because this embodiment performs multimodal fusion based on comprehensive multimodal data collection... By extracting dynamic features and enhancing features, the system effectively mines, integrates, and utilizes disease characteristics. Through precise evaluation using a multi-task model and dynamic optimization of treatment pathways, it breaks through the limitations of traditional myasthenia gravis diagnosis and treatment, which rely on single data, have fixed static pathways, and rely on subjective human decision-making. It effectively mines multi-dimensional disease characteristics of patients, improving the accuracy and scientific nature of diagnosis and treatment assessment; it enables personalized and dynamic adaptation of treatment pathways, improving the rationality and timeliness of diagnosis and treatment decisions; it reduces irrational treatment behaviors, lowers treatment costs, and improves treatment efficiency; it provides reliable intelligent technical support for clinical diagnosis and treatment, promoting the transformation of myasthenia gravis diagnosis and treatment models from traditional experience-based to precise and intelligent, ultimately achieving the core goal of improving patient treatment outcomes and enhancing the quality of treatment.

[0295] refer to Figure 4 , Figure 4 This is a flowchart illustrating the second embodiment of the dynamic diagnosis and treatment path optimization method for myasthenia gravis based on multimodal fusion of the present invention.

[0296] Based on the first embodiment described above, in this embodiment, step S20 further includes:

[0297] Step S201: Use a pre-trained bidirectional encoder representation model to extract features from the text modal data to obtain initial text features.

[0298] It should be noted that the pre-trained bidirectional encoder representation model can be a BERT-based (Bidirectional Encoder Representations from Transformers) model. BERT-based models possess bidirectional semantic understanding capabilities, enabling them to deeply mine semantic features from text data and adapt to specific text feature extraction tasks without retraining.

[0299] In the specific implementation, the optimized device uses the BERT-base model to extract global text features:

[0300] Input layer: Medical record text is converted into embedded vectors after being segmented by WordPiece. ;

[0301] Feature extraction: The parameters of the first 6 layers are frozen (based on PubMedBERT pre-trained weights), and the last 6 layers are fine-tuned, outputting the [CLS] token as a global feature.

[0302]

[0303] in, This represents the embedding vector obtained after word segmentation of the medical record text. This represents the initial text features (i.e., global text features).

[0304] Step S202: Use a residual network to fuse a feature pyramid network architecture to extract multi-scale features from the image modal data to obtain initial image features.

[0305] It should be noted that the Residual Network Fusion Feature Pyramid Network architecture refers to the ResNet50+FPN architecture. ResNet effectively solves the gradient vanishing problem in deep network training and is used to extract deep texture and contour features from image data. The Feature Pyramid Network (FPN) is a multi-scale feature extraction architecture that can simultaneously extract shallow detail features and deep semantic features from image data, improving the comprehensiveness of image features.

[0306] In its implementation, the optimization device employs a ResNet50+FPN (Residual Network 50 + FeaturePyramid Network) architecture to extract multi-scale image features:

[0307] Low-level features: The first 4 layers of ResNet50 extract low-level visual features. ;

[0308] Multi-scale fusion: FPN generates feature maps at 5 scales (P2-P6), and unifies the dimensions to 256 through 1×1 convolution.

[0309]

[0310] in, This represents the result of stitching together the 5 scale feature maps generated by FPN. The initial image features are... Perform global average pooling to obtain initial image features. .

[0311] Step S203: A one-dimensional convolutional neural network fused with a long short-term memory network hybrid architecture is used to dynamically extract features from the physiological signal modal data to obtain the initial physiological signal features.

[0312] It should be noted that the hybrid architecture of one-dimensional convolutional neural networks combined with long short-term memory networks can be a 1D-CNN + LSTM hybrid architecture. One-dimensional convolutional neural networks (1D-CNN) are suitable for temporal signal processing, and can extract local temporal features from physiological signals to capture short-term change patterns. Long short-term memory networks (LSTM) have the ability to remember long-term dependencies, and can be combined with 1D-CNN to extract long-term dynamic change features of physiological signals.

[0313] In practical implementation, the optimized device can use a hybrid architecture of 1D-CNN + LSTM (One-Dimensional Convolutional Neural Network + Long Short-Term Memory) to extract dynamic features of the signal:

[0314] Temporal features: Extracting RNS waveform features using 3-layer 1D convolution:

[0315]

[0316] in, , Represents raw physiological signal data. ( (length after downsampling) This represents the RNS waveform features extracted by a 3-layer 1D convolution.

[0317] Temporal modeling: Bidirectional LSTM captures dynamic changes (forward / backward hidden state concatenation):

[0318]

[0319] in, This represents the initial physiological signal features extracted by bidirectional LSTM.

[0320] Step S204: Use a temporal convolutional network to extract temporal features from the structured modal data to obtain initial structural features.

[0321] It should be noted that Temporal Convolutional Network (TCN) refers to a convolutional network for processing time-series data. It can effectively extract the temporal features of indicators changing over time in structured modal data and capture the temporal correlation of data.

[0322] In the specific implementation, the optimized device uses a Temporal Convolutional Network (TCN) to extract temporal features:

[0323] Time-series coding: Inspection indicators Injecting timing information through position coding ;

[0324] Convolutional module: 3 layers of dilated convolutions (dilation rates 20, 21, 2^2) to extract temporal dependencies:

[0325]

[0326] in, This represents the temporal dependency features extracted by the temporal convolutional network. This indicates the time-series information of the inspection indicators after location coding;

[0327] Global temporal features: Initial structural features are obtained through weighted summation using temporal attention. .

[0328] In some embodiments, the optimization device preprocesses the structured modal data (missing value imputation, standardization, and temporal sorting); pre-constructs a temporal convolutional network adapted to the structured temporal data, sets reasonable dilation coefficients and convolutional kernel sizes, and expands the network's receptive field to capture long-range temporal correlations; converts the preprocessed structured temporal data into network input vectors and inputs them into the temporal convolutional network; ensures temporal rationality through causal convolution of the temporal convolutional network, captures the correlations of structured data at different time nodes through dilated convolution, and outputs feature vectors, which are the initial structural features.

[0329] Step S205: Perform feature enhancement processing on the initial text features, the initial image features, the initial physiological signal features, and the initial structural features respectively to obtain multimodal features.

[0330] In some embodiments, the optimization device can enhance the four initial features mentioned above respectively, and use the L2 normalization method to unify the dimensions of each initial feature; use the attention mechanism to assign weights to the initial features, strengthen the feature weights related to myasthenia gravis, and suppress noise features; use the interpolation augmentation method to supplement the samples for the initial features of scarce modalities (such as some image and physiological signal features), and finally obtain the denoised and enhanced multimodal features for subsequent matrix construction and fusion.

[0331] Furthermore, to improve the quality of multimodal features and thus enhance evaluation accuracy, step S205 may include:

[0332] Step S2051: Map the extracted clinical entities to entity embedding vectors, and use the entity embedding vectors to enhance the initial text features to obtain text modality features.

[0333] Understandably, device optimization can employ entity feature enhancement: extracting clinical entities (such as "ptosis" and "pyridostigmine") using the BERT-CRF model and mapping them to entity embedding vectors. ,and For splicing, refer to the following formula:

[0334]

[0335] in, Representing text modal features, Represents entity weight ( ), The entity embedding vector representing the clinical entity. This represents the initial text features.

[0336] The BERT-CRF model is a deep learning architecture that combines BERT (Bidirectional Encoder Representations from Transformers) and CRF (Conditional Random Field).

[0337] Step S2052: Perform pooling processing on the region of interest in the lesion image to extract lesion features, and perform feature enhancement on the initial image features based on the lesion features to obtain image modal features.

[0338] In practical implementation, the optimization equipment can employ lesion feature enhancement, performing 3D pooling on the ROI (Region of Interest) of thymus CT to extract lesion features. ,and For splicing, refer to the following formula:

[0339]

[0340] in, Represents image modal features, Indicates the weight of the lesion ( ), This refers to lesion features extracted by pooling the region of interest in a lesion image (e.g., lesion features extracted by 3D pooling of the ROI region in a thymus CT scan). This indicates the initial image features.

[0341] Step S2053: Extract the RNS decay rate feature through the convolution kernel of the temporal convolutional network, and perform feature enhancement processing on the initial physiological signal feature based on the RNS decay rate feature to obtain the temporal feature of the physiological signal.

[0342] It should be noted that the RNS decay rate characteristic refers to the characteristic corresponding to the degree of decay of the amplitude of physiological signals (such as electromyography) during repetitive neural electrical stimulation (RNS), which is used to reflect the neuromuscular functional state.

[0343] In practical implementation, the optimized device can employ attenuation rate feature enhancement, extracting RNS attenuation rate features through a dedicated convolutional kernel. ,and For splicing, refer to the following formula:

[0344]

[0345] in, This represents the RNS decay rate feature extracted by the convolution kernel. Indicates the decay rate weight ( ), Indicates the temporal characteristics of physiological signals. This indicates the characteristics of the initial physiological signal.

[0346] In some embodiments, the optimization device filters out physiological signal segments related to repetitive neural electrical stimulation (RNS) from the preprocessed physiological signal modal data; it extracts RNS decay rate features using the convolution kernels of a pre-trained temporal convolutional network adapted to the RNS signals, slides along the time dimension to extract amplitude decay-related features in the RNS signal segments, and obtains an RNS decay rate feature vector; it fuses the RNS decay rate features with the initial physiological signal features obtained in the third step using feature superposition or weighted fusion to enhance the dynamic information related to neuromuscular function in the initial physiological signal features; it smooths the fused feature vector to eliminate noise interference and outputs the processed feature vector, which is the physiological signal temporal feature.

[0347] Step S2054: Input the QMG score time series of myasthenia gravis patients into a pre-trained long short-term memory network for time series analysis to obtain trend features. The QMG score time series is extracted based on structured modal data.

[0348] Step S2055: Perform feature enhancement processing on the initial structural features based on the trend features to obtain structural temporal features.

[0349] In practical implementation, device optimization can employ QMG trend enhancement and QMG scoring timing. Extracting trend features using LSTM ,and The splicing should be done according to the following formula:

[0350]

[0351] in, Represents the temporal characteristics of the structure. Indicates the QMG score weight ( ), This indicates the trend characteristics of the QMG score over time. This represents the initial structural features.

[0352] It should be noted that the QMG score, or Quantitative Myasthenia Gravis Score, is a standardized clinical scale used to objectively assess the severity of myasthenia gravis (MG) in patients. This score helps determine disease activity, treatment response, and disease progression by quantifying the strength and endurance of the affected muscle groups.

[0353] This embodiment achieves accurate extraction of core features from various modalities through a targeted feature extraction architecture, adapting to the characteristics of different modalities and improving the representativeness and discriminative power of initial features. The application of pre-trained models and hybrid network architectures reduces model training costs and improves feature extraction efficiency. Through feature enhancement processing, noise and redundancy in the initial features are effectively suppressed, feature dimensions are unified, key disease features are strengthened, and the limitations of single feature extraction are overcome. This ensures that the obtained multimodal features are of higher quality and stronger stability, providing reliable feature support for the construction of a three-dimensional dynamic feature matrix and cross-modal fusion, and improving the training accuracy and evaluation effect of subsequent diagnosis and treatment path optimization models.

[0354] refer to Figure 5 , Figure 5 This is a flowchart illustrating the third embodiment of the dynamic diagnosis and treatment path optimization method for myasthenia gravis based on multimodal fusion of the present invention.

[0355] Based on the above embodiments, in this embodiment, step S30 further includes:

[0356] Step S301: Construct a patient state feature matrix based on the structural temporal features, and construct a medical institution capability feature vector based on the medical institution capability indicators.

[0357] It should be noted that the patient status feature matrix can be a two-dimensional matrix constructed based on structural temporal features, with time as the row and structural temporal features as the column. It is used to centrally characterize the patient's condition status and change patterns at different time points, reflecting the individual patient's condition.

[0358] Medical institution capability indicators refer to a set of quantitative and qualitative indicators used to measure a medical institution's ability to diagnose and treat myasthenia gravis, covering core dimensions such as the level of diagnostic and treatment equipment, the professional competence of medical staff, treatment experience, and specialist qualifications. Medical institution capability feature vectors can be feature vectors obtained by standardizing and encoding the medical institution capability indicators. These vectors are used to quantitatively represent the medical institution's diagnostic and treatment capabilities, achieving a calculable match with patient condition characteristics.

[0359] Step S302: Perform an outer product operation between the patient state feature matrix and the medical institution capability feature vector to obtain a three-dimensional dynamic feature matrix.

[0360] In the specific implementation, the multimodal features are organized into a three-dimensional dynamic feature matrix of "patient status-time series-institutional capability", as shown in the following formula:

[0361]

[0362] in, Represents a three-dimensional dynamic feature matrix. Represents the patient state feature matrix. This represents the feature vector of a medical institution's capabilities. This represents the outer product operation, used to broadcast the capability feature vector of a medical institution to the time dimension. The dimension of the three-dimensional dynamic feature matrix is ​​represented. Represents the patient's state dimension. Indicates the time dimension. This indicates the capability dimension of medical institutions.

[0363] Step S303: Extract dynamic tensor features based on the three-dimensional dynamic feature matrix.

[0364] It should be noted that dynamic tensor features can be high-order features extracted from a three-dimensional dynamic feature matrix, which can capture the relationship between patient status, medical institution capabilities, and time, highlighting the synergistic effect of multi-dimensional information.

[0365] In some embodiments, the optimization device may use a tensor decomposition algorithm to extract features from the three-dimensional dynamic feature matrix, decompose it to obtain core tensor components, screen out dynamic tensor features that can reflect the adaptation relationship between the patient's condition and the medical institution's capabilities, transform them into feature vectors that can be used for subsequent fusion, and retain high-order correlation information.

[0366] Step S304: Calculate the multimodal weights using an attention mechanism, and perform weighted fusion of the multimodal features based on the multimodal weights to obtain multimodal fused features.

[0367] In its implementation, the optimization device dynamically allocates the weights of each modality through an attention mechanism, as shown in the following formula:

[0368]

[0369]

[0370] in, Indicates multimodal fusion features, and They represent the first The and the first Modal features, This represents the multimodal weights of the i-th mode. This represents a multilayer perceptron (e.g., 2-layer perceptron). ).

[0371] Step S305: Model the patient state sequence based on the patient state feature matrix, and extract the patient treatment trajectory features based on the patient treatment trajectory.

[0372] It should be noted that the patient state sequence can be a time series obtained by modeling the patient state feature matrix in chronological order, which can be used to intuitively reflect the dynamic evolution of the patient's condition over time.

[0373] Patient treatment trajectory characteristics can be extracted from the patient's past and current treatment records, characterizing the patient's treatment process (medication, examination, treatment method adjustment, etc.), and reflecting the correlation between treatment behavior and changes in the condition.

[0374] Furthermore, in order to accurately extract the patient's treatment trajectory features, step S305 above may include:

[0375] Step S3051: Model the patient state sequence based on the patient state feature matrix and intervention actions.

[0376] In practical implementation, the optimized device can model the patient's state sequence and intervention actions as trajectory features, referring to the following formula:

[0377]

[0378] The state transition equation is:

[0379]

[0380] in, Represents a sequence of patient states. This represents the patient's state at time n. Indicates the first Intervention actions at any time, and This represents the patient's state at time t and t+1. Represents the state transition function. Let represent the random perturbation at time t. Indicates the first Intervention actions at specific moments;

[0381] Step S3052: Input the patient state sequence into the Transformer encoder to extract the patient's diagnosis and treatment trajectory features.

[0382] In the specific implementation, a Transformer encoder is used to extract patient treatment trajectory features, referring to the following formula:

[0383]

[0384] in, This indicates the characteristics of the patient's medical treatment trajectory. This indicates feature splicing.

[0385] Step S306: The dynamic tensor features, the multimodal fusion features, and the patient diagnosis and treatment trajectory features are concatenated to obtain the target fusion features.

[0386] In the specific implementation, the multimodal fusion features, dynamic tensor features, and patient diagnosis and treatment trajectory features are spliced ​​together, referring to the following formula:

[0387]

[0388] in, Indicates target fusion features, Represents the characteristics of a dynamic tensor. Indicates multimodal fusion features, This indicates the characteristics of the patient's medical treatment trajectory. This indicates a splicing operation.

[0389] This embodiment constructs a patient state feature matrix and a medical institution capability feature vector to quantitatively represent the individual patient's condition and the medical institution's capabilities, providing a data foundation for the adaptation analysis of the two. External product operations and dynamic tensor feature extraction effectively uncover higher-order correlation information between the patient's condition and the medical institution's capabilities, breaking the limitations of single-dimensional features. Adaptive allocation of multimodal weights through an attention mechanism highlights modal features more valuable for diagnosis and treatment decisions, improving the rationality and effectiveness of multimodal fusion. Patient state sequence modeling and patient treatment trajectory feature extraction supplement key information on the evolution of the patient's condition and the treatment process, enriching the feature dimensions. The target fusion feature obtained by splicing multiple types of features integrates multiple core information aspects, possessing stronger representativeness and discriminative power, effectively improving the training effect of subsequent multi-task deep network models, and providing high-quality feature input for the diagnosis and treatment path optimization model to achieve accurate adaptation analysis, referral judgment, and effect prediction.

[0390] In one embodiment, to verify the technical effect of the present invention, experimental verification was conducted:

[0391] Data source: 300 MG cases from 2019 to 2022 from 3 tertiary hospitals (neuromuscular disease treatment centers) (210 cases in training set, 45 cases in validation set, and 45 cases in test set), covering AChR antibody positive (62%), MuSK positive (18%), and antibody negative (20%), with a mild / moderate / severe ratio of 3:5:2.

[0392] Expert Committee: 15 members (8 neurologists with an annual MG treatment volume of >200 cases; 7 health management experts), with ICC=0.88 (excellent consistency).

[0393] Comparison methods include:

[0394] Traditional guide recommendations (2020 China MG Guide);

[0395] Single-modal model (text / image / signal / structured modal data only);

[0396] Clinical expert panel (3 senior neurologists).

[0397] Evaluation indicators include:

[0398] Model performance: Pearson correlation coefficient between predicted scores and expert ratings MAE (mean absolute error) and RMSE (root mean square error) are the two most commonly used error assessment indicators in regression models.

[0399] Clinical utility: referral decision compliance rate (consistency with expert decision-making), crisis prediction F1 score;

[0400] Patient follow-up results: Improvement in QMG over 3 months (i.e. ), incidence of crisis.

[0401] The experimental results are as follows:

[0402] (1) Model performance, see Table 1 below:

[0403] Table 1. Model Performance Comparison Table

[0404]

[0405] (2) Clinical efficacy, see Table 2 below:

[0406] Table 2. Comparison of Clinical Effects

[0407]

[0408] (3) The follow-up results of the patients are shown in Table 3 below:

[0409] Table 3. Comparison of Patient Follow-up Results

[0410]

[0411] Therefore, the multimodal adaptation method proposed in this application significantly improves the scientific rigor and accuracy of the MG diagnosis and treatment pathway, and the model performance approaches expert level. (v vs. experts 0.92), the referral decision compliance rate reached 83.6%, the improvement of patients' QMG by 62% in 3 months and the incidence of crisis decreased by 57%, which verified the feasibility of the program in optimizing the treatment pathway.

[0412] Furthermore, this embodiment of the invention also proposes a computer-readable storage medium storing a dynamic diagnosis and treatment path optimization program for myasthenia gravis based on multimodal fusion. When the dynamic diagnosis and treatment path optimization program for myasthenia gravis based on multimodal fusion is executed by a processor, it implements the steps of the dynamic diagnosis and treatment path optimization method for myasthenia gravis based on multimodal fusion as described above.

[0413] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0414] The aforementioned computer-readable storage medium may be included in the multimodal fusion-based dynamic diagnosis and treatment pathway optimization device for myasthenia gravis; or it may exist independently and not assembled into the multimodal fusion-based dynamic diagnosis and treatment pathway optimization device for myasthenia gravis.

[0415] Furthermore, this invention also proposes a computer program product, including a dynamic diagnosis and treatment path optimization program for myasthenia gravis based on multimodal fusion. When the dynamic diagnosis and treatment path optimization program for myasthenia gravis based on multimodal fusion is executed by a processor, it implements the steps of the dynamic diagnosis and treatment path optimization method for myasthenia gravis based on multimodal fusion as described above.

[0416] The specific implementation of the computer program product of the present invention is basically the same as the embodiments of the above-mentioned dynamic diagnosis and treatment path optimization method for myasthenia gravis based on multimodal fusion, and will not be repeated here.

[0417] Reference Figure 6 , Figure 6 This is a structural block diagram of the first embodiment of the dynamic diagnosis and treatment path optimization system for myasthenia gravis based on multimodal fusion of the present invention.

[0418] like Figure 6 As shown in the embodiments of the present invention, the dynamic diagnosis and treatment path optimization system for myasthenia gravis based on multimodal fusion includes:

[0419] The data acquisition module 10 is used to collect multimodal data of patients with myasthenia gravis, including text modal data, image modal data, physiological signal modal data and structured modal data.

[0420] The feature extraction module 20 is used to perform feature extraction and feature enhancement processing on the multimodal data to obtain multimodal features, including text modal features, image modal features, physiological signal time series features, and structural time series features;

[0421] The feature fusion module 30 is used to construct a three-dimensional dynamic feature matrix based on the multimodal features and perform cross-modal feature fusion to obtain the target fused features;

[0422] The model training module 40 is used to train a multi-task deep network model based on the target fusion features to obtain a treatment path optimization model. The treatment path optimization model is configured to perform adaptive score regression analysis, referral necessity analysis, and treatment effect prediction.

[0423] The treatment pathway dynamic optimization module 50 is used to collect multimodal data of myasthenia gravis patients based on a preset time interval, input the multimodal data into the treatment pathway optimization model, output the treatment assessment results of the myasthenia gravis patients, and dynamically optimize the treatment pathway of the myasthenia gravis patients based on the treatment assessment results. The treatment assessment results include the suitability score, the referral determination probability, and the treatment effect prediction results.

[0424] Furthermore, the feature extraction module 20 is also used to extract features from text modal data using a pre-trained bidirectional encoder representation model to obtain initial text features; to extract multi-scale features from image modal data using a residual network fused with a feature pyramid network architecture to obtain initial image features; to extract dynamic features from physiological signal modal data using a one-dimensional convolutional neural network fused with a long short-term memory network hybrid architecture to obtain initial physiological signal features; to extract temporal features from structured modal data using a temporal convolutional network to obtain initial structural features; and to perform feature enhancement processing on the initial text features, the initial image features, the initial physiological signal features, and the initial structural features respectively to obtain multimodal features.

[0425] This embodiment collects multimodal data from patients with myasthenia gravis, extracts and enhances these data to obtain multimodal features. Based on these multimodal features, a three-dimensional dynamic feature matrix is ​​constructed and cross-modal fusion is performed to obtain target fusion features. A multi-task deep network model is trained using these target fusion features to obtain a treatment pathway optimization model. Data is dynamically collected at preset intervals and input into the model, outputting treatment evaluation results to achieve dynamic optimization of the treatment pathway. This transforms discrete treatment pathways into quantifiable dynamic adaptation problems. Multimodal fusion overcomes the limitations of single data dimensions, improving decision-making accuracy. The dynamic feature matrix captures spatiotemporal correlations, enabling real-time updates of the treatment pathway. Because this embodiment performs multimodal fusion based on comprehensive multimodal data collection... By extracting dynamic features and enhancing features, the system effectively mines, integrates, and utilizes disease characteristics. Through precise evaluation using a multi-task model and dynamic optimization of treatment pathways, it breaks through the limitations of traditional myasthenia gravis diagnosis and treatment, which rely on single data, have fixed static pathways, and rely on subjective human decision-making. It effectively mines multi-dimensional disease characteristics of patients, improving the accuracy and scientific nature of diagnosis and treatment assessment; it enables personalized and dynamic adaptation of treatment pathways, improving the rationality and timeliness of diagnosis and treatment decisions; it reduces irrational treatment behaviors, lowers treatment costs, and improves treatment efficiency; it provides reliable intelligent technical support for clinical diagnosis and treatment, promoting the transformation of myasthenia gravis diagnosis and treatment models from traditional experience-based to precise and intelligent, ultimately achieving the core goal of improving patient treatment outcomes and enhancing the quality of treatment.

[0426] The multimodal fusion-based dynamic diagnosis and treatment pathway optimization system for myasthenia gravis provided in this application employs the multimodal fusion-based dynamic diagnosis and treatment pathway optimization method for myasthenia gravis in the above embodiments, and can solve the technical problem of dynamic diagnosis and treatment pathway optimization for myasthenia gravis based on multimodal fusion. Compared with the prior art, the beneficial effects of the multimodal fusion-based dynamic diagnosis and treatment pathway optimization system for myasthenia gravis provided in this application are the same as the beneficial effects of the multimodal fusion-based dynamic diagnosis and treatment pathway optimization method for myasthenia gravis provided in the above embodiments, and other technical features of the multimodal fusion-based dynamic diagnosis and treatment pathway optimization system for myasthenia gravis are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0427] It should be understood that the above are merely illustrative examples and do not constitute any limitation on the technical solutions of the present invention. In specific applications, those skilled in the art can make settings as needed, and the present invention does not impose any restrictions on this.

[0428] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this invention. In practical applications, those skilled in the art can select some or all of the workflow to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0429] In addition, for technical details not described in detail in this embodiment, please refer to the method for optimizing dynamic diagnosis and treatment path for myasthenia gravis based on multimodal fusion provided in any embodiment of the present invention, which will not be repeated here.

[0430] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0431] It should be noted that the user information (including but not limited to user device information, user personal information, user location information, user behavior information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0432] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0433] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0434] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for optimizing dynamic diagnosis and treatment pathways for myasthenia gravis based on multimodal fusion, characterized in that, The multimodal fusion-based dynamic diagnosis and treatment pathway optimization method for myasthenia gravis includes: Multimodal data of patients with myasthenia gravis were collected, including text modal data, image modal data, physiological signal modal data, and structured modal data. Feature extraction and feature enhancement processing are performed on the multimodal data to obtain multimodal features, which include text modal features, image modal features, physiological signal temporal features, and structural temporal features. A three-dimensional dynamic feature matrix is ​​constructed based on the multimodal features, and cross-modal feature fusion is performed to obtain the target fused features; The multi-task deep network model is trained based on the target fusion features to obtain a treatment path optimization model. The treatment path optimization model is configured to perform adaptability score regression analysis, referral necessity analysis, and treatment effect prediction. Multimodal data of myasthenia gravis patients are collected at preset time intervals and input into the treatment path optimization model. The model outputs the treatment assessment results of the myasthenia gravis patients and dynamically optimizes the treatment path of the myasthenia gravis patients based on the treatment assessment results. The treatment assessment results include the suitability score, the probability of referral, and the prediction results of treatment effect. The process of constructing a three-dimensional dynamic feature matrix based on the multimodal features and performing cross-modal feature fusion to obtain the target fused features includes: A patient state feature matrix is ​​constructed based on the aforementioned structural temporal features, and a medical institution capability feature vector is constructed based on the medical institution capability indicators. The patient state feature matrix is ​​multiplied by the medical institution capability feature vector to obtain a three-dimensional dynamic feature matrix, as shown in the following formula: in, Represents a three-dimensional dynamic feature matrix. Represents the patient state feature matrix. This represents the feature vector of a medical institution's capabilities. This represents the outer product operation, used to broadcast the capability feature vector of a medical institution to the time dimension. The dimension of the three-dimensional dynamic feature matrix is ​​represented. Represents the patient's state dimension. Indicates the time dimension. This indicates the capability dimension of the medical institution; Dynamic tensor features are extracted based on the aforementioned three-dimensional dynamic feature matrix; The multimodal weights are calculated using an attention mechanism, and the multimodal features are then weighted and fused based on these weights to obtain the multimodal fused features, as shown in the following formula: in, Indicates multimodal fusion features, and They represent the first The and the first Modal features, This represents the multimodal weights of the i-th mode. This represents a multilayer perceptron; The patient state sequence is modeled based on the patient state feature matrix, and the patient treatment trajectory features are extracted based on the patient treatment trajectory. The dynamic tensor features, the multimodal fusion features, and the patient's diagnostic trajectory features are concatenated to obtain the target fusion features, as shown in the following formula: in, Indicates target fusion features, Represents the characteristics of a dynamic tensor. Indicates multimodal fusion features, This indicates the characteristics of the patient's medical treatment trajectory. This indicates a splicing operation.

2. The method for optimizing the dynamic diagnosis and treatment pathway for myasthenia gravis based on multimodal fusion as described in claim 1, characterized in that, The step of performing feature extraction and feature enhancement processing on the multimodal data to obtain multimodal features includes: A pre-trained bidirectional encoder representation model is used to extract features from text modal data to obtain initial text features; A residual network fusion feature pyramid network architecture is used to extract multi-scale features from image modal data to obtain initial image features; A hybrid architecture combining a one-dimensional convolutional neural network and a long short-term memory network is used to dynamically extract features from physiological signal modal data to obtain initial physiological signal features. Temporal convolutional networks are used to extract temporal features from structured modal data to obtain initial structural features; Feature enhancement processing is performed on the initial text features, the initial image features, the initial physiological signal features, and the initial structural features respectively to obtain multimodal features.

3. The method for optimizing the dynamic diagnosis and treatment pathway for myasthenia gravis based on multimodal fusion as described in claim 2, characterized in that, The step involves performing feature enhancement processing on the initial text features, the initial image features, the initial physiological signal features, and the initial structural features to obtain multimodal features, including: The extracted clinical entities are mapped to entity embedding vectors, and the entity embedding vectors are used to enhance the initial text features to obtain text modality features, as shown in the following formula: in, Representing text modal features, Represents entity weight, The entity embedding vector representing the clinical entity. Indicates initial text features; The region of interest in the lesion image is pooled to extract lesion features, and the initial image features are then enhanced based on these lesion features to obtain image modal features, as shown in the following formula: in, Represents image modal features, Indicates the weight of the lesion. This represents the lesion features extracted by region-of-interest pooling in lesion images. Indicates initial image features; The RNS decay rate feature is extracted by the convolution kernel of the temporal convolutional network, and the initial physiological signal features are then enhanced based on the RNS decay rate feature to obtain the temporal features of the physiological signal, as shown in the following formula: in, This represents the RNS decay rate feature extracted by the convolution kernel. Indicates the decay rate weight. Indicates the temporal characteristics of physiological signals. Indicates the characteristics of the initial physiological signal; The QMG score time series of myasthenia gravis patients is input into a pre-trained long short-term memory network for time series analysis to obtain trend features. The QMG score time series is extracted based on structured modal data. Based on the trend characteristics, feature enhancement processing is performed on the initial structural features to obtain structural temporal features, as shown in the following formula: in, Represents the temporal characteristics of the structure. Indicates the QMG score weight. This indicates the trend characteristics of the QMG score over time. This represents the initial structural features.

4. The method for optimizing the dynamic diagnosis and treatment pathway for myasthenia gravis based on multimodal fusion as described in claim 3, characterized in that, The process of modeling a patient state sequence based on the patient state feature matrix and extracting patient treatment trajectory features based on the patient treatment trajectory includes: Based on the patient state feature matrix and intervention actions, a patient state sequence is modeled, referring to the following formula: in, Represents a sequence of patient states. This represents the patient's state at time n. Indicates the first Intervention actions at any time, and This represents the patient's state at time t and t+1. Represents the state transition function. Let represent the random perturbation at time t. Indicates the first Intervention actions at specific moments; The patient state sequence is input into the Transformer encoder to extract the patient's treatment trajectory features, as shown in the following formula: in, This indicates the characteristics of the patient's medical treatment trajectory. This indicates feature splicing.

5. The method for optimizing dynamic diagnosis and treatment pathways for myasthenia gravis based on multimodal fusion as described in any one of claims 1 to 4, characterized in that, The diagnostic pathway optimization model includes an input layer, a feature extraction layer, a fusion layer, and a multi-task output layer; The input layer is configured to receive multimodal data structured as time series, including text modal data, image modal data, physiological signal modal data, and structured modal data. The feature extraction layer is configured to perform feature extraction and feature enhancement processing on the multimodal data to obtain multimodal features. The multimodal features include text modality features, image modality features, physiological signal temporal features, and structural temporal features. The feature extraction layer includes a pre-trained bidirectional encoder representation module, a residual network fusion feature pyramid network architecture module, a one-dimensional convolutional neural network fusion long short-term memory network hybrid architecture module, and a temporal convolutional network module. The fusion layer is configured to construct a three-dimensional dynamic feature matrix based on the multimodal features and perform cross-modal feature fusion to obtain the target fused features; The multi-task output layer is configured for adaptability score regression analysis, referral necessity analysis, and treatment effect prediction.

6. The method for optimizing dynamic diagnosis and treatment pathways for myasthenia gravis based on multimodal fusion as described in claim 5, characterized in that, The total loss function of the multi-task output layer is: in, Represents the total loss item. This represents the regression loss term for the fit score. This indicates the loss item categorized by the necessity of referral. This represents the loss term in the prediction of treatment effectiveness. and These represent the weights of the loss terms; The loss function for the fitness score regression loss term is: in, Indicates the number of samples. This represents the regression loss weight for the i-th sample. This represents the fitness score predicted by the model for the i-th sample. The expert annotation fit score of the i-th sample; The loss function for the referral necessity classification loss term is: in, Indicates the weight of positive examples. This represents a hard-case mining factor, used to enhance attention to hard-to-classify samples. This represents the referral necessity label for the i-th sample. This represents the referral probability of the i-th sample output by the model; The loss function for the treatment effect prediction loss term is: in, This represents the change in the QMG score predicted by the model for the i-th sample. This represents the actual change in QMG score for the i-th sample. Indicates the baseline QMG score, The QMG score represents the QMG score after treatment. The QMG score is a quantitative myasthenia gravis score.

7. The method for optimizing the dynamic diagnosis and treatment pathway for myasthenia gravis based on multimodal fusion as described in claim 6, characterized in that, The dynamic optimization of the treatment pathway for the myasthenia gravis patient based on the diagnostic and treatment assessment results includes: The suitability score is compared with the first threshold and the second threshold, and the referral determination probability is compared with the third threshold to obtain the comparison result. The first threshold is greater than the second threshold. If the suitability score is not lower than the first threshold and the referral determination probability is lower than the third threshold, the current institution is determined to be suitable for myasthenia gravis patients, and the current treatment path is maintained. If the suitability score is lower than the first threshold and not lower than the second threshold, the diagnosis and treatment nodes in the diagnosis and treatment path of the myasthenia gravis patient user will be optimized. If the fit score is lower than the second threshold, or if the fit score decreases by more than a preset decrease threshold multiple times, a referral warning is triggered. An optimal list of referral target institutions is generated based on the medical institution capability feature vector, and the treatment path of the myasthenia gravis patient is adjusted based on the list of target institutions. Repeat the steps of collecting multimodal data of myasthenia gravis patients at preset time intervals and inputting the multimodal data into the treatment pathway optimization model to achieve dynamic optimization of the treatment pathway.

8. A dynamic diagnosis and treatment pathway optimization system for myasthenia gravis based on multimodal fusion, characterized in that, The system is configured to implement the dynamic diagnosis and treatment pathway optimization method for myasthenia gravis based on multimodal fusion as described in any one of claims 1 to 7, the system comprising: The data acquisition module is used to collect multimodal data from patients with myasthenia gravis. The multimodal data includes text modal data, image modal data, physiological signal modal data, and structured modal data. The feature extraction module is used to extract features and enhance features from the multimodal data to obtain multimodal features, including text modal features, image modal features, physiological signal time-series features, and structural time-series features. The feature fusion module is used to construct a three-dimensional dynamic feature matrix based on the multimodal features and perform cross-modal feature fusion to obtain the target fused features; The model training module is used to train a multi-task deep network model based on the target fusion features to obtain a treatment path optimization model. The treatment path optimization model is configured to perform adaptive score regression analysis, referral necessity analysis, and treatment effect prediction. The dynamic optimization module for treatment pathways is used to collect multimodal data of myasthenia gravis patients based on preset time intervals, input the multimodal data into the treatment pathway optimization model, output the treatment assessment results of the myasthenia gravis patients, and dynamically optimize the treatment pathway of the myasthenia gravis patients based on the treatment assessment results. The treatment assessment results include suitability score, referral determination probability, and treatment effect prediction results.

9. The dynamic diagnosis and treatment path optimization system for myasthenia gravis based on multimodal fusion as described in claim 8, characterized in that, The feature extraction module is further configured to: extract features from text modal data using a pre-trained bidirectional encoder representation model to obtain initial text features; extract multi-scale features from image modal data using a residual network fused with a feature pyramid network architecture to obtain initial image features; extract dynamic features from physiological signal modal data using a one-dimensional convolutional neural network fused with a long short-term memory network hybrid architecture to obtain initial physiological signal features; and extract temporal features from structured modal data using a temporal convolutional network to obtain initial structural features. Feature enhancement processing is performed on the initial text features, the initial image features, the initial physiological signal features, and the initial structural features respectively to obtain multimodal features.

Images