Asthma condition grading method based on multi-modal data

Abstract

The invention relates to the technical field of intelligent monitoring. The asthma condition grading method based on the multi-modal data comprises the following steps: performing multi-modal space-time alignment processing on clinical data of a patient, monitoring data of wearable equipment and environmental exposure data to generate a standardized feature matrix; performing grading processing on the multi-modal data acquired in real time by utilizing the trained updating model, and outputting an asthma condition grade result; wherein the updating model is obtained by performing dynamic fine tuning on the initial illness state grading model through a federal learning framework; the initial illness condition grading model is constructed by optimizing hyper-parameter combination of a back-propagation neural network based on a grid search algorithm, so that the problem of data distribution offset caused by airway reconstruction of the patient is solved, and the multi-modal feature fusion precision and the severe case recognition sensitivity are improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent monitoring technology, and in particular to an asthma condition grading method based on multimodal data. Background Art

[0002] With the continuous advancement of medical technology, the accuracy and adaptability of asthma classification have received increasing attention. The purpose of asthma classification is to accurately assess the severity of the patient's condition and provide a scientific basis for clinical treatment.

[0003] Traditional techniques address data distribution shifts caused by patient airway remodeling by periodically retraining the global model. However, this approach requires re-collecting large amounts of annotated data, which is not only costly but also suffers from significant update delays. In contrast, while online learning techniques support dynamic updates, they are highly sensitive to noise in medical data and can lead to excessive model error fluctuations. Summary of the Invention

[0004] Based on the above technical problems, a method for grading asthma based on multimodal data is provided to solve the problem of data distribution deviation caused by patient airway reconstruction, and to improve the accuracy of multimodal feature fusion and the sensitivity of severe case identification.

[0005] The present application provides a method for grading asthma conditions based on multimodal data, the method comprising:

[0006] Perform multimodal spatiotemporal alignment of patient clinical data, wearable device monitoring data, and environmental exposure data to generate a standardized feature matrix;

[0007] Based on the standardized feature matrix, the trained update model is used to perform hierarchical processing on the multimodal data collected in real time and output the asthma grade result;

[0008] Among them, the updated model is obtained by dynamically fine-tuning the initial disease grading model through the federated learning framework; the initial disease grading model is constructed by optimizing the hyperparameter combination of the back-propagation neural network based on the grid search algorithm.

[0009] Furthermore, the updated model is obtained by:

[0010] Distribute the initial disease classification model to multiple terminal devices and generate local parameter updates based on the local data of each terminal;

[0011] Use the following formula to perform encrypted aggregation processing on the local parameter updates to generate the updated model:

[0012]

[0013] Among them, G represents the global aggregation result, n represents the total number of participants, Dec(c i ) represents the decrypted local parameter, λ represents the attenuation coefficient, d i represents the trust score, U represents the parameters of the updated model, γ represents the learning rate, t represents the number of iterations, ω k Represents the weight parameter, Mask(v k ) represents the mask vector, and ⊙ represents element-wise multiplication.

[0014] Furthermore, performing encryption aggregation processing on the local parameter update amount also includes:

[0015] Encrypted aggregation processing uses homomorphic encryption technology to perform arithmetic operations on local parameter updates to generate an updated model.

[0016] Furthermore, the trained update model is used to perform hierarchical processing on the multimodal data collected in real time, and output asthma grade results, including:

[0017] Compare the asthma severity rating results with the clinician's revised labels to generate a feedback signal;

[0018] Iteratively optimize and update the decision boundary of the model based on the feedback signal;

[0019] The following formula is used to perform hierarchical processing on the multimodal data collected in real time based on the optimized updated model, and the updated asthma grade result is output:

[0020]

[0021] Among them, L(t) represents the disease level update function, T represents the historical data window size, μ n Indicates the confidence at the nth time point, S n represents the symptom score at the nth time point, S0 represents the baseline symptom score, and σ represents the standard deviation parameter.

[0022] Furthermore, the initial disease classification model is obtained by the following method:

[0023] Define the candidate range of hyperparameters for the backpropagation neural network, including the number of hidden layer nodes, activation function type, and regularization coefficient;

[0024] Evaluate the hierarchical performance of different hyperparameter combinations through cross-validation, and select the optimal hyperparameter combination based on the accuracy of the validation set;

[0025] The initial disease classification model is trained based on the optimal hyperparameter combination.

[0026] Furthermore, multimodal spatiotemporal alignment of the patient's clinical data, wearable device monitoring data, and environmental exposure data is performed, including:

[0027] The second-level respiratory rate in the wearable device monitoring data and the monthly-level lung function indicators in the clinical data are time-scale aligned to generate a synchronized spatiotemporal feature matrix.

[0028] Furthermore, the initial disease classification model is distributed to multiple terminal devices, and local parameter updates are generated based on local data of each terminal, further comprising:

[0029] Monitor the feature distribution offset between local data and historical data, and trigger dynamic fine-tuning when the offset exceeds the preset threshold.

[0030] Furthermore, the asthma severity level results are compared with the clinician's revised labels to generate feedback signals, including:

[0031] The classification error value is calculated based on the difference measure between the asthma grade result and the clinician's revised label;

[0032] A feedback signal is generated based on the classification error value, and the feedback signal is used to adjust the output layer weights of the updated model.

[0033] Furthermore, the decision boundary of the model is iteratively optimized and updated based on the feedback signal, including:

[0034] Calculate the gradient direction based on the feedback signal;

[0035] The classification hyperplane parameters of the updated model are adjusted along the gradient direction to generate an optimized decision boundary.

[0036] Furthermore, the initial disease classification model is trained based on the optimal hyperparameter combination, including:

[0037] Initialize the weight matrix of the back propagation neural network based on the number of hidden layer nodes to generate an initialized weight matrix;

[0038] The input data is processed by feature mapping through the forward propagation algorithm to obtain the output layer prediction value;

[0039] Perform nonlinear transformation on the output layer prediction value based on the activation function type to generate a nonlinear prediction result;

[0040] The error between the nonlinear prediction result and the true label is calculated through the loss function to obtain the classification error value;

[0041] Based on the regularization coefficient, the classification error value is subjected to overfitting suppression processing to generate a regularized error value;

[0042] The regularized error value is gradient-backed through the back-propagation algorithm to update the initialization weight matrix and generate a trained initialization disease grading model.

[0043] The technical solution provided in this application includes the following technical effects: by providing an asthma disease classification method based on multimodal data, including: multimodal spatiotemporal alignment processing of the patient's clinical data, wearable device monitoring data and environmental exposure data to generate a standardized feature matrix; based on the standardized feature matrix, using a trained update model to classify the multimodal data collected in real time, and output the asthma disease grade result; wherein, the update model is an initial disease classification model obtained by dynamic fine-tuning through a federated learning framework; the initial disease classification model is constructed based on a hyperparameter combination of a back-propagation neural network optimized by a grid search algorithm to solve the data distribution offset problem caused by the patient's airway reconstruction, and improve the accuracy of multimodal feature fusion and the sensitivity of severe case identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 This is a flowchart of an asthma condition classification method based on multimodal data in one embodiment of the present invention;

[0046] Figure 2 This is a flowchart of a method for grading asthma conditions based on multimodal data in another embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0048] like Figure 1 As shown, the present application provides an asthma condition grading method based on multimodal data, comprising:

[0049] S101: Perform multimodal spatiotemporal alignment processing on the patient's clinical data, wearable device monitoring data, and environmental exposure data to generate a standardized feature matrix.

[0050] Specifically, clinical data: Collect patients' electronic health records, including medical history, symptoms, and lung function indicators, and perform data cleaning and standardization. Wearable device monitoring data: Collect respiratory signals, heart rate, activity level, and other data monitored by wearable devices (such as smartwatches and chest straps), perform noise filtering, and perform preliminary feature extraction. Environmental exposure data: Collect data on air quality, allergen concentrations, temperature, and humidity in the patient's living environment, and perform data integration and preprocessing. Data from different sources are matched according to timestamps to ensure that each data item has corresponding time information. Align data of different frequencies using a unified time scale (such as seconds, minutes, or hours), addressing inconsistent time intervals through interpolation or sampling. Key features are extracted from the aligned data, such as lung function indicator trends in clinical data, respiratory rate and depth in wearable device data, and pollutant concentration peaks in environmental data. The extracted features are fused using a multi-scale adaptive fusion mechanism to dynamically match the temporal resolution of different modalities, improving data alignment accuracy and feature validity. The fused features are normalized to ensure that data from different modalities are within the same numerical range, facilitating subsequent model processing. Arrange the normalized features in chronological order to generate a standardized feature matrix as model input.

[0051] Through the above steps, multimodal spatiotemporal alignment of patients' clinical data, wearable device monitoring data, and environmental exposure data can be achieved to generate a standardized feature matrix, providing a high-quality data basis for asthma disease classification.

[0052] S102: Based on the standardized feature matrix, the multimodal data collected in real time is graded using the trained update model to output the asthma grade result;

[0053] Among them, the updated model is obtained by dynamically fine-tuning the initial disease grading model through the federated learning framework; the initial disease grading model is constructed by optimizing the hyperparameter combination of the back-propagation neural network based on the grid search algorithm.

[0054] Specifically, a grid search algorithm was used to optimize the hyperparameter combinations of a back-propagation neural network (BPNN) to construct an initial disease grading model. Grid search exhaustively enumerated different hyperparameter combinations (such as the number of hidden layer nodes and the regularization coefficient) and selected the optimal combination to improve model performance. A large amount of multimodal data, including patient clinical data, wearable device monitoring data, and environmental exposure data, was collected to train the initial model. During training, this data was used to adjust the model's weights and biases, enabling the model to learn the associations and characteristics between data modalities. To address data distribution shifts caused by patient airway remodeling, a federated learning framework was employed to dynamically fine-tune the initial model. In federated learning, multiple medical institutions or data holders collaborate to train a model without sharing the original data. This approach fully leverages data from different institutions, improving the model's generalization and adaptability. Each participant trains the model using its own local data and periodically sends model updates to a central server. The central server aggregates these updates to generate a new global model, which is then distributed to all participants for the next round of training. In this way, the model can continuously adapt to new data distribution and improve the accuracy of asthma classification.

[0055] In actual applications, multimodal data of patients is collected in real time, including clinical data (such as symptoms, lung function indicators, etc.), wearable device monitoring data (such as respiratory signals, heart rate, etc.), and environmental exposure data (such as air quality, temperature and humidity, etc.). The collected real-time data is preprocessed, including data cleaning, noise filtering, feature extraction and other operations to ensure data quality and consistency. The preprocessed multimodal data is input into the trained update model. The model classifies the patient's asthma condition based on the learned features and associations, and outputs the condition level result. This result can provide a reference for clinicians to help them develop more accurate treatment plans.

[0056] In one embodiment, asthma disease classification is based on a standardized feature matrix after fusion of multimodal data. The grade results are output by updating the model. The specific classification standards follow the quantitative indicators of clinical guidelines, including lung function parameters, symptom frequency, drug dependence and acute exacerbation risk. The updated model divides the grades after weighted calculation of multi-dimensional features, and dynamically modifies the classification thresholds in combination with real-time environmental exposure data to ensure dual adaptation of medical standards and personalized data-driven.

[0057] An embodiment of the present application provides an asthma disease grading method based on multimodal data, which generates a standardized feature matrix by performing multimodal spatiotemporal alignment processing on the patient's clinical data, wearable device monitoring data and environmental exposure data; uses a trained update model to grade the multimodal data collected in real time and outputs the asthma disease grade result; wherein, the update model is an initial disease grading model obtained by dynamic fine-tuning through a federated learning framework; the initial disease grading model is constructed by optimizing the hyperparameter combination of the back-propagation neural network based on a grid search algorithm to solve the data distribution offset problem caused by the patient's airway reconstruction, and improve the accuracy of multimodal feature fusion and the sensitivity of severe case identification.

[0058] Furthermore, the updated model is obtained by:

[0059] Distribute the initial disease classification model to multiple terminal devices and generate local parameter updates based on the local data of each terminal;

[0060] Use the following formula to perform encrypted aggregation processing on the local parameter updates to generate the updated model:

[0061]

[0062] Among them, G represents the global aggregation result, n represents the total number of participants, Dec(c i ) represents the decrypted local parameter, λ represents the attenuation coefficient, d i represents the trust score, U represents the parameters of the updated model, γ represents the learning rate, t represents the number of iterations, ω k Represents the weight parameter, Mask(v k ) represents the mask vector, and ⊙ represents element-wise multiplication.

[0063] Specifically, the initial disease classification model is distributed to multiple devices. These devices can be found in different medical institutions, research institutes, or individuals, each with corresponding local data. Each device trains the initial model based on its local data, generating local parameter updates. In this step, each device independently trains the model, without data sharing, thus protecting data privacy.

[0064] To protect data privacy and improve model robustness, local parameter updates are encrypted and aggregated. First, a global aggregation of all participants' local parameter updates is calculated. During the aggregation process, decrypted local parameters are used to ensure the authenticity and validity of parameter updates. A decay coefficient is introduced to adjust the impact of historical parameter updates on the current model, allowing the model to better adapt to the new data distribution. Each participant is assigned a different trust score based on their historical performance and data quality, with participants with higher weights having greater influence on global model updates. The updated model parameters obtained through this calculation are then used to generate a new global model.

[0065] The learning rate determines the step size of parameter updates, affecting the convergence speed and stability of the model. The number of iterations represents the number of cycles in the federated learning process. As the iterations progress, the model is gradually optimized. Weight parameters are used to balance the contributions of different participants, ensuring fair and effective global model updates. Mask vectors are used to selectively update certain parts of the model, improving the efficiency and targeting of updates. During the parameter update process, element-by-element multiplication operations are used to apply different parameter adjustments to the model.

[0066] Through the above steps, this application achieves dynamic fine-tuning of the initial disease classification model, generating an updated model that can adapt to the new data distribution. This process not only protects data privacy but also improves the accuracy and robustness of the model, providing a more reliable basis for asthma disease classification.

[0067] Furthermore, performing encryption aggregation processing on the local parameter update amount also includes:

[0068] Encrypted aggregation processing uses homomorphic encryption technology to perform arithmetic operations on local parameter updates to generate an updated model.

[0069] Specifically, each terminal device encrypts its local parameter updates using a homomorphic encryption algorithm and then sends the encrypted updates to the aggregation server. Without decrypting the encrypted updates, the aggregation server performs arithmetic operations, such as summing or averaging, on the encrypted parameter updates to generate a global aggregate result. Finally, the aggregation server distributes the encrypted global aggregate result to each terminal device, which decrypts it using its private key to obtain the updated model.

[0070] Furthermore, the trained update model is used to perform hierarchical processing on the multimodal data collected in real time, and output asthma grade results, including:

[0071] Compare the asthma severity rating results with the clinician's revised labels to generate a feedback signal;

[0072] Iteratively optimize and update the decision boundary of the model based on the feedback signal;

[0073] The following formula is used to perform hierarchical processing on the multimodal data collected in real time based on the optimized updated model, and the updated asthma grade result is output:

[0074]

[0075] Among them, L(t) represents the disease level update function, T represents the historical data window size, μ n Indicates the confidence at the nth time point, S n represents the symptom score at the nth time point, S0 represents the baseline symptom score, and σ represents the standard deviation parameter.

[0076] Specifically, first, the real-time multimodal data is input into the trained update model. Based on previous learning and optimization, the model analyzes and processes the input data to obtain a preliminary asthma grade result. This preliminary result is then compared with the clinician's revised label, and the difference between the two is calculated to generate a feedback signal. This feedback signal reflects the deviation between the model output and the actual clinical diagnosis, providing an important basis for subsequent model optimization. Next, based on the generated feedback signal, the decision boundary of the updated model is iteratively optimized. By adjusting the model's parameters and weights, the model can better adapt to the new data features and clinical diagnostic criteria, thereby improving the model's accuracy and reliability. Finally, the optimized updated model is used to perform classification processing on the real-time multimodal data again, outputting an updated asthma grade result. This result is an optimized result adjusted by the feedback signal and can more accurately reflect the patient's asthma condition, providing stronger support for clinical diagnosis and treatment.

[0077] Furthermore, the initial disease classification model is obtained by the following method:

[0078] Define the candidate range of hyperparameters for the backpropagation neural network, including the number of hidden layer nodes, activation function type, and regularization coefficient;

[0079] Evaluate the hierarchical performance of different hyperparameter combinations through cross-validation, and select the optimal hyperparameter combination based on the accuracy of the validation set;

[0080] The initial disease classification model is trained based on the optimal hyperparameter combination.

[0081] Specifically, first, the candidate range of hyperparameters for the back-propagation neural network is defined. These hyperparameters include the number of hidden layer nodes, the type of activation function, and the regularization coefficient. Then, cross-validation is used to evaluate the classification performance of different hyperparameter combinations. During the cross-validation process, the dataset is divided into multiple subsets, and one part of them is used as the training set and the rest as the validation set in turn, so that the model is trained and evaluated multiple times to obtain more stable performance evaluation results. According to the results of cross-validation, the optimal hyperparameter combination is screened out based on the accuracy of the validation set. The hyperparameter combination that shows the highest accuracy on the validation set is selected because it can ensure that the model has a good classification effect while ensuring the appropriate complexity of the model, avoiding overfitting or underfitting problems. Finally, the optimal hyperparameter combination screened out is used to train the initial disease classification model, so that the model can accurately classify asthma conditions, laying the foundation for subsequent dynamic fine-tuning and classification processing.

[0082] Furthermore, multimodal spatiotemporal alignment of the patient's clinical data, wearable device monitoring data, and environmental exposure data is performed, including:

[0083] The second-level respiratory rate in the wearable device monitoring data and the monthly-level lung function indicators in the clinical data are time-scale aligned to generate a synchronized spatiotemporal feature matrix.

[0084] Specifically, multimodal spatiotemporal alignment processing is performed on the patient's clinical data, wearable device monitoring data and environmental exposure data. Specifically, the second-level respiratory rate in the wearable device monitoring data and the monthly lung function indicators in the clinical data are aligned in time scale. Through interpolation or sampling methods, data of different time scales are matched on a unified time scale, thereby generating a synchronized spatiotemporal feature matrix, providing a data basis for subsequent disease grading model training and grading processing.

[0085] Furthermore, the initial disease classification model is distributed to multiple terminal devices, and local parameter updates are generated based on local data of each terminal, further comprising:

[0086] Monitor the feature distribution offset between local data and historical data, and trigger dynamic fine-tuning when the offset exceeds the preset threshold.

[0087] Specifically, the initial disease grading model is first distributed to multiple devices, each with its own local data. Local parameter updates are then generated based on each device's local data, while the deviation in feature distribution between local data and historical data is monitored. When this deviation exceeds a preset threshold, dynamic fine-tuning is triggered. This process ensures that the model can promptly adapt to changes in data distribution, improving its accuracy and adaptability.

[0088] Furthermore, the asthma severity level results are compared with the clinician's revised labels to generate feedback signals, including:

[0089] The classification error value is calculated based on the difference measure between the asthma grade result and the clinician's revised label;

[0090] A feedback signal is generated based on the classification error value, and the feedback signal is used to adjust the output layer weights of the updated model.

[0091] Specifically, the classification error between the model output and the physician's corrected label is first calculated. This step quantifies the model's predictive accuracy by comparing the difference between the two. This classification error is then used to generate a feedback signal, which is used to adjust the model's output layer weights to reduce error in future predictions. This approach allows the model to continuously learn and adapt to new data and clinical diagnostic criteria, thereby improving grading accuracy.

[0092] Furthermore, the decision boundary of the model is iteratively optimized and updated based on the feedback signal, including:

[0093] Calculate the gradient direction based on the feedback signal;

[0094] The classification hyperplane parameters of the updated model are adjusted along the gradient direction to generate an optimized decision boundary.

[0095] Specifically, the gradient direction is calculated based on the feedback signal, indicating the direction in which the model parameters need to be adjusted to reduce classification error. The model's classification hyperplane parameters are adjusted and updated along this gradient direction. This involves updating the model's weight parameters through an optimization algorithm (such as gradient descent) to optimize the position and direction of the classification hyperplane, thereby generating a more accurate decision boundary. This process is repeated over and over again, with each iteration utilizing new feedback signals to further adjust the model parameters, gradually bringing the model's decision boundary closer to the true label distribution, ultimately improving the model's classification accuracy and reliability.

[0096] Further, if Figure 2 As shown, the initial disease classification model is trained based on the optimal hyperparameter combination, including:

[0097] S201: Initializing the weight matrix of the back propagation neural network based on the number of hidden layer nodes to generate an initialized weight matrix;

[0098] S202: Perform feature mapping on the input data using a forward propagation algorithm to obtain an output layer prediction value;

[0099] S203: Performing nonlinear conversion processing on the output layer prediction value based on the activation function type to generate a nonlinear prediction result;

[0100] S204: performing error calculation between the nonlinear prediction result and the true label using a loss function to obtain a classification error value;

[0101] S205: performing overfitting suppression processing on the classification error value based on the regularization coefficient to generate a regularized error value;

[0102] S206: Performing gradient backpropagation processing on the regularized error value through the backpropagation algorithm, updating the initialized weight matrix, and generating a trained initialized disease grading model.

[0103] Specifically, first, the weight matrix of the back-propagation neural network is initialized based on the number of hidden layer nodes to generate an initialized weight matrix. Then, the input data is feature mapped using the forward propagation algorithm to obtain the output layer prediction value. Next, the output layer prediction value is nonlinearly transformed based on the activation function type to generate a nonlinear prediction result. The error between the nonlinear prediction result and the true label is calculated using the loss function to obtain the classification error value. The classification error value is overfitting suppressed based on the regularization coefficient to generate a regularized error value. Finally, the regularized error value is gradient-backed using the back-propagation algorithm to update the initialized weight matrix and generate a trained initial disease grading model. This process ensures that the model can learn effective feature representations from the data, and prevents overfitting through regularization, thereby improving the model's generalization ability and classification accuracy.

[0104] In one embodiment, a Matlab program is used to read a data set and normalize the feature data, dividing the data into a training set and a test set; a grid search algorithm is used to determine the optimal number of hidden layer neurons, different hidden layer sizes are trained, and their accuracy is evaluated on a validation set, the best performing hidden layer size is selected, and the neural network model is retrained using the optimal hidden layer size; a BP neural network is trained, wherein the BP neural network includes an input layer, multiple hidden layer units, and an output layer connected in sequence; the trained model is used on a test set to predict and grade the asthma conditions to be classified, and indicators such as precision, recall rate, F1 score, and Kappa coefficient are calculated to evaluate the model performance, and a confusion matrix diagram is drawn to intuitively display the classification effect of the model.

[0105] Among them, the grid search algorithm is used to select the optimal number of hidden layer neurons, including the following steps: define the hyperparameter search space: clarify the model parameters that need to be tuned (the hidden layer size in the neural network) and their value range. Create an array hiddenLayerSizes containing the candidate number of hidden layer neurons. Define the search space to include five values ​​[10, 20, 30, 40, 50]; initialize the best parameter: initialize bestAccuracy to 0 to store the highest validation set accuracy found so far. Initialize bestSize to the first element of the hiddenLayerSizes array as a placeholder for the current optimal number of hidden layer neurons; hyperparameter search: divide the data set into a training set and a validation set. Loop through the search space and perform the following steps for each possibility: (1) Create a network: For each candidate hidden layer size, create a new BP neural network and set the hidden layer size to the current candidate value, and set the training function of the neural network; (2) Set training parameters: Set the training parameters of the network; (3) Train the network: Use the train function to train the network; (4) Output results: Update the optimal parameter values ​​and obtain the output results.

[0106] In one embodiment, patient data, primarily including physiological indicators, was collected through clinical trials and observational studies. Large biological databases such as the UK Biobank were also accessed to collect genetic information, health records, and clinical data from a large number of individuals. Ultimately, a dataset was created for the experiment. The feature data was normalized in the program, and the data was divided into a training set and a test set. Because adult and pediatric grading standards differ, the training set was divided into an adult set and a pediatric set, each containing data (expiratory time, insufflation flow rate, NO concentration, and CO2 concentration) from 3,000 individuals (1,000 normal, 1,000 borderline, and 1,000 abnormal) respectively. The training set was then consolidated into a CSV file, which was then read using the Matlab program.

[0107] In one embodiment, the RELU activation function is used to propagate the output nodes of the hidden layer nodes to the output nodes, and then the classification results are output through the Softmax classifier, and finally the weights and thresholds between the neurons in each layer are modified through the back propagation algorithm (BP algorithm).

[0108] In one embodiment, the input and output of the hidden layer and the output layer are calculated, including: assuming that the activation function is σ(z), the L1 layer is the input layer, the L2 layer is the hidden layer, and the L3 layer is the output layer, and the output y1, y2, y3, ..., y7 of each node in the hidden layer are respectively expressed as:

[0109] y1=σ(w 11 x1+w 12 x2+w 13x3+...+b1)

[0110] y2=σ(w 21 x1+w 22 x2+w 23 x3+...+b1)

[0111] y3=σ(w 31 x1+w 32 x2+w 33 x3+...+b1)

[0112] y4=σ(w 41 x1+w 42 x2+w 43 x3+...+b1)

[0113] y5=σ(w 51 x1+w 52 x2+w 53 x3+...+b1)

[0114] y6=σ(w 61 x1+w 62 x2+w 63 x3+...+b1)

[0115] y7=σ(w 71 x1+w 72 x2+w 73 x3+...+b1)

[0116] For the output layer L3, its output y is expressed as:

[0117] y=σ(w 11 y1+w 12 y2+w 13 y3+...+b2)

[0118] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0119] The above-described embodiments merely represent several implementation methods of the embodiments of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the concept of the embodiments of the present application, and these modifications and improvements fall within the scope of protection of the embodiments of the present application.

Claims

1. A method for grading asthma based on multimodal data, characterized in that: The method comprises: Perform multimodal spatiotemporal alignment of patient clinical data, wearable device monitoring data, and environmental exposure data to generate a standardized feature matrix; Based on the standardized feature matrix, the multimodal data collected in real time is graded using the trained update model to output an asthma condition grade result; Among them, the updated model is obtained by dynamically fine-tuning the initial disease grading model through a federated learning framework; the initial disease grading model is constructed by optimizing the hyperparameter combination of the back propagation neural network based on the grid search algorithm.

2. The asthma condition grading method based on multimodal data according to claim 1, characterized in that: The updated model is obtained by the following method: Distributing the initial disease grading model to multiple terminal devices, and generating local parameter updates based on local data of each terminal; The following formula is used to perform encryption aggregation processing on the local parameter update amount to generate the update model: Among them, G represents the global aggregation result, n represents the total number of participants, Dec(c i ) represents the decrypted local parameter, λ represents the attenuation coefficient, d i represents the trust score, U represents the parameters of the updated model, γ represents the learning rate, t represents the number of iterations, ω k Represents the weight parameter, Mask(v k ) represents the mask vector, and ⊙ represents element-wise multiplication.

3. The asthma condition grading method based on multimodal data according to claim 2, characterized in that: The performing encryption aggregation processing on the local parameter update amount further includes: The encryption aggregation process uses homomorphic encryption technology to generate the updated model after performing arithmetic operations on the local parameter update amount.

4. The asthma condition grading method based on multimodal data according to claim 1, characterized in that: Based on the standardized feature matrix, the multimodal data collected in real time is graded using the trained update model to output the asthma condition grade result, including: Comparing the asthma condition grade result with the clinician's revised label to generate a feedback signal; iteratively optimizing the decision boundary of the updated model based on the feedback signal; The following formula is used to perform graded processing on the multimodal data collected in real time based on the optimized updated model, and the updated asthma condition grade result is output: Among them, L(t) represents the disease level update function, T represents the historical data window size, μ n Indicates the confidence at the nth time point, S n represents the symptom score at the nth time point, S0 represents the baseline symptom score, and σ represents the standard deviation parameter.

5. The asthma condition grading method based on multimodal data according to claim 1, characterized in that: The initial disease grading model is obtained by the following method: Defining a candidate range of hyperparameters of the back propagation neural network, wherein the hyperparameters include the number of hidden layer nodes, the activation function type, and the regularization coefficient; Evaluate the classification performance of different hyperparameter combinations through cross-validation, and select the optimal hyperparameter combination based on the accuracy of the validation set; The initial disease classification model is trained based on the optimal hyperparameter combination.

6. The asthma condition grading method based on multimodal data according to claim 1, characterized in that: The multimodal spatiotemporal alignment of the patient's clinical data, wearable device monitoring data, and environmental exposure data includes: The second-level respiratory rate in the wearable device monitoring data and the monthly-level lung function index in the clinical data are aligned in time scale to generate a synchronized spatiotemporal feature matrix.

7. The asthma condition grading method based on multimodal data according to claim 2, characterized in that: The method of distributing the initial disease grading model to a plurality of terminal devices and generating a local parameter update based on local data of each terminal further includes: The characteristic distribution offset between the local data and the historical data is monitored, and the dynamic fine-tuning is triggered when the offset exceeds a preset threshold.

8. The asthma condition grading method based on multimodal data according to claim 4, characterized in that: The step of comparing the asthma condition grade result with the clinician's revised label to generate a feedback signal includes: Calculating a classification error value based on a difference measure between the asthma condition grade result and the clinician-corrected label; The feedback signal is generated according to the classification error value, and the feedback signal is used to adjust the output layer weight of the updated model.

9. The asthma condition grading method based on multimodal data according to claim 8, characterized in that: The iteratively optimizing the decision boundary of the updated model based on the feedback signal includes: Calculating a gradient direction based on the feedback signal; The classification hyperplane parameters of the updated model are adjusted along the gradient direction to generate the optimized decision boundary.

10. The asthma condition grading method based on multimodal data according to claim 5, characterized in that: The training of the initial disease classification model based on the optimal hyperparameter combination includes: Initializing the weight matrix of the back propagation neural network based on the number of hidden layer nodes to generate an initialized weight matrix; The input data is processed by feature mapping through the forward propagation algorithm to obtain the output layer prediction value; Performing nonlinear conversion processing on the output layer prediction value based on the activation function type to generate a nonlinear prediction result; Perform error calculation on the nonlinear prediction result and the true label using a loss function to obtain a classification error value; performing overfitting suppression processing on the classification error value based on the regularization coefficient to generate a regularized error value; The regularized error value is subjected to gradient backpropagation processing through a backpropagation algorithm, the initialized weight matrix is ​​updated, and the trained initialized disease grading model is generated.

Images