A method for interpretable anomaly tracing of multimodal medical data in smart hospitals during cloud-edge collaborative transmission.

Abstract

This invention discloses an interpretable anomaly tracing method for multimodal medical data in smart hospitals during cloud-edge collaborative transmission. Applied to a collaborative system including an edge layer and a cloud layer, the method includes: real-time acquisition, preprocessing, and autoencoder-based initial anomaly screening of multimodal medical data at the edge layer, and encapsulating structured tracing identifiers for data packets; integrity verification, Transformer-based multimodal deep fusion, and Graph Attention Network (GAT)-based collaborative anomaly fine-tuning at the cloud layer; when an anomaly is detected, the SHAP algorithm is used to analyze the judgment criteria, calculate the contribution of each input feature, and generate a structured tracing report; the tracing results are constructed into a knowledge graph, and the confirmed anomaly data is used to perform closed-loop optimization of the models at the cloud and edge layers using knowledge distillation technology; this invention significantly improves the accuracy of anomaly detection.

Description

Technical Field

[0001] This invention relates to the fields of smart healthcare and medical big data technology, specifically to a method for tracing the source of interpretable anomalies in the cloud-edge collaborative transmission of multimodal medical data in smart hospitals. Background Technology

[0002] To achieve the integration and efficient utilization of regional medical resources, the cloud-edge collaborative architecture is widely used in the informatization construction of smart hospitals: the edge (internal hospital servers or smart gateways) is responsible for the initial processing and real-time response of data, while the cloud (regional or national medical data center) undertakes the tasks of large-scale data storage, complex calculations and model training.

[0003] In this complex distributed architecture, multimodal medical data faces severe data quality and security challenges throughout its entire lifecycle, from generation and processing to transmission. The following problems exist: First, an overemphasis on content anomalies while neglecting the transmission process. Most anomaly detection methods primarily focus on content anomalies related to business logic, such as hospitalization costs, while paying insufficient attention to dynamic anomalies generated during the cloud-edge collaborative transmission link (such as transmission delays, data packet corruption, and timestamp discrepancies between modalities). Second, a lack of interpretability and tracing capabilities. Existing anomaly detection models (especially deep learning models) are mostly "black box" models; even if anomalies are detected, the basis for the judgment cannot be clearly explained, making it difficult to pinpoint the root cause. This makes responsibility determination difficult and fails to provide effective guidance for system repair. Third, a focus on single-modal detection, lacking a multimodal fusion perspective. Many methods only detect single types of data and cannot detect logical conflicts between cross-modal data. Fourth, insufficient interdisciplinary collaboration. In the highly complex scenario of multimodal medical data, which demands extremely high security and reliability, there is still no mature and complete solution. Summary of the Invention

[0004] Purpose of the Invention: The purpose of this invention is to provide an interpretable anomaly tracing method for multimodal medical data in smart hospitals during cloud-edge collaborative transmission. This method can effectively monitor the status of multimodal medical data in the cloud-edge collaborative transmission link of smart hospitals, not only accurately detect anomalies, but also provide transparent, reliable, and interpretable tracing analysis, thereby ensuring the quality and security of medical data throughout its entire lifecycle and solving the problems existing in the background technology.

[0005] Technical Solution: The present invention provides a method for interpretable anomaly tracing in the cloud-edge collaborative transmission of multimodal medical data in smart hospitals. This method is applied to a collaborative system including an edge layer and a cloud layer, and includes the following steps:

[0006] (1) At the edge layer, multimodal medical data from different medical devices are collected in real time, preprocessed and initially extracted, and a lightweight anomaly detection model deployed at the edge is used to perform real-time anomaly screening of single-modal data streams.

[0007] (2) Generate and encapsulate a unique, structured traceability identifier for the processed data packet through the traceability identifier encapsulation module, and transmit the encapsulated data packet to the cloud layer through a secure channel;

[0008] (3) At the cloud layer, the received data packets are verified for integrity and traceability identification. The multimodal deep fusion model based on attention mechanism is used to align and deeply fuse different modal features from the same analysis unit to generate a unified high-dimensional joint feature representation.

[0009] (4) Using a collaborative anomaly detection model based on graph neural networks, the high-dimensional joint feature representation is constructed as a graph structure with data units as nodes and spatiotemporal or logical relationships as edges. Graph convolution analysis is then performed on the graph structure to detect cross-modal correlation anomalies.

[0010] (5) When the collaborative anomaly detection model determines that the data is abnormal, the interpretability tracing engine is started to analyze the decision-making process of the collaborative anomaly detection model and calculate the contribution of each original input feature to the anomaly determination result.

[0011] (6) Based on the contribution and source identification, and combined with the preset fault mode rule base, a structured source tracing report is automatically generated. The source tracing report contains key evidence features and their quantitative contribution, and infers possible abnormal root causes.

[0012] (7) Convert the structured information in the source tracing report into graph entities and relationships, and construct or update the source tracing knowledge graph to provide a visual analysis of the propagation path, scope of impact and root cause of abnormal events;

[0013] (8) In the cloud layer, the abnormal data confirmed by the source is used as training samples to perform incremental training on the collaborative anomaly detection model and the multimodal deep fusion model on a regular basis, and the optimized model parameters or the lightweight edge detection model are sent to the edge layer to realize continuous adaptive optimization of system performance.

[0014] Furthermore, in step (1), the lightweight anomaly detection model is a reconstruction error detection model based on an autoencoder. It calculates the reconstruction error between the input data and the reconstructed data, and compares the error with a dynamic threshold determined based on the reconstruction error distribution of historical normal data. When the reconstruction error exceeds the dynamic threshold, the corresponding data stream segment is marked as a suspicious anomaly.

[0015] Furthermore, in step (2), the traceability identifier includes: a globally unique identifier, a source device identifier, an edge node identifier, a data acquisition timestamp, a data modality type, an edge screening result, an edge screening confidence level, and a hash digest of the data payload.

[0016] Furthermore, in step (3), the multimodal deep fusion model is a cross-modal attention fusion network based on the Transformer architecture. It maps each modal feature to a unified dimensional space through linear projection and uses a multi-head self-attention mechanism to learn the intrinsic correlation weights between different modal features, thereby generating the high-dimensional joint feature representation.

[0017] Furthermore, in step (4), the collaborative anomaly detection model is a graph attention network model. The edges of the graph structure include temporal edges based on temporal dependencies, spatial edges based on spatial relationships, and device edges based on device relationships. The graph convolutional analysis adaptively learns the importance weights of neighboring nodes through the attention mechanism and updates the node features accordingly.

[0018] Furthermore, in step (5), the contribution is calculated based on the Shapley value theory. The marginal contribution of each feature to the model prediction result is calculated under all possible combinations of feature subsets, and a sampling-based approximation algorithm is used to reduce the computational complexity.

[0019] Furthermore, in step (6), the source knowledge graph takes abnormal events as the central node, and the entity types include abnormal events, medical devices, edge nodes, data modalities, abnormal types and root cause types. The relationship types include occurrence, origin, belonging to type, causing and affecting, and the weight of the edge is determined by the corresponding contribution value.

[0020] Furthermore, in step (7), before the optimized model parameters are sent to the edge layer, the knowledge of the complex model in the cloud is transferred to the lightweight edge detection model through knowledge distillation technology. During the distillation process, the hard label loss and soft label loss are balanced to train the lightweight student model with improved performance.

[0021] The present invention discloses an interpretability anomaly tracing system for multimodal medical data in smart hospitals during cloud-edge collaborative transmission, which is used to execute any of the methods described herein, including:

[0022] The edge layer includes: a multimodal data adaptation and acquisition module for real-time acquisition of various types of multimodal medical data; a data preprocessing and feature extraction module for data cleaning, format normalization, and preliminary feature extraction; an edge anomaly detection module deploying a lightweight autoencoder model for real-time anomaly screening of single-modal data streams; and a data traceability identification and encapsulation module for generating and encapsulating unique, structured traceability identifiers for the processed data packets.

[0023] The cloud layer includes: a data aggregation and verification module for verifying the integrity and source identification of received data packets; a multimodal data deep fusion module deploying a cross-modal attention fusion network based on the Transformer architecture to generate a unified high-dimensional joint feature representation; a collaborative anomaly detection module deploying a graph attention network model for precise collaborative anomaly judgment on the high-dimensional joint feature representation; an interpretable source tracing engine integrating an interpretation algorithm based on Shapley value theory to generate a structured source tracing report when data is determined to be anomaly; a knowledge graph construction module for building or updating the source tracing knowledge graph based on the source tracing results; and a model training and deployment module for training and optimizing models in the cloud and deploying the optimized lightweight model to the edge layer through knowledge distillation technology.

[0024] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0025] First, it improves the accuracy and breadth of anomaly detection. Through a collaborative architecture of edge detection and cloud-based precision judgment, and a deep model combination of Transformer and GAT, this method can detect various anomalies ranging from simple format errors to complex cross-modal logical conflicts. Comparative experiments on a simulated dataset built based on an extension of the MIMIC-III database show that, compared to traditional single models, the detection accuracy is improved by more than 2.1%, and the cross-modal anomaly recall rate is improved by more than 6.9%.

[0026] Second, it enables the interpretability and traceability of anomaly detection. By innovatively applying the SHAP algorithm, this invention makes the decision-making process of the black-box model transparent, providing a quantitative and understandable chain of evidence for each anomaly detection, with the average tracing time controlled in milliseconds (approximately 15.8ms / detection).

[0027] Third, it improves system efficiency and resource utilization. The lightweight initial screening at the edge filters a large amount of normal data, greatly reducing the amount of data uploaded to the cloud and saving network bandwidth (approximately 64.5%) and cloud computing resources (CPU load reduced by approximately 42.3%).

[0028] Fourth, build a continuously optimized adaptive system. Through a closed-loop feedback mechanism of cloud-edge collaboration and knowledge distillation technology, the system can continuously learn and evolve using newly discovered anomalous data, maintaining the model's robustness in dynamic medical environments. Attached Figure Description

[0029] Figure 1 This is a flowchart of the present invention;

[0030] Figure 2 This is a schematic diagram of the cloud-edge collaborative architecture of the present invention.

[0031] Figure 3 This is a schematic diagram of the structure of the edge-end lightweight autoencoder anomaly screening model of the present invention.

[0032] Figure 4 This is a schematic diagram of the data traceability identifier of the present invention.

[0033] Figure 5 This is a schematic diagram of the structure of the Transformer-based multimodal data deep fusion model of the present invention.

[0034] Figure 6 This is a schematic diagram of the graph structure construction and attention convolution of the collaborative anomaly detection model based on graph attention network (GAT) of the present invention.

[0035] Figure 7 This is a schematic diagram illustrating the working principle of the SHAP interpretability tracing engine of the present invention.

[0036] Figure 8 This is a schematic diagram of the structure of the knowledge graph for tracing the origins of this invention.

[0037] Figure 9 This is a schematic diagram of the cloud-edge collaborative closed-loop optimization process of the present invention.

[0038] Figure 10 The figure shows the experimental comparison results of this invention. Detailed Implementation

[0039] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0040] like Figure 1 As shown, this embodiment of the invention provides a method for interpretable anomaly tracing in the cloud-edge collaborative transmission of multimodal medical data in smart hospitals. Applied to a collaborative system including an edge layer and a cloud layer, the method includes the following steps:

[0041] Step 1: Edge data processing and initial anomaly screening.

[0042] S101: At the edge layer, through the multimodal data adaptation and acquisition module, various types of data from different medical devices (such as monitors, ventilators, infusion pumps, electronic medical record systems, etc.) are acquired in real time, including but not limited to physiological signal waveform data, medical image data and electronic medical record text data.

[0043] S102: The data preprocessing and feature extraction module cleans and normalizes the collected data, and extracts preliminary features for each modality. Specifically, for physiological signal waveform data (such as ECG), a bandpass filter is used for noise reduction, and time-domain and frequency-domain features such as heart rate (HR) and RR interval variability (HRV) are extracted; for text data (such as nursing records), word segmentation and stop word removal are performed, and a pre-trained medical text embedding model (such as BioBERT) is used to convert it into fixed-dimensional data. The word vector representation; for numerical data (such as blood pressure, blood oxygen), Z-Score normalization is used for normalization, and the formula is:

[0044]

[0045] Where x is the original value, mu is the mean of the feature in historical data, and sigma is the standard deviation.

[0046] S103: A lightweight autoencoder model is deployed through the edge anomaly detection module to perform real-time anomaly screening of a single-modal data stream. The autoencoder consists of an encoder E and a decoder D. For input data x, the encoder maps it to a low-dimensional latent space representation z, and the decoder then reconstructs z into x'. Reconstruction error. The calculation formula is:

[0047]

[0048] Where ||.||^2 represents the square of the L2 norm. When the reconstruction error of a data point exceeds the dynamic threshold tau, that data stream segment is marked as "Suspicious". The dynamic threshold tau is calculated based on the reconstruction error distribution of historical data, using the mean plus k times the standard deviation.

[0049]

[0050] in, and These are the mean and standard deviation of the reconstruction error of historical normal data, respectively, and k is the sensitivity coefficient (k=3 in this embodiment).

[0051] S104: The data tracing identifier and encapsulation module generates a unique, structured tracing identifier T for each processed data packet or data stream segment. This tracing identifier contains the following fields (Table 1).

[0052] Table 1

[0053] S105: Transmit the data packet encapsulating the traceability identifier T to the cloud layer via a TLS 1.3 secure encrypted channel.

[0054] Step 2: Cloud data fusion and collaborative anomaly detection.

[0055] S201: At the cloud layer, the data aggregation and verification module is responsible for receiving data packets from each edge node. It first decrypts the packets and then recalculates the SHA-256 hash value of the data payload. and in the traceability identifier Perform a comparison. If... Not equal to If the data packet is found to have been tampered with or corrupted during transmission, it will be marked as "transmission abnormal" and recorded.

[0056] S202: The multimodal data deep fusion module utilizes a cross-modal attention fusion network based on the Transformer architecture to align and deeply fuse features from different modalities within the same analysis unit (such as the same patient within the same time window). Let the feature vectors from M different modalities be... , , ..., First, they are mapped to a unified one through their respective linear projection layers. Dimensional space:

[0057]

[0058] Then, the embedding vectors of all modalities are concatenated into a sequence. And incorporate learnable modal position encoding. Then, the input is fed into the Transformer encoder. The core of the Transformer encoder is the Multi-Head Self-Attention (MHSA) mechanism, whose calculation formula is as follows:

[0059]

[0060]

[0061]

[0062] Where Q, K, and V represent the query matrix, key matrix, and value matrix, respectively, all derived from the input sequence. Obtained through linear transformation; The dimension of each attention head; h is the number of attention heads (h=8 in this embodiment); and is a learnable parameter matrix.

[0063] After passing through an L-layer Transformer encoder, the vector corresponding to the [CLS] label in the output sequence is taken as a unified high-dimensional joint feature representation. .

[0064] S203: The collaborative anomaly detection module deploys a Graph Attention Network (GAT) model. First, the data is constructed as a graph structure G = (V, E), where each node in the node set V... Fusion features corresponding to an analysis unit Edge set E is constructed according to the following rules:

[0065] (a) Temporal edge: A directed temporal edge is established between nodes in adjacent time windows of the same patient to represent temporal dependencies.

[0066] (b) Spatial edge: An undirected spatial edge is established between nodes of different patients in the same department within the same time window to represent spatial relationships.

[0067] (c) Device edge: An undirected device edge is established between nodes corresponding to data generated by the same medical device to represent the device association relationship.

[0068] GAT adaptively learns the importance weights of neighboring nodes through an attention mechanism. The attention coefficient between node i and node j... The calculation formula is:

[0069]

[0070]

[0071] in, and The feature vectors of node i and node j are respectively (initially...) W is the shared linear transformation weight matrix, a is the learnable parameter vector of the attention mechanism, and || denotes the vector concatenation operation. Let i be the set of neighboring nodes of node i.

[0072] The node feature update formula is:

[0073]

[0074] Here, sigma is the ELU (Exponential Linear Unit) non-linear activation function. To enhance the model's expressive power, a K-head attention mechanism is used, concatenating the outputs of K independent attention heads:

[0075]

[0076] After multiple GAT convolutions, the final output layer passes through a fully connected layer and a sigmoid activation function to output the anomaly probability score for each node. The value range is [0, 1]. When Exceeding the preset threshold (In this embodiment, we take) When the value is 0.5, the data corresponding to that node is considered abnormal.

[0077] Step 3: Explainability tracing and knowledge graph construction.

[0078] S301: When the collaborative anomaly detection module determines that the data is abnormal, the interpretability tracing engine is activated. This engine integrates the SHAP (SHapley Additive exPlanations) algorithm, which, based on the Shapley value theory in cooperative game theory, provides ex-post interpretation of the decision-making process of the GAT model.

[0079] S302: For a data point x* identified as an anomaly, the SHAP algorithm calculates the marginal contribution of each original input feature i to the final anomaly determination result. The SHAP value of feature i. The calculation formula is:

[0080]

[0081] Where F is the set of all input features (|F| is the total number of features), and S is any subset of features that does not contain feature i. This represents the predicted output of the model when only a subset of features S is used. The SHAP value satisfies the following three important properties:

[0082] (d) Local Accuracy: The sum of the SHAP values ​​of all features equals the difference between the model's predicted value and the baseline value, i.e. ,in This is the baseline predicted value.

[0083] (e) Consistency: If the marginal contribution of a feature does not decrease in all possible subsets of features, then its SHAP value does not decrease either.

[0084] (f) Missingness: If a feature is missing in the input (i.e. has no effect on the model's prediction), its SHAP value is zero.

[0085] In practical calculations, since the complexity of accurately calculating the SHAP value is O(2^|F|), this invention adopts a sampling-based approximation algorithm (Kernel SHAP) to approximate the SHAP value through weighted linear regression, thereby reducing the computational complexity to O(|F|^2).

[0086] S303: Based on the SHAP value sorting result and the packet source identifier T, combined with the preset fault mode rule base (containing known anomaly modes and their corresponding root cause mappings), the system automatically generates a structured source report R. This report contains the following key fields (Table 2).

[0087] Table 2

[0088] S304: Convert the structured information in the source tracing report R into graph entities and relations, and construct or update a source tracing knowledge graph. The entity types in the knowledge graph include: AnomalyEvent, Device, EdgeNode, Modality, AnomalyType, RootCause, and Operator. Relationship types include: OccurredAt, OriginatedFrom, BelongsToType, CausedBy, and Affects. The weight of an edge is determined by its corresponding SHAP contribution value, thus visually displaying the anomaly propagation path and scope of impact on the visualization interface.

[0089] Step 4: Cloud-edge collaborative closed-loop optimization.

[0090] S401: The model training and distribution module gathers all data uploaded by edge nodes, as well as abnormal data that has been traced and confirmed, as high-quality labeled training samples and stores them in the cloud training database.

[0091] S402: Perform incremental training or retraining of the Cooperative Anomaly Detection Model (GAT) and the Multimodal Fusion Model (Transformer) periodically in the cloud (e.g., weekly or when the number of new anomaly samples reaches a preset threshold). The total loss function during training... It consists of the following parts:

[0092]

[0093] in, The prediction loss for anomaly detection (using binary cross-entropy loss):

[0094]

[0095] This is the L2 regularization term, used to prevent the model from overfitting.

[0096]

[0097] Cross-modal consistency loss is used to constrain the semantic consistency of features from different modalities after fusion.

[0098]

[0099] and These are the weight hyperparameters for each loss term.

[0100] S403: Transfer the trained and optimized complex cloud model to a lightweight edge detection model using knowledge distillation. Distillation loss. The calculation formula is:

[0101]

[0102] in, For cross-entropy loss, Let KL divergence loss be y, and y be the true label. This is the prediction output for the student model (a lightweight model at the edge). and These are the logits outputs of the teacher model (a complex cloud model) and the student model, respectively. T is the temperature parameter (T=4 in this embodiment), and alpha is the weight balance coefficient between hard label loss and soft label loss (alpha=0.3 in this embodiment).

[0103] S404: After the distilled lightweight model is validated, it is distributed to each edge node through a secure channel to update its edge anomaly detection module. Through this cycle of "detection—source tracing—training—distillation—distribution", the performance of the entire system is continuously and adaptively optimized.

[0104] Example 1:

[0105] Step S1: Edge data processing and initial anomaly screening.

[0106] The system performs this step on an edge computing node deployed within the hospital (configured with an Intel Xeon E-2278G processor, 32GB of memory, and an NVIDIA T4 GPU). The multimodal data adaptation and acquisition module simultaneously acquires real-time physiological signals, equipment operating parameters, and nursing record texts filled out by nurses from monitors (Philips IntelliVue MX800), ventilators (Drager Evita V500), infusion pumps, and the electronic medical record (EMR) system within the ICU. Data acquisition frequencies are: ECG signal 250Hz, blood pressure data 1Hz, ventilator parameters 0.5Hz, and nursing records triggered by events.

[0107] The data preprocessing and feature extraction module performs normalization on these heterogeneous data. The ECG waveform data is denoised using a 0.5-40Hz bandpass Butterworth filter, and time-domain features such as heart rate (HR), RR interval variability (HRV), and QRS complex width, as well as frequency-domain features such as power spectral density (PSD), are extracted to form a dimensionality... =32 feature vectors; segment the nursing record text, remove stop words, and convert it using a pre-trained BioBERT model. =768-dimensional word vector representation; numerical data such as blood pressure and blood oxygen are Z-Score standardized using formula (1).

[0108] The lightweight autoencoder model embedded in the edge anomaly detection module (encoder: 3-layer fully connected network, with hidden layer dimensions of 128-64-32 respectively; decoder: 3-layer fully connected network, with hidden layer dimensions of 32-64-128 respectively) reconstructs the processed data streams and calculates the reconstruction error according to formula (2). The dynamic threshold tau is calculated according to formula (3), and the sensitivity coefficient k=3. For example, when the reconstruction error of the ECG signal is 0.85 at a certain moment, and the current dynamic threshold tau=0.62, the data stream segment is marked as "suspicious anomaly".

[0109] Reference Figure 4 The data traceability identification and encapsulation module generates a traceability identifier T for this data packet, as shown in the following example: I =="a3f7e2b1-...", =="MX800-ICU-03", =="EDGE-HOSP-A-01", ="2025-03-15T14:32:05.123Z", =ECG, =Suspicious, =0.87, =="sha256:e3b0c442...". The encapsulated data packet is uploaded to the cloud via a TLS 1.3 encrypted channel.

[0110] Step S2: Cloud data fusion and collaborative anomaly detection.

[0111] After the cloud server (configured as an NVIDIA A100 GPU cluster) receives the data packet, the data aggregation and verification module first performs TLS decryption and then recalculates the SHA-256 hash value of the data payload. With the traceability label The data was compared to confirm its integrity.

[0112] Reference Figure 5 The multimodal data deep fusion module will combine ECG feature vectors from the same 5-minute time window ( =32), ventilator parameter feature vector ( =16), blood pressure and blood oxygen feature vector ( =8) and word vector features of nursing record text ( =768) are mapped to a unified path through a linear projection layer (Equation 4). The data is processed in a 256-dimensional space, concatenated into a sequence, and modality position encoding is added before being input into a 4-layer Transformer encoder (h=8-head attention). After processing by the Transformer encoder, the 256-dimensional vector corresponding to the [CLS] label is taken as a unified high-dimensional joint feature representation. .

[0113] Reference Figure 6 The collaborative anomaly detection module initializes each node in the graph structure G to its corresponding... The time edge, spatial edge, and device edge are constructed according to the rules described in S203. The GAT model uses a 3-layer graph attention convolution (K=4 attention heads per layer), and performs attention-weighted graph convolution operations according to formulas (8)-(11). The final output layer outputs the anomaly probability score through a fully connected layer and a sigmoid function. .when If the value is greater than 0.5, it is considered abnormal.

[0114] Step S3: Explainability tracing and knowledge graph construction.

[0115] Suppose the GAT model outputs anomaly probability for a certain node. = 0.92, which is considered an anomaly. The interpretability tracing engine immediately starts the Kernel SHAP algorithm to analyze the input features of this node. An example of the SHAP calculation results is shown in Table 3.

[0116] Table 3

[0117] The above results indicate that the HRV feature contributed the most (SHAP value of +0.45), making it the most critical factor leading to the data being judged as abnormal. Based on this, the system generated a source tracing report and inferred that the possible root cause was "abnormal HRV fluctuations caused by poor electrode contact in the ECG signal acquisition device MX800-ICU-03".

[0118] Reference Figure 8 The source tracing report was updated in the source tracing knowledge graph. On the visualization interface, users can see a graph with the abnormal event as the central node, connected to entity nodes such as device "MX800-ICU-03", modality "ECG", and root cause type "poor electrode contact" through directed edges. The thickness of the edges is determined by the SHAP value.

[0119] Step S4: Cloud-edge collaborative closed-loop optimization.

[0120] Reference Figure 9 The confirmed abnormal case is stored in the cloud training database. When the cumulative number of newly added abnormal samples reaches 500, incremental training is started in the cloud, and the total loss is calculated and the model parameters are updated according to formulas (13)-(16). =0.001, = 0.01). At the same time, knowledge distillation is performed according to formula (17) (T=4, alpha=0.3), and the optimized lightweight model is sent to the edge node EDGE-HOSP-A-01 to update its autoencoder model parameters.

[0121] Experimental Data Comparison: To further demonstrate the beneficial effects of this invention, comparative experiments were conducted on a dataset containing 10,000 simulated data transmission events (built based on an extension of the MIMIC-III public database, with anomalous events accounting for approximately 8%). Benchmark method A is a centralized anomaly detection method based on traditional CNNs, and benchmark method B is an edge-independent detection method based on Isolation Forest. The experimental environment was an NVIDIA A100 GPU, and the training epochs were 100. (Table 4)

[0122] Table 4

[0123] Experimental results show that the method of this invention outperforms the two benchmark methods in terms of detection accuracy, precision, recall, and F1 score, while significantly reducing cloud computing load and network bandwidth consumption. More importantly, the method of this invention possesses interpretability and tracing capabilities that the two benchmark methods completely lack, providing a quantitative chain of evidence and root cause inference for each anomaly determination.

Claims

1. A method for interpretable anomaly tracing of multimodal medical data in cloud-edge collaborative transmission in smart hospitals, characterized in that, For collaborative systems that include both edge and cloud layers, the following steps are included: (1) At the edge layer, multimodal medical data from different medical devices are collected in real time, preprocessed and initially extracted, and a lightweight anomaly detection model deployed at the edge is used to perform real-time anomaly screening of single-modal data streams. (2) Generate and encapsulate a unique, structured traceability identifier for the processed data packet through the traceability identifier encapsulation module, and transmit the encapsulated data packet to the cloud layer through a secure channel; (3) At the cloud layer, the received data packets are verified for integrity and traceability identification. The multimodal deep fusion model based on attention mechanism is used to align and deeply fuse different modal features from the same analysis unit to generate a unified high-dimensional joint feature representation. (4) Using a collaborative anomaly detection model based on graph neural networks, the high-dimensional joint feature representation is constructed as a graph structure with data units as nodes and spatiotemporal or logical relationships as edges. Graph convolution analysis is then performed on the graph structure to detect cross-modal correlation anomalies. (5) When the collaborative anomaly detection model determines that the data is abnormal, the interpretability tracing engine is started to analyze the decision-making process of the collaborative anomaly detection model and calculate the contribution of each original input feature to the anomaly determination result. (6) Based on the contribution and source identification, and combined with the preset fault mode rule base, a structured source tracing report is automatically generated. The source tracing report contains key evidence features and their quantitative contribution, and infers possible abnormal root causes. (7) Convert the structured information in the source tracing report into graph entities and relationships, and construct or update the source tracing knowledge graph to provide a visual analysis of the propagation path, scope of impact and root cause of abnormal events; (8) In the cloud layer, the abnormal data confirmed by the source is used as training samples to perform incremental training on the collaborative anomaly detection model and the multimodal deep fusion model on a regular basis, and the optimized model parameters or the lightweight edge detection model are sent to the edge layer to realize continuous adaptive optimization of system performance.

2. The method for interpretable anomaly tracing in the cloud-edge collaborative transmission of multimodal medical data in a smart hospital, as described in claim 1, is characterized in that... In step (1), the lightweight anomaly detection model is a reconstruction error detection model based on an autoencoder. It calculates the reconstruction error between the input data and the reconstructed data, and compares the error with a dynamic threshold determined based on the reconstruction error distribution of historical normal data. When the reconstruction error exceeds the dynamic threshold, the corresponding data stream segment is marked as a suspicious anomaly.

3. The method for interpretable anomaly tracing of multimodal medical data in cloud-edge collaborative transmission in a smart hospital, as described in claim 1, is characterized in that... In step (2), the traceability identifier includes: a globally unique identifier, a source device identifier, an edge node identifier, a data acquisition timestamp, a data modality type, an edge screening result, an edge screening confidence level, and a hash digest of the data payload.

4. The method for interpretable anomaly tracing in the cloud-edge collaborative transmission of multimodal medical data in a smart hospital, as described in claim 1, is characterized in that... In step (3), the multimodal deep fusion model is a cross-modal attention fusion network based on the Transformer architecture. It maps each modal feature to a unified dimensional space through linear projection and uses a multi-head self-attention mechanism to learn the intrinsic correlation weights between different modal features, thereby generating the high-dimensional joint feature representation.

5. The method for interpretable anomaly tracing of multimodal medical data in cloud-edge collaborative transmission in a smart hospital, as described in claim 1, is characterized in that... In step (4), the collaborative anomaly detection model is a graph attention network model. The edges of the graph structure include time edges based on temporal dependencies, spatial edges based on spatial relationships, and device edges based on device relationships. The graph convolutional analysis adaptively learns the importance weights of neighboring nodes through the attention mechanism and updates the node features accordingly.

6. The method for interpretable anomaly tracing of multimodal medical data in cloud-edge collaborative transmission in a smart hospital, as described in claim 1, is characterized in that... In step (5), the contribution is calculated based on the Shapley value theory. The marginal contribution of each feature to the model prediction result is calculated under all possible combinations of feature subsets, and the computational complexity is reduced by using a sampling-based approximation algorithm.

7. The method for interpretable anomaly tracing of multimodal medical data in cloud-edge collaborative transmission in a smart hospital, as described in claim 1, is characterized in that... In step (6), the source knowledge graph takes abnormal events as the central node, and the entity types include abnormal events, medical devices, edge nodes, data modalities, abnormal types and root cause types. The relationship types include occurrence, origin, belonging to type, causing and affecting, and the weight of the edge is determined by the corresponding contribution value.

8. The method for interpretable anomaly tracing of multimodal medical data in cloud-edge collaborative transmission in a smart hospital, as described in claim 1, is characterized in that... In step (7), before the optimized model parameters are sent to the edge layer, the knowledge of the complex model in the cloud is transferred to the lightweight edge detection model through knowledge distillation technology. During the distillation process, the hard label loss and soft label loss are balanced to train the lightweight student model with improved performance.

9. A system for tracing the interpretability of multimodal medical data in cloud-edge collaborative transmission in smart hospitals, used to execute the method according to any one of claims 1-8, characterized in that, include: The edge layer includes: a multimodal data adaptation and acquisition module for real-time acquisition of various types of multimodal medical data; a data preprocessing and feature extraction module for data cleaning, format normalization, and preliminary feature extraction; an edge anomaly detection module deploying a lightweight autoencoder model for real-time anomaly screening of single-modal data streams; and a data traceability identification and encapsulation module for generating and encapsulating unique, structured traceability identifiers for the processed data packets. The cloud layer includes: a data aggregation and verification module for verifying the integrity and source identification of received data packets; a multimodal data deep fusion module deploying a cross-modal attention fusion network based on the Transformer architecture to generate a unified high-dimensional joint feature representation; a collaborative anomaly detection module deploying a graph attention network model for precise collaborative anomaly judgment on the high-dimensional joint feature representation; an interpretable source tracing engine integrating an interpretation algorithm based on Shapley value theory to generate a structured source tracing report when data is determined to be anomaly; a knowledge graph construction module for building or updating the source tracing knowledge graph based on the source tracing results; and a model training and deployment module for training and optimizing models in the cloud and deploying the optimized lightweight model to the edge layer through knowledge distillation technology.

Images