Medical knowledge constrained multi-modal time series data dynamic evaluation method and wearable medical system

Abstract

The invention discloses a medical knowledge constrained multi-modal time series data dynamic evaluation method and a wearable medical system. According to the method, a mixed perception architecture is constructed through an improved Swin-Transform time sequence encoder and a graph neural network, a medical priori mask matrix is embedded to constrain an attention mechanism, and time sequence feature extraction and cross-parameter correlation modeling of multi-modal physiological signals are enhanced; a dynamic quality evaluation model is designed, data integrity, cross-modal consistency and diagnosis effectiveness indexes are fused, an evaluation weight is adjusted in real time in combination with an adaptive weight function, and quantitative association of data quality and clinical diagnosis confidence is achieved; and when the motion artifacts or pathological conflicts are judged to exist, starting an abnormal positioning and correcting mechanism. According to the method, noise such as motion artifacts and interference can be monitored at any time, the attention to different indexes is adjusted in real time according to the risk level, and the pathological recognition accuracy and clinical credibility are improved by fusing multi-mode perception of medical knowledge.

Description

Technical Field

[0001] The present invention belongs to the technical field of data quality assessment and intelligent medical monitoring, and specifically relates to a medical knowledge-constrained multimodal time series data dynamic assessment method and a wearable medical system. Background Art

[0002] In the application of telemedicine and wearable devices, the quality of physiological signal collection (such as electrocardiogram, blood oxygen, blood pressure, respiration, motion information, etc.) directly affects the accuracy of clinical decision-making. Existing technologies mainly include:

[0003] (1) Threshold detection based on statistical process control (SPC): For example, CN113554123A monitors data quality using a fixed threshold (e.g., a heart rate > 120 bpm is considered abnormal). However, this method is difficult to adapt to individual differences and dynamic scenarios, and is prone to omissions or false positives.

[0004] (2) Single-modal deep learning models: For example, US20220139374A1 uses LSTM or CNN to detect anomalies of a single type of signal (such as ECG), but lacks logical association modeling between multiple parameters, making it difficult to effectively distinguish pathological abnormalities from noise.

[0005] (3) Traditional data cleaning tools: such as EP3674943B1, which rely on manual rule bases for data repair and have insufficient processing capabilities for complex artifacts (such as motion artifacts mixed with pathological features) or the relationship between multiple parameters.

[0006] Existing technical deficiencies and clinical impacts:

[0007] Insufficient dynamic adaptability: Unable to distinguish transient noise (such as motion artifacts) from true pathological abnormalities, resulting in limited monitoring accuracy. For example:

[0008] (1) In medical devices (such as ECG monitors and oximeters), sensors may detect non-pathological interference (such as PPG signal fluctuations caused by hand shaking). These noises may appear similar to true pathological abnormalities (such as atrial fibrillation or hypoxia) in waveform. If the algorithm cannot distinguish between the two, it may falsely report pathological problems or miss real health risks.

[0009] (2) Lack of cross-modal consistency: There is a lack of medical logic verification between different physiological parameters, and it is difficult to automatically detect and process contradictory data.

[0010] (3) Single evaluation dimension: Most methods only focus on data integrity or a certain statistical indicator and are unable to quantify their contribution to actual clinical diagnosis (ignoring the quantitative evaluation of clinical diagnostic effectiveness). Summary of the Invention

[0011] The present invention aims to overcome the deficiencies in the existing technology for multimodal medical data quality assessment, specifically to address the following problems: the confusion between wearable devices in identifying dynamic noise (motion artifacts, etc.) and pathological features in multimodal physiological signal acquisition; the reduced reliability of clinical decisions due to the lack of cross-parameter medical logic; and the problem that data quality measurement dimensions are single and difficult to link to diagnostic confidence.

[0012] A first aspect of the present invention provides a method for dynamic evaluation of multimodal time series data constrained by medical knowledge, comprising the following steps:

[0013] Data collection and preprocessing:

[0014] Collect multiple physiological signals, including but not limited to electrocardiogram (ECG), photoplethysmography (PPG), blood pressure, respiratory rate, and three-dimensional acceleration;

[0015] The collected physiological signals are denoised and synchronized to generate structured temporal features and correlation features.

[0016] Hybrid Perceptual Processing:

[0017] Inputting the structured temporal features and the correlation features into a hybrid perception architecture, wherein the hybrid perception architecture includes an improved temporal encoder and a physiological parameter correlation model;

[0018] The improved temporal encoder is used to extract temporal features, and the physiological parameter association model is used to extract cross-modal association features;

[0019] The temporal features and cross-modal correlation features are fused to generate a comprehensive feature vector.

[0020] Dynamic quality assessment:

[0021] Based on the comprehensive feature vector, the data completeness, cross-parameter medical consistency and diagnostic validity are evaluated to form a dynamic quality score;

[0022] An adaptive weight function based on risk assessment is introduced to make the assessment results adapt to the patient's current status in real time.

[0023] A second aspect of the present invention provides a wearable medical monitoring system based on dynamic quality assessment of multimodal time series data, comprising:

[0024] Data acquisition module: used to collect ECG, PPG, blood pressure, respiratory rate, three-dimensional acceleration and other physiological signals;

[0025] Hybrid perception module: includes an improved Swin-Transformer and GNN fusion unit for temporal feature extraction and medical prior constraints;

[0026] Dynamic evaluation module: Calculates data integrity, cross-parameter consistency, and diagnostic effectiveness indicators based on the multimodal fusion features to form a data quality score;

[0027] Contradiction detection module: If motion artifacts or medical logic conflicts are detected, rule tracing and abnormality location are performed;

[0028] Output interface: Delivers filtered or corrected high-quality physiological signals and decision recommendations to clinicians or upper-level applications.

[0029] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect of the present invention.

[0030] The fourth aspect of the present invention provides an electronic device, comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the first aspect of the present invention when executing the program.

[0031] One or more of the above technical solutions have the following beneficial effects:

[0032] 1. Dynamic adaptive capability: It can monitor motion artifacts, interference and other noise at any time, and adjust the focus on different indicators in real time according to the risk level, significantly reducing the false alarm rate and missed alarm rate.

[0033] 2. Multimodal perception integrating medical knowledge: Fully utilize the medical logical associations between multiple signals such as heart rate, blood oxygen, blood pressure, respiratory rate, EDA, etc. to improve the accuracy of pathology recognition and clinical credibility.

[0034] 3. Comprehensive data quality quantification: Through the three dimensions of integrity, consistency, and effectiveness, data quality can be directly linked to clinical diagnostic confidence, providing more reliable protection for telemedicine and real-time monitoring.

[0035] It should be understood that all combinations of the aforementioned concepts and the additional concepts described in more detail below, as long as such concepts are not mutually inconsistent, can be considered part of the present invention. In addition, all combinations of the claimed subject matter are considered part of the present invention.

[0036] The foregoing and other aspects, embodiments, and features of the present teachings will be more fully understood from the following description taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings, which constitute a part of the present invention, are used to provide further understanding of the present invention. The exemplary examples and descriptions of the present invention are used to explain the present invention and do not constitute improper limitations on the present invention.

[0038] Figure 1 : Schematic diagram of the overall system architecture of the present invention, showing the hybrid perception engine and dynamic evaluation process.

[0039] Figure 2 : Schematic diagram of the dynamic quality scoring model, showing the β(t) adaptive regulation mechanism. DETAILED DESCRIPTION

[0040] The following embodiments are only used to illustrate the best implementation of the present invention and are not intended to limit the scope of protection of the present invention. Any equivalent replacement or improvement made within the scope of the present invention should be considered to fall within the scope of protection of the present invention.

[0041] The present application embodiment proposes a dynamic evaluation method for multimodal time series data constrained by medical knowledge, such as Figure 1 As shown, the following steps are included:

[0042] Step 1: Multimodal data acquisition and preprocessing.

[0043] Physiological signals are collected through wearable devices (ECG electrode patches, PPG sensors, accelerometers, etc.), and wavelet transform and morphological filtering are used to jointly denoise the data. Multi-channel data are synchronized and divided into sliding time windows to generate structured time series features and correlation features.

[0044] In one example:

[0045] Wearable devices collect real-time physiological signal data from users, including electrocardiogram (ECG) electrode patches (sampling rate 200Hz), blood oxygen sensors (PPG, 125Hz), three-dimensional accelerometers (ACC, 50Hz), and skin conductance (EDA, 10Hz). The collected data is transmitted to edge computing devices (such as Jetson Xavier) via Bluetooth / 5G modules.

[0046] During the preprocessing phase, the ECG and PPG signals were subjected to wavelet transform (Symlet-5 basis functions) combined with morphological filtering to remove high-frequency noise and baseline drift. The ACC signals were bandpass filtered (0.5-15Hz) to extract motion features, which served as a reference for subsequent artifact detection. After this initial preprocessing, relatively clean time-domain and frequency-domain feature representations were obtained.

[0047] After the multi-channel data is synchronized and aligned, it is divided into 5-10 second sliding time windows with a step of 1 second. The time domain (such as waveform, statistics) and frequency domain features (such as STFT spectrum) of each signal are extracted and finally spliced ​​into a three-dimensional tensor (time step × number of channels × feature dimension) to obtain the feature tensor X 时序 (Multi-channel timing characteristics).

[0048] Step 2: Input the structured temporal features and correlation features into the hybrid perception architecture.

[0049] The hybrid perception architecture consists of an improved Swin-Transformer temporal encoder and a GNN physiological parameter association model.

[0050] Swin-Transformer is a hierarchical Transformer structure suitable for computer vision tasks. This embodiment uses it for time series data and makes two improvements: the first is to give it local-global time series perception capabilities. The original Swin-Transformer is mainly used for image processing, and here it is extended to time series modeling, enhancing short-term dependencies and long-term trend capture. The second is the attention mechanism guided by the medical prior mask, using the medical prior mask matrix M med Constraining the Cross-Attention calculation process guides the network to focus on feature channels relevant to medical rules and deemphasize channels that do not conform to clinical logic. For example, if the heart rate HR>120bpm and SpO2<90% may indicate hypoxia risk according to clinical guidelines, the weight of this feature pair can be increased in the Swin-Transformer attention calculation.

[0051] The GNN physiological parameter association model is constructed based on medical knowledge graphs or rules, and each physiological parameter node is connected in the graph.

[0052] Finally, the temporal features of Swin-Transformer are concatenated with the cross-modal features of GNN to generate a comprehensive feature vector.

[0053] In one example:

[0054] The improved Swin-Transformer temporal encoder processes the temporal data in the window in blocks and embeds the medical prior mask matrix M in the multi-head cross attention mechanism. med , and its calculation formula is:

[0055]

[0056] Where q (query) is the query vector, representing the input features of the current time step (related to the model's attention query mechanism); K (Key) is the key vector, representing the model's feature storage for different time steps; V (Value) is the value vector, representing the features of each time step; M medIt is constructed based on basic medical rules and quantified as an attention-weighted penalty or enhancement coefficient. For example, an excessively high heart rate and severely low blood oxygen often indicate the risk of acute hypoxia. When the heart rate (HR) is >130bpm and the blood oxygen (SpO2) is <90%, the corresponding attention weight is increased by 30%, prompting the model to focus on the abnormal pattern of this parameter combination.

[0057] In one example:

[0058] Construct a GNN model, and use parameters such as heart rate, blood pressure, and blood oxygen as graph nodes V. Node features include statistics such as mean and variability. The edge weight of the graph is e uv According to the medical rule base (such as the clinical guidelines for heart rate HR and blood pressure BP set a high correlation, then e (HR,BP) Take 0.8, otherwise take a lower value of 0.4) and the historical data correlation to form the node feature matrix X 关联 and adjacency matrices for graph neural network modeling cross-parameter associations.

[0059] GNN iterative update formula:

[0060]

[0061] in, represents the feature vector of node v in layer l, represents the set of neighbor nodes of node v, e uv is the edge weight between node v and node u, φ function is the aggregation function combining neighbor node features and edge weight, W (l) is a learnable weight matrix used for feature transformation, σ is a nonlinear activation function (such as ReLU), and || represents the feature concatenation operation, which combines the node's own features with the neighbor aggregation features.

[0062] Finally, the output of Swin-Transformer is cascaded and fused with the output of GNN to obtain a comprehensive feature vector F(X) across time series and parameters.

[0063] Furthermore, it also includes: differential detection of noise and pathological characteristics

[0064] A two-stream network is used to detect motion artifacts: the spectrum branch uses ResNet-18 to extract the short-time Fourier transform (STFT) features of the PPG signal; the time domain branch uses a dilated temporal convolutional network (Dilated TCN) to analyze the time domain changes of the acceleration signal. The detection method is:

[0065] y MA =MLP(CONCAT(f spectral (X PPG ),f temporal (XACC ))) (3)

[0066] Among them, spectrum feature extraction:

[0067] f spectral (X PPG )=ResNet-18(S STFT (X PPG )) (4)

[0068] Time domain feature extraction:

[0069] f temporal (X ACC )=Dilated-TCN(X ACC ,dilation=[1,2,4,8]) (5)

[0070] If y MA If the value is greater than θ (threshold), it is considered a significant motion artifact. If an anomaly is detected but the ACC data also shows significant motion, a "Noise / Pathology Uncertainty Alert" is triggered, and further investigation is conducted based on the rule base.

[0071] Step 3: Input the comprehensive feature vector generated by the hybrid perception architecture into the dynamic quality assessment system for evaluation.

[0072] like Figure 2 As shown in Figure 1, the dynamic quality assessment system comprehensively considers data integrity I(t), cross-parameter medical consistency C(t), and diagnostic validity V(t) to form a time-varying quality score q(t). An adaptive weighting function β(t) based on risk assessment is introduced into the assessment, allowing the assessment results to adapt in real time to the patient's current state (e.g., exercise, rest, condition fluctuations, etc.).

[0073] In one example:

[0074] The calculation formula of the dynamic quality scoring model is:

[0075] Q(t)=α·I(t)+β(t)·C(t)+γ·V(t) (6)

[0076]

[0077] C(t)=exp(-λ·D JS (MLP(F GNN (X))||p med )) (8)

[0078] V(t)=Sigmoid(w T Conv1D(F(X))) (9)

[0079]

[0080] The data integrity I(t) is quantified by the data missing rate and sampling anomaly rate within the time window T. is an indicator function, which takes 1 when the data at time point τ is complete and has no sampling anomalies, and takes 0 otherwise; C(t) represents the cross-modal consistency index, MLP(F GNN (X)) represents mapping the GNN output features to generate observation distribution, using Jensen-Shannon divergence D JS Characterizing the observed distribution and the medical prior distribution p med ; V(t) represents the diagnostic validity score, such as the confidence of pathological classification such as atrial fibrillation and myocardial ischemia. Conv1D is used to perform one-dimensional convolution on F(X) to extract pathological classification features; β(t) represents the urgency adaptive weight, which is affected by the risk assessment function Δ risk (t) control, Δ risk (t) Based on the real-time risk assessment model, if the system detects a high-risk event, the weighting of clinical consistency or diagnostic validity will be appropriately increased.

[0081] Further, Δ risk (t) is calculated as follows:

[0082]

[0083] Among them, φ(·) is the temporal feature activation function, which captures abnormal patterns (such as sudden increase in heart rate), w i is the time series feature weight, determined by clinical importance (e.g., HR has a higher weight than ACC), F ST,i (X) represents the temporal features from Swin-Transformer; ψ(·) is the cross-modal correlation deviation function, which calculates the logical conflict between parameters based on PPIM, v j is the cross-modal feature weight, the clinical association strength of the response parameter pair, F GNN,j (X) is the cross-modal correlation feature from GNN; Γ med It is a medical rule correction item that directly injects risk markers into the medical rule base, such as adding a fixed risk value when SpO2<90%.

[0084] Step 4: When the cross-parameter medical consistency C(t) score is abnormal, the cross-modal verification and contradiction detection function is triggered.

[0085] A physiological parameter interaction matrix (PPIM) (see the table below for a visual example of the PPIM matrix, which identifies the strength of associations between different parameters) and a medical rule base are constructed to quantify the logical connections between different physiological parameters. If the system detects a significant conflict between specific parameters and medical priors, a traceability mechanism is triggered to locate the sensor data most likely to be abnormal or noisy, completing the system's cross-modal verification and contradiction detection capabilities.

[0086] Physiological parameter interaction matrix PPIM table

[0087]

[0088] In one example, PPIM is constructed as follows:

[0089]

[0090] When the system detects that the parameter pairs in the PPIM are significantly different from the measured values, the system uses the contradiction location formula:

[0091]

[0092] in, represents the expected value based on the conditional probability table.

[0093] Trigger the attribution mechanism to identify the sensor data that is most likely to be abnormal or noisy.

[0094] Combine Q(t) with the confidence of each parameter or pathological classification, such as establishing a quality-diagnosis regression model for clinical effectiveness mapping:

[0095]

[0096] The parameter k is adjusted according to the disease type, for example, k = 2.3 is used for cardiovascular monitoring.

[0097] Obtain multi-dimensional comprehensive reports such as "high-quality data + high confidence" or "low-quality data + high risk" to assist remote doctors or automatic diagnostic systems in making decisions.

[0098] Step 5: Implement this solution into a wearable detection system. Run the solution in real time on embedded hardware (such as the NVIDIA Jetson series or other low-power AI chips) to implement functions such as noise identification, data repair, cross-parameter consistency verification, and diagnostic confidence quantification.

[0099] The present application also provides a wearable medical monitoring system based on dynamic evaluation of multimodal time series data quality, including:

[0100] Data acquisition module: used to collect ECG, PPG, blood pressure, respiratory rate, three-dimensional acceleration and other physiological signals;

[0101] Hybrid perception module: includes an improved Swin-Transformer and GNN fusion unit for temporal feature extraction and medical prior constraints;

[0102] Dynamic evaluation module: Calculates data integrity, cross-parameter consistency, and diagnostic effectiveness indicators based on the multimodal fusion features to form a data quality score;

[0103] Contradiction detection module: If motion artifacts or medical logic conflicts are detected, rule tracing and abnormality location are performed;

[0104] Output interface: Delivers filtered or corrected high-quality physiological signals and decision recommendations to clinicians or upper-level applications.

[0105] An embodiment of the present application also provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in the method for dynamic evaluation of multimodal time series data constrained by medical knowledge.

[0106] An embodiment of the present application also provides an electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps in the method for dynamic evaluation of multimodal time series data constrained by medical knowledge are implemented.

[0107] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A dynamic evaluation method for multimodal time series data constrained by medical knowledge, characterized by: The following steps are involved: Data collection and preprocessing: Collect multiple physiological signals, including ECG, photoplethysmography, blood pressure, respiratory rate, and three-dimensional acceleration; De-noise and synchronize the collected physiological signals to generate structured temporal features and correlation features; Hybrid Perceptual Processing: Inputting the structured temporal features and the correlation features into a hybrid perception architecture, wherein the hybrid perception architecture includes an improved temporal encoder and a physiological parameter correlation model; The improved temporal encoder is used to extract temporal features, and the physiological parameter association model is used to extract cross-modal association features; Fuse temporal features and cross-modal correlation features to generate a comprehensive feature vector; Dynamic quality assessment: Based on the comprehensive feature vector, the data completeness, cross-parameter medical consistency and diagnostic validity are evaluated to form a dynamic quality score; An adaptive weight function based on risk assessment is introduced to make the assessment results adapt to the patient's current status in real time.

2. The method for dynamic evaluation of multimodal time series data constrained by medical knowledge according to claim 1 is characterized in that: In the data collection and preprocessing steps: Wavelet transform and morphological filtering are used to denoise the ECG and photoplethysmography to eliminate high-frequency noise and baseline drift. Bandpass filtering is used on the three-dimensional acceleration signal to extract motion features, which serve as a reference for subsequent motion artifact detection; After the multi-channel data are synchronously aligned, they are divided into sliding time windows of 5 to 10 seconds with a step of 1 second. The time domain and frequency domain features of each signal are extracted and spliced ​​into a three-dimensional tensor.

3. The method for dynamic evaluation of multimodal time series data constrained by medical knowledge according to claim 1 or 2, characterized in that: In the hybrid perception processing step: The improved temporal encoder is a temporal encoder based on Swin-Transformer, which has local-global temporal perception capability; The physiological parameter association model is a graph neural network-based model, in which node features include mean and variability, and edge weights are generated by mixing medical rules and historical data correlation.

4. The method for dynamic evaluation of multimodal time series data constrained by medical knowledge according to claim 3 is characterized in that: In the hybrid perception processing step: A medical prior mask matrix is ​​embedded in the attention calculation of Swin-Transformer to guide the network to focus on feature channels related to medical rules and weaken channels that do not conform to clinical logic.

5. The method for dynamic evaluation of multimodal time series data constrained by medical knowledge according to claim 1 is characterized in that: In the dynamic quality assessment step: Data completeness is quantified by the data missing rate and sampling anomaly rate within the time window; Cross-parameter medical consistency is characterized by the Jensen-Shannon divergence to characterize the difference between the observed distribution and the medical prior distribution; The diagnostic validity score was scored by extracting pathological classification features through one-dimensional convolution; Adaptive weights are calculated based on a real-time risk assessment model and are adjusted according to the risk level.

6. The method for dynamic evaluation of multimodal time series data constrained by medical knowledge according to claim 5 is characterized in that: In the dynamic quality assessment step: The diagnostic validity score is mapped to clinical validity using a quality-diagnosis regression model, directly linking data quality to clinical diagnostic confidence.

7. The method for dynamic evaluation of multimodal time series data constrained by medical knowledge according to claim 1, characterized in that: Also includes contradiction detection and tracing: When anomalies occur in the cross-parameter medical consistency score, the cross-modality verification and contradiction detection functions are triggered; Construct a physiological parameter interaction matrix and a medical rule base to quantify the logical associations between different physiological parameters; Locate sensor data that is most likely to be anomaly or noisy.

8. The method for dynamic evaluation of multimodal time series data constrained by medical knowledge according to claim 7 is characterized in that: In the contradiction detection and tracing step: When a parameter pair is detected that differs greatly from the measured value, the sensor data that is most likely to be abnormal or noisy is determined through the contradiction location formula; Trigger the traceability mechanism and combine it with the medical rule base to further investigate anomalies.

9. The method for dynamic evaluation of multimodal time series data constrained by medical knowledge according to claim 1, characterized in that: A dual-stream network is used to detect motion artifacts. The spectrum branch uses ResNet-18 to extract the short-time Fourier transform features of the photoplethysmography signal, and the time domain branch uses a dilated temporal convolutional network to analyze the time domain changes of the three-dimensional acceleration signal. If an abnormality is detected but the accelerometer data also shows significant movement, a "noise / pathology uncertainty alert" is triggered and further investigation is carried out in conjunction with the medical rule base.

10. The medical knowledge-constrained multimodal time series data dynamic evaluation method according to claim 1, characterized in that: It also includes output and application: delivering high-quality physiological signals and decision-making recommendations to clinicians or upper-level applications.

Images