Clinical time series data cleaning method and system based on hierarchical agent reinforcement learning

Abstract

The invention discloses a clinical time series data cleaning method and system based on hierarchical agent reinforcement learning, and belongs to the technical field of medical data processing, and the method comprises the steps: carrying out the detection of real-time multivariable time series data, and obtaining a data missing rate, a data abnormal rate, and a constraint violation rate; taking processing priorities and repair types of data missing, data exception and constraint violation as a first action space, constructing a first reward function through the overall data quality index improvement degree, taking a cleaning operator as a second action space, and constructing a second reward function through the single data quality index improvement degree; constructing a global quality control agent and a local cleaning agent; and cleaning the real-time multivariable time sequence data through the global quality control agent and the local cleaning agent. The technical problem that data cleaning quality and cleaning efficiency are difficult to improve in the prior art is solved.

Description

A Method and System for Cleaning Clinical Time-Series Data Based on Hierarchical Agent Reinforcement Learning Technical Field

[0001] This invention relates to the field of medical data processing technology, specifically to a method and system for cleaning clinical time-series data based on hierarchical agent reinforcement learning. Background Technology

[0002] With the rapid development of information-based healthcare and intelligent diagnostic systems, a large amount of multivariate time-series data is generated in obstetrics and gynecology clinical practice, such as pregnant women's vital signs, fetal heart rate monitoring signals, uterine contraction curves, biochemical test indicators, and medication records. This type of data is of significant value in assisting clinical decision-making, disease risk warning, and the construction of intelligent diagnostic models. However, due to the complex clinical data collection environment, strong equipment heterogeneity, and frequent human intervention, the actual multivariate time-series data generally suffers from various quality problems such as missing values, outliers, and violated constraints, severely restricting the reliable use of the data. Obstetrics and gynecology clinical multivariate time-series data not only exhibits significant dynamic evolution characteristics in the time dimension, but also has strong constraints between different variables. For example, there is a clear physiological correlation between fetal heart rate, uterine contraction intensity, and gestational age; unreasonable data repair may disrupt the original medical consistency. However, most existing data cleaning methods often treat missing value imputation, outlier detection, and consistency repair as independent processing steps, adopting a serial pipeline mode of "detect first, repair later." This fragmented approach ignores the interplay between various quality defects, easily introducing new anomalies or violating logical constraints while fixing one problem, making it difficult to improve the overall quality of clinical data. Furthermore, in multivariate time-series scenarios in obstetrics and gynecology, the decision space for cleaning strategies is highly complex. With the increase in the number of variables, types of cleaning operators, and parameter configuration dimensions, the combined decisions regarding "cleaning operator selection, variable location, and parameter scheduling" grow exponentially. Traditional heuristic methods or single reinforcement learning models are prone to the curse of dimensionality in such high-dimensional, multi-layered decision-making problems, making it difficult to find stable and effective cleaning strategies within an acceptable timeframe, thus limiting their feasibility in real-world clinical settings. Therefore, improving data cleaning quality and efficiency remains a technical challenge that current technologies struggle to address. Summary of the Invention

[0003] To address the technical problem that existing technologies struggle to improve the quality and efficiency of data cleaning, this invention provides a clinical time-series data cleaning method and system based on hierarchical agent reinforcement learning. By constructing a global-local hierarchical agent collaborative cleaning architecture and combining precise design of multi-dimensional action space and reward function, it achieves efficient and high-quality cleaning of real-time multivariate time-series data. This effectively solves the technical problems in existing technologies, such as fragmented processing ignoring the mutual influence between various quality defects and the dimensionality curse caused by high-dimensional decision space, which makes it difficult to improve the quality and efficiency of data cleaning.

[0004] To address the aforementioned technical problems, this invention provides a clinical time-series data cleaning method based on hierarchical agent reinforcement learning, comprising the following steps: performing missing data, anomaly detection, and constraint violation detection on real-time multivariate time-series data to obtain the missing data rate, anomaly rate, and constraint violation rate; using the processing priorities and repair types of missing data, anomalies, and constraint violations as the first action space, constructing a first reward function based on the overall data quality index improvement degree; using the cleaning operator as the second action space, constructing a second reward function based on the improvement degree of individual data quality indicators; constructing a global quality control agent based on the first action space and the first reward function, and constructing a local cleaning agent based on the second action space and the second reward function; inputting the missing data rate, anomaly rate, and constraint violation rate into the global quality control agent to obtain the actual processing priorities and actual repair types of missing data, anomalies, and constraint violations; inputting the actual repair types into the local cleaning agent according to the actual processing priorities of missing data, anomalies, and constraint violations to obtain the actual cleaning operator, and cleaning the real-time multivariate time-series data using the actual cleaning operator.

[0005] Preferably, the step of detecting missing, anomalies, and constraints in real-time multivariate time-series data to obtain the missing data rate, anomaly rate, and constraint violation rate includes: performing time alignment processing on the real-time multivariate time-series data; performing missing data detection on the real-time multivariate time-series data to obtain the missing data rate; performing feature extraction on the real-time multivariate time-series data to obtain data features; selecting an anomaly detection model based on the data features; performing anomaly detection on the real-time multivariate time-series data using the anomaly detection model to obtain the anomaly rate; obtaining historical multivariate time-series data; performing constraint mining on the historical multivariate time-series data to obtain a constraint set; and obtaining the constraint violation rate based on the constraint set; wherein, continuous redundant data in the real-time multivariate time-series data are marked as missing data.

[0006] Preferably, the step of extracting features from real-time multivariate time-series data to obtain data features, selecting an anomaly detection model based on the data features, and using the anomaly detection model to detect anomalies in the real-time multivariate time-series data to obtain the data anomaly rate includes: extracting conventional features from the real-time multivariate time-series data to obtain conventional data features; comparing the real-time multivariate time-series data with the normal data range to obtain anomaly detection features; inputting the real-time multivariate time-series data into a candidate anomaly detection model to obtain basic model features; using meta-learning to select an anomaly detection model based on the conventional data features, anomaly detection features, and basic model features; inputting the real-time multivariate time-series data into the anomaly detection model to obtain the average anomaly score for each data point in the real-time multivariate time-series data, and determining data with an average anomaly score greater than a preset threshold as anomaly data, thereby obtaining the data anomaly rate.

[0007] Preferably, the step of obtaining a constraint set by constraint mining of historical multivariate time series data and obtaining the constraint violation rate based on the constraint set includes: using multinomial features and OLS regression to mine the functional relationships between variables in historical multivariate time series data to obtain inter-variable functional constraints; obtaining intra-variable time series change constraints based on historical time series data of the same variable in historical multivariate time series data; obtaining inter-variable time series monotonic constraints based on the monotonic influence relationships between variables in historical multivariate time series data; obtaining change intervals based on historical time series data of the same variable in historical multivariate time series data, combining the change intervals of different variables in the same time period to obtain candidate negation intervals, and obtaining inter-variable negation constraints based on the candidate negation intervals; and obtaining violation data in real-time multivariate time series data based on inter-variable functional constraints, intra-variable time series change constraints, inter-variable time series monotonic constraints, and inter-variable negation constraints, thereby obtaining the constraint violation rate.

[0008] Preferably, the step of constructing the first reward function based on the improvement of the overall data quality index includes: constructing the first reward function by weighted summation of the improvement of the overall data quality index, the time cost penalty during the cleaning process, and the invalid action penalty.

[0009] Preferably, the step of constructing the second reward function based on the improvement degree of a single data quality indicator includes: constructing the second reward function by weighted summation of the improvement degree of the single data quality indicator, the smoothness of the data structure, and the data disturbance indicator.

[0010] Preferably, the step of constructing a global quality control agent based on a first action space and a first reward function, and constructing a local cleaning agent based on a second action space and a second reward function, includes: constructing an initial global quality control agent based on the first action space and the first reward function, and constructing an initial local cleaning agent based on the second action space and the second reward function; inputting the historical data missing rate, historical data anomaly rate, and historical constraint violation rate of historical multivariate time series data into the initial global quality control agent to obtain the historical processing priorities and historical repair types for data missing, data anomalies, and constraint violations; inputting the historical repair types into the initial local cleaning agent to obtain historical cleaning operators; cleaning the historical multivariate time series data sequentially through the historical cleaning operators according to the historical processing priorities for data missing, data anomalies, and constraint violations to obtain cleaned historical multivariate time series data, and obtaining reference index values ​​for the cleaned historical multivariate time series data; updating the state-action value functions in the initial global quality control agent and the initial local cleaning agent based on the reference index values ​​to obtain the global quality control agent and the local cleaning agent.

[0011] Preferably, the step of sequentially cleaning historical multivariate time-series data using historical cleaning operators to obtain cleaned historical multivariate time-series data based on the historical processing priorities of missing data, data anomalies, and constraint violations includes: cleaning the historical multivariate time-series data using historical cleaning operators to obtain first cleaned historical multivariate time-series data based on the historical processing priorities of missing data, data anomalies, and constraint violations; cleaning the first cleaned historical multivariate time-series data using historical cleaning operators to obtain second cleaned historical multivariate time-series data based on the historical processing priorities of missing data, data anomalies, and constraint violations; and cleaning the second cleaned historical multivariate time-series data using historical cleaning operators to obtain third cleaned historical multivariate time-series data based on the historical processing priorities of missing data, data anomalies, and constraint violations.

[0012] Preferably, the step of updating the state-action value function of the initial global quality control agent and the initial local cleaning agent based on reference index values ​​to obtain the global quality control agent and the local cleaning agent includes: obtaining a first reward value and a second reward value based on reference index values ​​of the third cleaning historical multivariate time series data; updating the state-action value function of the initial global quality control agent based on the first reward value to obtain the global quality control agent; obtaining a third reward value based on reference index values ​​of the first cleaning historical multivariate time series data; obtaining a fourth reward value based on reference index values ​​of the second cleaning historical multivariate time series data; and updating the state-action value function of the initial local cleaning agent based on the second reward value, the third reward value, and the fourth reward value to obtain the local cleaning agent.

[0013] After adopting the above technical solution, the present invention has the following advantages: By constructing a global-local hierarchical intelligent agent collaborative cleaning architecture, combined with the precise design of multi-dimensional action space and reward function, efficient and high-quality cleaning of real-time multivariate time series data is achieved. This effectively solves the technical problems in the prior art where fragmented processing ignores the mutual influence between various quality defects and the dimensionality curse caused by high-dimensional decision space, making it difficult to improve the quality and efficiency of data cleaning. The cleaning decision is split into two low-dimensional action spaces: a global strategy layer (processing priority and repair type) and a local execution layer (cleaning operator). This architecturally solves the dimensionality curse faced by traditional single models. The first reward function incorporates the improvement degree of overall data quality indicators, time cost penalty, and invalid action penalty. The second reward function incorporates the improvement degree of individual data quality indicators, data structure smoothness, and data disturbance indicators. This allows the global quality control intelligent agent to focus on overall quality and efficiency, and the local cleaning intelligent agent to focus on precise repair and data pattern preservation, thereby avoiding excessive cleaning and pattern destruction under the guidance of a single quality indicator. This achieves a triple improvement in cleaning quality, cleaning efficiency, and data pattern preservation, thereby improving the feasibility of the cleaned data. Attached Figure Description

[0014] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings. The drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings.

[0015] Figure 1 is a flowchart illustrating the clinical time-series data cleaning method based on hierarchical agent reinforcement learning according to the present invention; Figure 2 is an overall framework diagram of the clinical time-series data cleaning system based on hierarchical agent reinforcement learning according to the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only one preferred embodiment of this invention and are only used to explain this invention. They do not limit the scope of protection of this invention. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0017] Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations (or steps) can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but it may also have additional steps not included in the figures; the process may correspond to a method, function, procedure, subroutine, subroutine, etc.

[0018] Example 1: As shown in Figure 1, the clinical time series data cleaning method based on hierarchical agent reinforcement learning includes the following steps: S1: The real-time multivariate time series data is subjected to missing, abnormal and default detection to obtain the data missing rate, data abnormal rate and constraint violation rate.

[0019] As an optional embodiment, the step of detecting missing, anomalies, and constraints in real-time multivariate time-series data to obtain the missing data rate, anomaly rate, and constraint violation rate includes: performing time alignment processing on the real-time multivariate time-series data; performing missing data detection on the real-time multivariate time-series data to obtain the missing data rate; performing feature extraction on the real-time multivariate time-series data to obtain data features; selecting an anomaly detection model based on the data features; performing anomaly detection on the real-time multivariate time-series data using the anomaly detection model to obtain the anomaly rate; obtaining historical multivariate time-series data; performing constraint mining on the historical multivariate time-series data to obtain a constraint set; and obtaining the constraint violation rate based on the constraint set; wherein, continuous redundant data in the real-time multivariate time-series data are marked as missing data.

[0020] Understandably, real-time multivariate time-series data refers to time-series data containing multiple clinical indicators collected under a unified time axis. Real-time multivariate time-series data can originate from hospital information systems, monitoring equipment, or clinical databases. Addressing issues such as timestamp chaos, uneven intervals, duplication, and jumps caused by heterogeneous data acquisition from multiple clinical devices and human intervention, time alignment processing ensures that timestamps meet the constraints of equal intervals, non-repetition, and monotonically increasing. This fundamentally solves the problem of time dimension disorder in real-time multivariate time-series data, guaranteeing the continuity and standardization of the time dimension within the same time-series data, and achieving time dimension synchronization between multiple variables. This avoids deviations in subsequent feature extraction, constraint mining, and operator execution caused by timestamp chaos, while providing a standardized time-series data foundation for subsequent anomaly detection and constraint violation detection, thus improving the reliability of detection results.

[0021] Understandably, in the process of obtaining the missing data rate by performing missing data detection on real-time multivariate time series data, it is necessary not only to mark the originally missing data as missing data, but also to mark the continuous redundant data in the real-time multivariate time series data as missing data. Continuous redundant data specifically refers to continuous data that has no clinical significance and is merely mechanically repeated. Taking fetal heart rate monitoring as an example, if due to contact failure of the acquisition equipment or data transmission delay, the fetal heart rate data collected for a certain period of time (such as 30 consecutive seconds) is always fixed at 140 beats / min, and the fetal heart rate in the first 10 seconds and the last 10 seconds of this 30 seconds is also 140 beats / min, forming continuous redundant data with the adjacent historical sequence, considering that the physiological heart rate of the fetus cannot be completely without fluctuation within 30 seconds, this continuous fixed data is not the true heart rate state of the fetus, but an invalid redundant value collected by the equipment, which has no clinical reference value. Therefore, the data acquired within this 30 seconds is marked as missing data. By truly reflecting the degree of missing effective information in clinical data, the authenticity and accuracy of the data missing rate statistics are improved.

[0022] Specifically, the step of extracting features from real-time multivariate time-series data to obtain data features, selecting an anomaly detection model based on the data features, and using the anomaly detection model to detect anomalies in the real-time multivariate time-series data to obtain the data anomaly rate includes: extracting conventional features from the real-time multivariate time-series data to obtain conventional data features; comparing the real-time multivariate time-series data with the normal data range to obtain anomaly detection features; inputting the real-time multivariate time-series data into a candidate anomaly detection model to obtain basic model features; using meta-learning to select an anomaly detection model based on the conventional data features, anomaly detection features, and basic model features; inputting the real-time multivariate time-series data into the anomaly detection model to obtain the average anomaly score for each data point in the real-time multivariate time-series data; and determining data with an average anomaly score greater than a preset threshold as anomaly data, thereby obtaining the data anomaly rate.

[0023] Understandably, conventional data features include statistical features (such as mean, variance, quantiles, etc.), distribution features, correlation features (correlation, covariance), and structural features (sparseness, entropy). Statistical features, including mean, variance, and quantiles, are used to describe the overall fluctuations and central tendency of real-time multivariate time series data. Distribution features describe the shape of the numerical distribution, reflecting whether the real-time multivariate time series data conforms to a classical probability distribution, the symmetry / skewness of the distribution, etc., and are key criteria for selecting anomaly detection models. Correlation features describe the interdependence and mutual influence among multiple variables. Structural features describe the overall data structure, time series organization, and effective information density of real-time multivariate time series data, reflecting the spatiotemporal distribution patterns, completeness, and sparsity of the real-time multivariate time series data. Outlier detection features represent the proportion, location, and other characteristics of suspected anomalies in real-time multivariate time series data. Candidate anomaly detection models are specifically unsupervised anomaly detection models, such as HBOS, Isolation Forest, PCA, LODA, etc. By inputting real-time multivariate time-series data into candidate anomaly detection models to extract basic model features (such as the model's anomaly score distribution and detection accuracy), the initial adaptability of different models to real-time multivariate time-series data is directly reflected. By combining regular data features, anomaly detection features, and basic model features into a fixed-size vector, and then using meta-learning to recommend the K most suitable unsupervised anomaly detection models for the real-time multivariate time-series data, these K unsupervised anomaly detection models are used to perform anomaly detection operations on the real-time multivariate time-series data to obtain corresponding anomaly scores. Subsequently, the average anomaly score for each data point is calculated and compared with a preset threshold. If the average anomaly score exceeds the preset threshold, the data is determined to be anomalous. Users can flexibly set the preset threshold according to their actual needs. In this embodiment, combining multi-dimensional indicators to select anomaly detection models significantly improves the adaptability of anomaly detection models to real-time multivariate time-series data, thereby improving the accuracy of the data anomaly rate.

[0024] Specifically, the process of obtaining a constraint set by constraint mining of historical multivariate time series data and obtaining the constraint violation rate based on the constraint set includes: using multinomial features and OLS regression to mine the functional relationships between variables in historical multivariate time series data to obtain inter-variable functional constraints; obtaining intra-variable temporal change constraints based on historical time series data of the same variable in historical multivariate time series data; obtaining inter-variable temporal monotonic constraints based on the monotonic influence relationships between variables in historical multivariate time series data; obtaining change intervals based on historical time series data of the same variable in historical multivariate time series data, combining the change intervals of different variables in the same time period to obtain candidate negation intervals, and obtaining inter-variable negation constraints based on the candidate negation intervals; and obtaining violation data in real-time multivariate time series data based on inter-variable functional constraints, intra-variable temporal change constraints, inter-variable temporal monotonic constraints, and inter-variable negation constraints, thereby obtaining the constraint violation rate.

[0025] Multinomial features: These feature processing methods extend and cross-combine historical multivariate time-series data using power functions, transforming linearly inseparable original features into high-dimensional features capable of fitting nonlinear relationships. This is a preliminary feature processing method for fitting complex functional relationships between variables in regression analysis. OLS regression: The most commonly used linear regression method, it solves for the coefficients of the regression model by minimizing the sum of squared errors between predicted and actual values, ultimately obtaining the functional relationship between variables. Understandably, intra-variable temporal change constraints specifically refer to the rate of change, acceleration, and fluctuation amplitude of a single variable over time, such as "heart rate changes no more than 10 times per minute." Inter-variable temporal monotonic constraints specifically refer to the monotonically positive / negative correlation between multiple variables over time, such as "when body temperature rises, the number of white blood cells also increases monotonically." Inter-variable negation constraints specifically refer to the constraint rules corresponding to the combined values ​​of multiple variables that do not exist in normal data, such as "when blood glucose is <3.9 mmol / L, the insulin injection dose cannot be >20 units." It is important to emphasize that after obtaining candidate negation intervals, each candidate negation interval needs to be verified. Inter-variable negation constraints are obtained based on candidate negation intervals that appear less than or equal to a preset number of times in historical multivariate time-series data. The preset number of times is flexibly set according to the scale of historical multivariate time-series data. In this embodiment, by conducting multi-dimensional mining of data constraints, the completeness and comprehensiveness of constraint mining are effectively improved, thereby significantly enhancing the scenario adaptability and practical application value of the constraint violation rate calculation results.

[0026] S2: The first action space is based on the processing priority and repair type of data missing, data anomaly and constraint violation. The first reward function is constructed by the improvement of the overall data quality index. The second action space is based on the cleaning operator. The second reward function is constructed by the improvement of individual data quality index.

[0027] S3: Construct a global quality control agent based on the first action space and the first reward function, and construct a local cleaning agent based on the second action space and the second reward function.

[0028] Understandably, the first reward function serves as the basis for the global quality control agent's decision-making reward and punishment. It is constructed with the overall data quality index improvement as its core and is a quantitative standard for measuring the quality of the global quality control agent's decisions. The reward and punishment logic is as follows: the higher the overall data quality index improvement, the greater the reward value of the first reward function, and vice versa. The purpose is to guide the global quality control agent to learn and select the cleaning priority and repair type that can optimize the overall data quality, avoiding the upper-level decision-making from focusing only on the repair of a single problem and ignoring the overall optimization of the quality of the entire data. This avoids the technical problem in existing technologies where fragmented processing ignores the mutual influence between multiple quality defects, making it difficult to improve the quality of data cleaning. The second reward function serves as the basis for rewarding and penalizing the local cleaning agent. It is constructed with the improvement of a single data quality indicator as its core and is a quantitative standard for measuring the quality of the cleaning operator selected by the local cleaning agent. The reward and penalty logic is as follows: the higher the improvement of a single data quality indicator, the greater the reward value of the second reward function, and vice versa. The purpose is to guide the local cleaning agent to learn and match the optimal cleaning operator for each type of repair, thereby maximizing the cleaning effect under a single repair type.

[0029] By constructing a global-local hierarchical intelligent agent collaborative cleaning architecture, the high-dimensional cleaning task is decoupled into "priority decision-making" and "operator scheduling", which effectively reduces the complexity of the cleaning strategy decision space and improves the efficiency and stability of cleaning strategy search in multivariate scenarios.

[0030] As an optional embodiment, the construction of a global quality control agent based on a first action space and a first reward function, and a local cleaning agent based on a second action space and a second reward function, includes: constructing an initial global quality control agent based on the first action space and the first reward function, and constructing an initial local cleaning agent based on the second action space and the second reward function; inputting the historical data missing rate, historical data anomaly rate, and historical constraint violation rate of historical multivariate time series data into the initial global quality control agent to obtain the historical processing priorities and historical repair types for data missing, data anomalies, and constraint violations; inputting the historical repair types into the initial local cleaning agent to obtain historical cleaning operators; cleaning the historical multivariate time series data sequentially through the historical cleaning operators according to the historical processing priorities for data missing, data anomalies, and constraint violations to obtain cleaned historical multivariate time series data, and obtaining reference index values ​​for the cleaned historical multivariate time series data; updating the state-action value functions in the initial global quality control agent and the initial local cleaning agent based on the reference index values ​​to obtain the global quality control agent and the local cleaning agent.

[0031] In this embodiment, the historical repair types include missing value repair, outlier value repair, and constraint violation repair.

[0032] Specifically, the step of sequentially cleaning historical multivariate time-series data using historical cleaning operators based on the historical processing priorities of missing data, data anomalies, and constraint violations to obtain cleaned historical multivariate time-series data includes: cleaning the historical multivariate time-series data using historical cleaning operators based on the historical processing priorities of missing data, data anomalies, and constraint violations to obtain first cleaned historical multivariate time-series data; cleaning the first cleaned historical multivariate time-series data using historical cleaning operators based on the historical processing priorities of missing data, data anomalies, and constraint violations to obtain second cleaned historical multivariate time-series data; and cleaning the second cleaned historical multivariate time-series data using historical cleaning operators based on the historical processing priorities of missing data, data anomalies, and constraint violations to obtain third cleaned historical multivariate time-series data.

[0033] Taking the historical processing priority of missing data, data anomalies, and constraint violations as data missing > data anomalies > constraint violations as an example, the historical multivariate time series data is first cleaned according to the historical cleaning operator corresponding to missing data to obtain the first cleaned historical multivariate time series data. Then, the first cleaned historical multivariate time series data is cleaned according to the historical cleaning operator corresponding to data anomalies to obtain the second cleaned historical multivariate time series data. Finally, the second cleaned historical multivariate time series data is cleaned according to the historical cleaning operator corresponding to constraint violations to obtain the third cleaned historical multivariate time series data.

[0034] Specifically, the step of updating the state-action value function of the initial global quality control agent and the initial local cleaning agent based on reference index values ​​to obtain the global quality control agent and the local cleaning agent includes: obtaining a first reward value and a second reward value based on reference index values ​​of the third cleaning historical multivariate time series data; updating the state-action value function of the initial global quality control agent based on the first reward value to obtain the global quality control agent; obtaining a third reward value based on reference index values ​​of the first cleaning historical multivariate time series data; obtaining a fourth reward value based on reference index values ​​of the second cleaning historical multivariate time series data; and updating the state-action value function of the initial local cleaning agent based on the second reward value, the third reward value, and the fourth reward value to obtain the local cleaning agent.

[0035] In this embodiment, obtaining the first reward value and the second reward value based on the reference index values ​​of the third-cleaned historical multivariate time series data specifically refers to: obtaining the overall data quality index improvement degree based on multiple reference index values ​​in the reference index values ​​of the third-cleaned historical multivariate time series data and the historical data missing rate, historical data anomaly rate, and historical constraint violation rate of the historical multivariate time series data, and inputting the overall data quality index improvement degree into the first reward function to obtain the first reward value; obtaining the first single data quality index improvement degree based on the single reference index value in the reference index values ​​of the third-cleaned historical multivariate time series data and the historical data missing rate, historical data anomaly rate, or historical constraint violation rate of the historical multivariate time series data, and inputting the first single data quality index improvement degree into the second reward function to obtain the second reward value; wherein, the multiple reference index values ​​include the historical data missing rate, historical data anomaly rate, and historical constraint violation rate of the third-cleaned historical multivariate time series data. If the historical processing priority for missing data, data anomalies, and constraint violations is missing data > data anomalies > constraint violations, then the single reference indicator value is the historical constraint violation rate of the third-cleaned historical multivariate time series data. Specifically, it refers to obtaining the improvement degree of the first single data quality indicator based on the historical constraint violation rate of the third-cleaned historical multivariate time series data and the historical constraint violation rate of the historical multivariate time series data, and inputting the improvement degree of the first single data quality indicator into the second reward function to obtain the second reward value. At this point, the reference indicator value for the first cleaning of historical multivariate time series data specifically refers to the historical data missing rate of the first cleaning of historical multivariate time series data. Based on the historical data missing rate of the first cleaning of historical multivariate time series data and the historical data missing rate of historical multivariate time series data, the improvement degree of the second single data quality indicator is obtained. The improvement degree of the second single data quality indicator is input into the second reward function to obtain the third reward value. The reference indicator value for the second cleaning of historical multivariate time series data specifically refers to the historical data anomaly rate of the second cleaning of historical multivariate time series data. Based on the historical data anomaly rate of the second cleaning of historical multivariate time series data and the historical data anomaly rate of historical multivariate time series data, the improvement degree of the third single data quality indicator is obtained. The improvement degree of the third single data quality indicator is input into the second reward function to obtain the fourth reward value.

[0036] In this embodiment, when the updated global quality control agent outputs a post-cleaning reference index value corresponding to the processing priority and repair type for the same batch of historical multivariate time-series data, and the fluctuation range is ≤ a preset value (e.g., ±1%), it indicates that the state-action value function in the initial global quality control agent has been updated. Similarly, when the updated local cleaning agent outputs a post-cleaning reference index value corresponding to the cleaning operator for the same batch of historical multivariate time-series data, and the improvement variance under different batches of historical multivariate time-series data tests is ≤ a preset value (e.g., 0.01), it indicates that the state-action value function in the initial local cleaning agent has been successfully updated.

[0037] S4: Input the data missing rate, data anomaly rate, and constraint violation rate into the global quality control agent to obtain the actual processing priority and actual repair type of data missing, data anomaly, and constraint violation.

[0038] S5: Based on the actual processing priority of missing data, data anomalies, and constraint violations, the actual repair type is input into the local cleaning agent to obtain the actual cleaning operator, and the real-time multivariate time series data is cleaned through the actual cleaning operator.

[0039] In this embodiment, by constructing a global-local hierarchical intelligent agent collaborative cleaning architecture and combining a precise design of a multi-dimensional action space and reward function, efficient and high-quality cleaning of real-time multivariate time-series data is achieved. This effectively solves the technical problems in the prior art where fragmented processing ignores the mutual influence between various quality defects and the dimensionality curse caused by high-dimensional decision space, making it difficult to improve the quality and efficiency of data cleaning.

[0040] Example 2: The difference from Example 1 is that the construction of the first reward function through the improvement of the overall data quality index includes: weighting and summing the improvement of the overall data quality index, the time cost penalty during the cleaning process, and the invalid action penalty to construct the first reward function.

[0041] The construction of the second reward function based on the improvement of a single data quality indicator includes: weighting and summing the improvement of the single data quality indicator, the smoothness of the data structure, and the data disturbance indicator to construct the second reward function.

[0042] In this embodiment, the first reward function incorporates the improvement of overall data quality indicators, time cost penalties, and invalid action penalties, while the second reward function incorporates the improvement of individual data quality indicators, data structure smoothness, and data disturbance indicators. This allows the global quality control agent to focus on overall quality and efficiency, and the local cleaning agent to focus on precise repair and data pattern preservation. This avoids over-cleaning and pattern destruction guided by a single quality indicator, achieving a triple improvement in cleaning quality, cleaning efficiency, and data pattern preservation, thereby enhancing the feasibility of the cleaned data.

[0043] In this embodiment, obtaining the first reward value and the second reward value based on the reference index values ​​of the third historical multivariate time series data specifically refers to: obtaining the overall data quality index improvement degree based on multiple reference index values ​​in the reference index values ​​of the third historical multivariate time series data and the historical data missing rate, historical data anomaly rate, and historical constraint violation rate of the historical multivariate time series data; obtaining the time cost penalty in the cleaning process based on the acquisition time in the reference index values ​​of the third historical multivariate time series data; obtaining the invalid action penalty based on the number of invalid actions in the reference index values ​​of the third historical multivariate time series data; inputting the overall data quality index improvement degree, time cost penalty, and invalid action penalty into the first reward function to obtain the first reward value; and obtaining the first reward value based on the reference index values ​​of the third historical multivariate time series data. The improvement degree of the first single data quality indicator is obtained by comparing the individual reference indicator value in the evaluation index with the historical data missing rate, historical data anomaly rate, or historical constraint violation rate of the historical multivariate time series data. The first data structure smoothness is obtained based on the time series data fluctuation characteristics and trend characteristics in the reference indicator values ​​of the third-cleaned historical multivariate time series data. The first data disturbance indicator is obtained based on the time series data fluctuation amplitude, mutation frequency, and dispersion in the reference indicator values ​​of the third-cleaned historical multivariate time series data. The improvement degree of the first single data quality indicator, the first data structure smoothness, and the first data disturbance indicator are input into the second reward function to obtain the second reward value. The multiple reference indicator values ​​include the historical data missing rate, historical data anomaly rate, and historical constraint violation rate of the third-cleaned historical multivariate time series data. Acquisition time is the total time taken from acquiring the historical multivariate time series data to acquiring the third-cleaned historical multivariate time series data. The number of invalid actions is the number of invalid actions taken during the process from acquiring the historical multivariate time series data to acquiring the third-cleaned historical multivariate time series data.

[0044] If the historical processing priority for missing data, data anomalies, and constraint violations is data missing > data anomalies > constraint violations, then the single reference indicator value is the historical constraint violation rate of the third-cleaned historical multivariate time series data. Specifically, this refers to obtaining the improvement degree of the first single data quality indicator based on the historical constraint violation rate of the third-cleaned historical multivariate time series data and the historical constraint violation rate of the historical multivariate time series data. At this time, obtaining the third reward value based on the reference indicator value of the first-cleaned historical multivariate time series data specifically refers to: obtaining the improvement degree of the second single data quality indicator based on the historical data missing rate of the first-cleaned historical multivariate time series data and the historical data missing rate of the historical multivariate time series data; obtaining the second data structure smoothness based on the time series data fluctuation characteristics and trend characteristics in the reference indicator value of the first-cleaned historical multivariate time series data; obtaining the second data disturbance indicator based on the time series data fluctuation amplitude, mutation frequency, and dispersion in the reference indicator value of the first-cleaned historical multivariate time series data; and inputting the second single data quality indicator improvement degree, the second data structure smoothness, and the second data disturbance indicator into the second reward function to obtain the third reward value. The fourth reward value is obtained based on the reference index values ​​of the second-cleaned historical multivariate time series data. Specifically, this involves: obtaining the improvement degree of the third single data quality index based on the historical data anomaly rate of the second-cleaned historical multivariate time series data; obtaining the third data structure smoothness based on the time series data fluctuation characteristics and trend characteristics in the reference index values ​​of the second-cleaned historical multivariate time series data; obtaining the third data disturbance index based on the time series data fluctuation amplitude, mutation frequency, and dispersion in the reference index values ​​of the second-cleaned historical multivariate time series data; and inputting the third single data quality index improvement degree, the third data structure smoothness, and the third data disturbance index into the second reward function to obtain the fourth reward value.

[0045] Example 3: This example also provides a clinical time-series data cleaning system based on hierarchical agent reinforcement learning, applicable to the aforementioned clinical time-series data cleaning method based on hierarchical agent reinforcement learning. It includes: a data quality detection module, used to detect missing, abnormal, and violation rates in real-time multivariate time-series data to obtain the missing data rate, abnormal data rate, and constraint violation rate; and an agent configuration module, used to construct a first reward function based on the processing priority and repair type of missing data, abnormal data, and constraint violations as the first action space, and the overall data quality indicator improvement degree as the second action space, and to construct a second reward function based on the improvement degree of individual data quality indicators, using the cleaning operator as the second action space. The system comprises: a hierarchical intelligent agent construction module, used to construct a global quality control intelligent agent based on a first action space and a first reward function, and a local cleaning intelligent agent based on a second action space and a second reward function; a global cleaning strategy decision module, used to input the data missing rate, data anomaly rate, and constraint violation rate into the global quality control intelligent agent to obtain the actual processing priority and actual repair type of data missing, data anomaly, and constraint violation; and a local cleaning execution module, used to input the actual repair type into the local cleaning intelligent agent according to the actual processing priority of data missing, data anomaly, and constraint violation to obtain the actual cleaning operator, and to clean the real-time multivariate time series data through the actual cleaning operator.

[0046] As shown in Figure 2, this system consists of three main modules: a data preprocessing module, a quality detection module, and an automatic cleaning module. The data preprocessing module and the quality detection module together constitute the data quality detection module. The automatic cleaning module consists of an agent configuration module, a hierarchical agent construction module, a global cleaning strategy decision module, and a local cleaning execution module. The upper-level agent specifically refers to the global quality control agent, and the lower-level agent specifically refers to the local cleaning agent.

[0047] The specific embodiments described above are preferred embodiments of the clinical time-series data cleaning method and system based on hierarchical agent reinforcement learning of the present invention, and are not intended to limit the specific scope of the present invention. The scope of the present invention includes but is not limited to the specific embodiments described above. All equivalent changes made in accordance with the shape and structure of the present invention are within the protection scope of the present invention.

Claims

1. A clinical time-series data cleaning method based on hierarchical agent reinforcement learning, characterized in that, Includes the following steps: The system performs missing data, anomaly detection, and constraint violation detection on real-time multivariate time-series data to obtain the missing data rate, anomaly rate, and constraint violation rate. Using the processing priority and repair type of missing data, anomalies, and constraint violations as the first action space, a first reward function is constructed based on the overall data quality indicator improvement. Using the cleaning operator as the second action space, a second reward function is constructed based on the improvement of individual data quality indicators. A global quality control agent is constructed based on the first action space and the first reward function, and a local cleaning agent is constructed based on the second action space and the second reward function. The missing data rate, anomaly rate, and constraint violation rate are input into the global quality control agent to obtain the actual processing priority and actual repair type of missing data, anomalies, and constraint violations. Based on the actual processing priority of missing data, anomalies, and constraint violations, the actual repair type is sequentially input into the local cleaning agent to obtain the actual cleaning operator, which is then used to clean the real-time multivariate time-series data.

2. The clinical time-series data cleaning method based on hierarchical agent reinforcement learning according to claim 1, characterized in that, The process of detecting missing, anomalies, and constraints in real-time multivariate time-series data to obtain missing data rate, anomaly rate, and constraint violation rate includes: performing time alignment processing on real-time multivariate time-series data; performing missing data detection on real-time multivariate time-series data to obtain missing data rate; extracting features from real-time multivariate time-series data to obtain data features; selecting an anomaly detection model based on the data features; performing anomaly detection on real-time multivariate time-series data using the anomaly detection model to obtain anomaly rate; acquiring historical multivariate time-series data; performing constraint mining on historical multivariate time-series data to obtain constraint set; and obtaining constraint violation rate based on constraint set; wherein, continuous redundant data in the real-time multivariate time-series data are marked as missing data.

3. The clinical time-series data cleaning method based on hierarchical agent reinforcement learning according to claim 2, characterized in that, The process of extracting features from real-time multivariate time-series data, selecting an anomaly detection model based on these features, and then using the anomaly detection model to detect anomalies in the real-time multivariate time-series data to obtain the data anomaly rate includes: extracting conventional features from the real-time multivariate time-series data to obtain conventional data features; comparing the real-time multivariate time-series data with the normal data range to obtain anomaly detection features; inputting the real-time multivariate time-series data into a candidate anomaly detection model to obtain basic model features; using meta-learning to select an anomaly detection model based on the conventional data features, anomaly detection features, and basic model features; inputting the real-time multivariate time-series data into the anomaly detection model to obtain the average anomaly score for each data point in the real-time multivariate time-series data; and determining data with an average anomaly score greater than a preset threshold as anomaly data, thereby obtaining the data anomaly rate.

4. The clinical time-series data cleaning method based on hierarchical agent reinforcement learning according to claim 2, characterized in that, The process of obtaining a constraint set through constraint mining of historical multivariate time series data, and then obtaining the constraint violation rate based on the constraint set, includes: using multinomial features and OLS regression to mine the functional relationships between variables in the historical multivariate time series data to obtain inter-variable functional constraints; obtaining intra-variable temporal change constraints based on the historical time series data of the same variable in the historical multivariate time series data; obtaining inter-variable temporal monotonic constraints based on the monotonic influence relationships between variables in the historical multivariate time series data; obtaining change intervals based on the historical time series data of the same variable in the historical multivariate time series data, combining the change intervals of different variables in the same time period to obtain candidate negation intervals, and obtaining inter-variable negation constraints based on the candidate negation intervals; and obtaining violation data in the real-time multivariate time series data based on inter-variable functional constraints, intra-variable temporal change constraints, inter-variable temporal monotonic constraints, and inter-variable negation constraints, thereby obtaining the constraint violation rate.

5. The clinical time-series data cleaning method based on hierarchical agent reinforcement learning according to claim 1, characterized in that, The step of constructing the first reward function based on the improvement of the overall data quality index includes: weighting and summing the improvement of the overall data quality index, the time cost penalty during the cleaning process, and the invalid action penalty to construct the first reward function.

6. The clinical time-series data cleaning method based on hierarchical agent reinforcement learning according to claim 5, characterized in that, The construction of the second reward function based on the improvement of a single data quality indicator includes: weighting and summing the improvement of the single data quality indicator, the smoothness of the data structure, and the data disturbance indicator to construct the second reward function.

7. The clinical time-series data cleaning method based on hierarchical agent reinforcement learning according to claim 1 or 6, characterized in that, The construction of a global quality control agent based on a first action space and a first reward function, and a local cleaning agent based on a second action space and a second reward function, includes: constructing an initial global quality control agent based on the first action space and the first reward function, and an initial local cleaning agent based on the second action space and the second reward function; inputting the historical data missing rate, historical data anomaly rate, and historical constraint violation rate of historical multivariate time series data into the initial global quality control agent to obtain the historical processing priorities and historical repair types for data missing, data anomalies, and constraint violations; inputting the historical repair types into the initial local cleaning agent to obtain historical cleaning operators; cleaning the historical multivariate time series data sequentially through the historical cleaning operators according to the historical processing priorities for data missing, data anomalies, and constraint violations to obtain cleaned historical multivariate time series data, and obtaining reference index values ​​for the cleaned historical multivariate time series data; updating the state-action value functions in the initial global quality control agent and the initial local cleaning agent based on the reference index values ​​to obtain the global quality control agent and the local cleaning agent.

8. The clinical time-series data cleaning method based on hierarchical agent reinforcement learning according to claim 7, characterized in that, The process of obtaining cleaned historical multivariate time-series data by sequentially cleaning historical multivariate time-series data using historical cleaning operators based on the historical processing priorities of missing data, data anomalies, and constraint violations includes: obtaining first cleaned historical multivariate time-series data by cleaning historical multivariate time-series data using historical cleaning operators based on the historical processing priorities of missing data, data anomalies, and constraint violations; obtaining second cleaned historical multivariate time-series data by cleaning first cleaned historical multivariate time-series data using historical cleaning operators based on the historical processing priorities of missing data, data anomalies, and constraint violations; and obtaining third cleaned historical multivariate time-series data by cleaning second cleaned historical multivariate time-series data using historical cleaning operators based on the historical processing priorities of missing data, data anomalies, and constraint violations.

9. The clinical time-series data cleaning method based on hierarchical agent reinforcement learning according to claim 8, characterized in that, The step of updating the state-action value function of the initial global quality control agent and the initial local cleaning agent based on reference index values ​​to obtain the global quality control agent and the local cleaning agent includes: obtaining a first reward value and a second reward value based on reference index values ​​of the third cleaning historical multivariate time series data; updating the state-action value function of the initial global quality control agent based on the first reward value to obtain the global quality control agent; obtaining a third reward value based on reference index values ​​of the first cleaning historical multivariate time series data; obtaining a fourth reward value based on reference index values ​​of the second cleaning historical multivariate time series data; and updating the state-action value function of the initial local cleaning agent based on the second reward value, the third reward value, and the fourth reward value to obtain the local cleaning agent.

10. A clinical time-series data cleaning system based on hierarchical agent reinforcement learning, applicable to the clinical time-series data cleaning method based on hierarchical agent reinforcement learning as described in any one of claims 1-9, characterized in that, include: The data quality inspection module is used to detect missing, abnormal, and default data in real-time multivariate time series data to obtain the data missing rate, data abnormality rate, and constraint violation rate. The intelligent agent configuration module is used to construct a first reward function based on the processing priority and repair type of data missing, data anomaly and constraint violation as the first action space, and the improvement degree of the overall data quality index as the second action space, and to construct a second reward function based on the improvement degree of individual data quality index as the cleaning operator. The hierarchical intelligent agent construction module is used to construct a global quality control intelligent agent based on a first action space and a first reward function, and a local cleaning intelligent agent based on a second action space and a second reward function. The global cleaning strategy decision module is used to input the data missing rate, data anomaly rate, and constraint violation rate into the global quality control intelligent agent to obtain the actual processing priority and actual repair type of data missing, data anomaly, and constraint violation. The local cleaning execution module is used to input the actual repair type into the local cleaning intelligent agent according to the actual processing priority of data missing, data anomaly, and constraint violation to obtain the actual cleaning operator, and clean the real-time multivariate time series data through the actual cleaning operator.

Images