A Method and System for Cleaning Clinical Time-Series Data Based on Hierarchical Agent Reinforcement Learning

Abstract

This invention discloses a method and system for cleaning clinical time-series data based on hierarchical agent reinforcement learning, belonging to the field of medical data processing technology. The method includes: detecting and obtaining data missing rate, data anomaly rate, and constraint violation rate of real-time multivariate time-series data; constructing a first reward function based on the processing priority and repair type of data missing, data anomaly, and constraint violation as the first action space, and constructing a second reward function based on the improvement degree of the overall data quality index; constructing a global quality control agent and a local cleaning agent based on the cleaning operator as the second action space, and the improvement degree of individual data quality indicators; and cleaning the real-time multivariate time-series data using the global quality control agent and the local cleaning agent. This invention solves the technical problem that existing technologies struggle to improve data cleaning quality and efficiency.

Description

Technical Field

[0001] This invention relates to the field of medical data processing technology, specifically to a method and system for cleaning clinical time-series data based on hierarchical agent reinforcement learning. Background Technology

[0002] With the rapid development of information-based healthcare and intelligent diagnostic systems, a large amount of multivariate time-series data is generated in obstetrics and gynecology clinical practice, such as pregnant women's vital signs, fetal heart rate monitoring signals, uterine contraction curves, biochemical test indicators, and medication records. This type of data is of significant value in assisting clinical decision-making, disease risk warning, and the construction of intelligent diagnostic models. However, due to the complex clinical data collection environment, strong equipment heterogeneity, and frequent human intervention, the actual multivariate time-series data generally suffers from various quality problems such as missing values, outliers, and violated constraints, severely restricting the reliable use of the data. Obstetrics and gynecology clinical multivariate time-series data not only exhibits significant dynamic evolution characteristics in the time dimension, but also has strong constraints between different variables. For example, there is a clear physiological correlation between fetal heart rate, uterine contraction intensity, and gestational age; unreasonable data repair may disrupt the original medical consistency. However, most existing data cleaning methods often treat missing value imputation, outlier detection, and consistency repair as independent processing steps, adopting a serial pipeline mode of "detect first, repair later." This fragmented approach ignores the interplay between various quality defects, easily introducing new anomalies or violating logical constraints while fixing one problem, making it difficult to improve the overall quality of clinical data. Furthermore, in multivariate time-series scenarios in obstetrics and gynecology, the decision space for cleaning strategies is highly complex. With the increase in the number of variables, types of cleaning operators, and parameter configuration dimensions, the combined decisions regarding "cleaning operator selection, variable location, and parameter scheduling" grow exponentially. Traditional heuristic methods or single reinforcement learning models are prone to the curse of dimensionality in such high-dimensional, multi-layered decision-making problems, making it difficult to find stable and effective cleaning strategies within an acceptable timeframe, thus limiting their feasibility in real-world clinical settings. Therefore, improving data cleaning quality and efficiency remains a technical challenge that current technologies struggle to address. Summary of the Invention

[0003] To address the technical problem that existing technologies struggle to improve the quality and efficiency of data cleaning, this invention provides a clinical time-series data cleaning method and system based on hierarchical agent reinforcement learning. By constructing a global-local hierarchical agent collaborative cleaning architecture and combining precise design of multi-dimensional action space and reward function, it achieves efficient and high-quality cleaning of real-time multivariate time-series data. This effectively solves the technical problems in existing technologies, such as fragmented processing ignoring the mutual influence between various quality defects and the dimensionality curse caused by high-dimensional decision space, which makes it difficult to improve the quality and efficiency of data cleaning.

[0004] To address the aforementioned technical problems, this invention provides a clinical time-series data cleaning method based on hierarchical agent reinforcement learning, comprising the following steps: The missing data rate, anomaly rate, and constraint violation rate are obtained by performing missing, anomaly, and default detection on real-time multivariate time series data. The first action space is defined by the processing priority and repair type of data missing, data anomaly, and constraint violation. The first reward function is constructed by the improvement of the overall data quality index. The second action space is defined by the cleaning operator. The second reward function is constructed by the improvement of individual data quality index. A global quality control agent is constructed based on the first action space and the first reward function, and a local cleaning agent is constructed based on the second action space and the second reward function. Input the data missing rate, data anomaly rate, and constraint violation rate into the global quality control agent to obtain the actual processing priority and actual repair type of data missing, data anomaly, and constraint violation; Based on the actual processing priority of missing data, data anomalies, and constraint violations, the actual repair type is input into the local cleaning agent to obtain the actual cleaning operator, and the real-time multivariate time series data is cleaned through the actual cleaning operator.

[0005] Preferably, the step of performing missing, anomaly, and default detection on real-time multivariate time-series data to obtain data missing rate, data anomaly rate, and constraint violation rate includes: Time alignment is performed on real-time multivariate time series data, and missing data detection is performed on real-time multivariate time series data to obtain the missing data rate; Feature extraction is performed on real-time multivariate time series data to obtain data features. An anomaly detection model is selected based on the data features. Anomaly detection model is used to detect anomalies in real-time multivariate time series data to obtain the data anomaly rate. Obtain historical multivariate time series data, perform constraint mining on the historical multivariate time series data to obtain a constraint set, and obtain the constraint violation rate based on the constraint set; In this process, continuous redundant data in real-time multivariate time series data are marked as missing data.

[0006] Preferably, the step of extracting features from real-time multivariate time-series data to obtain data features, selecting an anomaly detection model based on the data features, and using the anomaly detection model to perform anomaly detection on the real-time multivariate time-series data to obtain the data anomaly rate includes: Perform routine feature extraction on real-time multivariate time series data to obtain routine data features; Real-time multivariate time series data is compared with normal data intervals to obtain outlier detection features; Real-time multivariate time series data is input into the candidate anomaly detection model to obtain basic model features; Based on conventional data features, outlier detection features, and basic model features, meta-learning is used to select an anomaly detection model. Real-time multivariate time series data is input into the anomaly detection model to obtain the average anomaly score of each data point in the real-time multivariate time series data. Data with an average anomaly score greater than a preset threshold are judged as anomaly data, thereby obtaining the data anomaly rate.

[0007] Preferably, the step of obtaining a constraint set by performing constraint mining on historical multivariate time series data, and obtaining the constraint violation rate based on the constraint set, includes: Use multinomial features and OLS regression to mine the functional relationships between variables in historical multivariate time series data and obtain functional constraints between variables; Obtain constraints on intra-variable temporal changes based on historical time series data of the same variable in historical multivariate time series data; Obtain temporal monotonic constraints between variables based on the monotonic influence relationships among variables in historical multivariate time series data; The variation range of the same variable is obtained from the historical time series data of the same variable in the historical multivariate time series data. The variation ranges of different variables in the same time period are combined to obtain the candidate negation range. Negation constraints between variables are obtained based on the candidate negation range. Based on inter-variable functional constraints, intra-variable temporal change constraints, inter-variable temporal monotonic constraints, and inter-variable negation constraints, violation data in real-time multivariable time series data is obtained, thereby obtaining the constraint violation rate.

[0008] Preferably, the construction of the first reward function based on the improvement of the overall data quality index includes: The first reward function is constructed by weighting and summing the overall data quality improvement, the time cost penalty during the cleaning process, and the penalty for invalid actions.

[0009] Preferably, the construction of the second reward function based on the improvement of a single data quality indicator includes: The second reward function is constructed by weighting and summing the improvement of individual data quality indicators, the smoothness of data structure, and the data disturbance indicators.

[0010] Preferably, the construction of a global quality control agent based on a first action space and a first reward function, and the construction of a local cleaning agent based on a second action space and a second reward function, includes: An initial global quality control agent is constructed based on the first action space and the first reward function, and an initial local cleaning agent is constructed based on the second action space and the second reward function. Input the historical data missing rate, historical data anomaly rate and historical constraint violation rate of historical multivariate time series data into the initial global quality control agent to obtain the historical processing priority and historical repair type of data missing, data anomaly and constraint violation; Input the historical repair type into the initial local cleaning agent to obtain the historical cleaning operator. Based on the historical processing priority of missing data, abnormal data, and constraint violation, the historical multivariate time series data is cleaned sequentially through the historical cleaning operator to obtain the cleaned historical multivariate time series data, and the reference index value of the cleaned historical multivariate time series data is obtained. Based on the reference index values, the state-action value functions in the initial global quality control agent and the initial local cleaning agent are updated to obtain the global quality control agent and the local cleaning agent.

[0011] Preferably, the step of sequentially cleaning historical multivariate time-series data by using historical cleaning operators to obtain cleaned historical multivariate time-series data based on the historical processing priority of missing data, data anomalies, and constraint violations includes: Based on the historical processing priorities of missing data, data anomalies, and constraint violations, historical cleaning operators are used to clean historical multivariate time series data to obtain the first cleaned historical multivariate time series data. Based on the historical processing priorities of missing data, data anomalies, and constraint violations, the first cleaned historical multivariate time series data is cleaned using historical cleaning operators to obtain the second cleaned historical multivariate time series data. Based on the historical processing priorities of missing data, data anomalies, and constraint violations, the second-cleaned historical multivariate time series data is cleaned using historical cleaning operators to obtain the third-cleaned historical multivariate time series data.

[0012] Preferably, the step of updating the state-action value functions of the initial global quality control agent and the initial local cleaning agent based on reference index values ​​to obtain the global quality control agent and the local cleaning agent includes: The first reward value and the second reward value are obtained based on the reference index value of the third cleaning historical multivariate time series data. The state-action value function in the initial global quality control agent is updated based on the first reward value to obtain the global quality control agent. The third reward value is obtained based on the reference index value of the first cleaning historical multivariate time series data, and the fourth reward value is obtained based on the reference index value of the second cleaning historical multivariate time series data. The state-action value function in the initial local cleaning agent is updated based on the second, third, and fourth reward values ​​to obtain the local cleaning agent.

[0013] By adopting the above technical solution, the present invention has the following advantages: By constructing a global-local hierarchical intelligent agent collaborative cleaning architecture, and combining the precise design of multi-dimensional action space and reward function, we have achieved efficient and high-quality cleaning of real-time multivariate time series data. This effectively solves the technical problems in existing technologies, such as the fragmented processing ignoring the mutual influence between various quality defects and the dimensionality disaster caused by high-dimensional decision space, which makes it difficult to improve the quality and efficiency of data cleaning. The cleaning decision is broken down into two low-dimensional action spaces: a global strategy layer (processing priority and repair type) and a local execution layer (cleaning operators). This architecturally solves the dimensionality curse faced by traditional single models. The first reward function incorporates the improvement of overall data quality indicators, time cost penalties, and penalties for ineffective actions. The second reward function incorporates the improvement of individual data quality indicators, data structure smoothness, and data perturbation indicators. This allows the global quality control agent to focus on overall quality and efficiency, and the local cleaning agent to focus on precise repair and data pattern preservation. This avoids over-cleaning and pattern destruction guided by a single quality indicator, achieving a triple improvement in cleaning quality, cleaning efficiency, and data pattern preservation, thereby improving the feasibility of the cleaned data. Attached Figure Description

[0014] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings. The drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings.

[0015] Figure 1 This is a flowchart illustrating the clinical time-series data cleaning method based on hierarchical agent reinforcement learning according to the present invention. Figure 2 This is a diagram illustrating the overall framework of the clinical time-series data cleaning system based on hierarchical agent reinforcement learning, as described in this invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only one preferred embodiment of this invention and are only used to explain this invention. They do not limit the scope of protection of this invention. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0017] Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations (or steps) can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but it may also have additional steps not included in the figures; the process may correspond to a method, function, procedure, subroutine, subroutine, etc.

[0018] Example 1: As Figure 1 As shown, the clinical time-series data cleaning method based on hierarchical agent reinforcement learning includes the following steps: S1: Perform missing, anomaly, and default detection on real-time multivariate time series data to obtain the missing data rate, anomaly rate, and constraint violation rate.

[0019] As an optional embodiment, the step of detecting missing data, anomalies, and defaults in real-time multivariate time-series data to obtain data missing rate, data anomaly rate, and constraint violation rate includes: Time alignment is performed on real-time multivariate time series data, and missing data detection is performed on real-time multivariate time series data to obtain the missing data rate; Feature extraction is performed on real-time multivariate time series data to obtain data features. An anomaly detection model is selected based on the data features. Anomaly detection model is used to detect anomalies in real-time multivariate time series data to obtain the data anomaly rate. Obtain historical multivariate time series data, perform constraint mining on the historical multivariate time series data to obtain a constraint set, and obtain the constraint violation rate based on the constraint set; In this process, continuous redundant data in real-time multivariate time series data are marked as missing data.

[0020] Understandably, real-time multivariate time-series data refers to time-series data containing multiple clinical indicators collected under a unified time axis. Real-time multivariate time-series data can originate from hospital information systems, monitoring equipment, or clinical databases. Addressing issues such as timestamp chaos, uneven intervals, duplication, and jumps caused by heterogeneous data acquisition from multiple clinical devices and human intervention, time alignment processing ensures that timestamps meet the constraints of equal intervals, non-repetition, and monotonically increasing. This fundamentally solves the problem of time dimension disorder in real-time multivariate time-series data, guaranteeing the continuity and standardization of the time dimension within the same time-series data, and achieving time dimension synchronization between multiple variables. This avoids deviations in subsequent feature extraction, constraint mining, and operator execution caused by timestamp chaos, while providing a standardized time-series data foundation for subsequent anomaly detection and constraint violation detection, thus improving the reliability of detection results.

[0021] Understandably, in the process of obtaining the missing data rate by performing missing data detection on real-time multivariate time series data, it is necessary not only to mark the originally missing data as missing data, but also to mark the continuous redundant data in the real-time multivariate time series data as missing data. Continuous redundant data specifically refers to continuous data that has no clinical significance and is merely mechanically repeated. Taking fetal heart rate monitoring as an example, if due to contact failure of the acquisition equipment or data transmission delay, the fetal heart rate data collected for a certain period of time (such as 30 consecutive seconds) is always fixed at 140 beats / min, and the fetal heart rate in the first 10 seconds and the last 10 seconds of this 30 seconds is also 140 beats / min, forming continuous redundant data with the adjacent historical sequence, considering that the physiological heart rate of the fetus cannot be completely without fluctuation within 30 seconds, this continuous fixed data is not the true heart rate state of the fetus, but an invalid redundant value collected by the equipment, which has no clinical reference value. Therefore, the data acquired within this 30 seconds is marked as missing data. By truly reflecting the degree of missing effective information in clinical data, the authenticity and accuracy of the data missing rate statistics are improved.

[0022] Specifically, the step of extracting features from real-time multivariate time-series data to obtain data features, selecting an anomaly detection model based on the data features, and using the anomaly detection model to detect anomalies in the real-time multivariate time-series data to obtain the data anomaly rate includes: Perform routine feature extraction on real-time multivariate time series data to obtain routine data features; Real-time multivariate time series data is compared with normal data intervals to obtain outlier detection features; Real-time multivariate time series data is input into the candidate anomaly detection model to obtain basic model features; Based on conventional data features, outlier detection features, and basic model features, meta-learning is used to select an anomaly detection model. Real-time multivariate time series data is input into the anomaly detection model to obtain the average anomaly score of each data point in the real-time multivariate time series data. Data with an average anomaly score greater than a preset threshold are judged as anomaly data, thereby obtaining the data anomaly rate.

[0023] Understandably, conventional data features include statistical features (such as mean, variance, quantiles, etc.), distribution features, correlation features (correlation, covariance), and structural features (sparseness, entropy). Statistical features, including mean, variance, and quantiles, are used to describe the overall fluctuations and central tendency of real-time multivariate time series data. Distribution features describe the shape of the numerical distribution, reflecting whether the real-time multivariate time series data conforms to a classical probability distribution, the symmetry / skewness of the distribution, etc., and are key criteria for selecting anomaly detection models. Correlation features describe the interdependence and mutual influence among multiple variables. Structural features describe the overall data structure, time series organization, and effective information density of real-time multivariate time series data, reflecting the spatiotemporal distribution patterns, completeness, and sparsity of the real-time multivariate time series data. Outlier detection features represent the proportion, location, and other characteristics of suspected anomalies in real-time multivariate time series data. Candidate anomaly detection models are specifically unsupervised anomaly detection models, such as HBOS, Isolation Forest, PCA, LODA, etc. By inputting real-time multivariate time-series data into candidate anomaly detection models to extract basic model features (such as the model's anomaly score distribution and detection accuracy), the initial adaptability of different models to real-time multivariate time-series data is directly reflected. By combining regular data features, anomaly detection features, and basic model features into a fixed-size vector, and then using meta-learning to recommend the K most suitable unsupervised anomaly detection models for the real-time multivariate time-series data, these K unsupervised anomaly detection models are used to perform anomaly detection operations on the real-time multivariate time-series data to obtain corresponding anomaly scores. Subsequently, the average anomaly score for each data point is calculated and compared with a preset threshold. If the average anomaly score exceeds the preset threshold, the data is determined to be anomalous. Users can flexibly set the preset threshold according to their actual needs. In this embodiment, combining multi-dimensional indicators to select anomaly detection models significantly improves the adaptability of anomaly detection models to real-time multivariate time-series data, thereby improving the accuracy of the data anomaly rate.

[0024] Specifically, the step of performing constraint mining on historical multivariate time-series data to obtain a constraint set, and obtaining the constraint violation rate based on the constraint set, includes: Use multinomial features and OLS regression to mine the functional relationships between variables in historical multivariate time series data and obtain functional constraints between variables; Obtain constraints on intra-variable temporal changes based on historical time series data of the same variable in historical multivariate time series data; Obtain temporal monotonic constraints between variables based on the monotonic influence relationships among variables in historical multivariate time series data; The variation range of the same variable is obtained from the historical time series data of the same variable in the historical multivariate time series data. The variation ranges of different variables in the same time period are combined to obtain the candidate negation range. Negation constraints between variables are obtained based on the candidate negation range. Based on inter-variable functional constraints, intra-variable temporal change constraints, inter-variable temporal monotonic constraints, and inter-variable negation constraints, violation data in real-time multivariable time series data is obtained, thereby obtaining the constraint violation rate.

[0025] Multinomial features: These feature processing methods extend and cross-combine historical multivariate time-series data using power functions, transforming linearly inseparable original features into high-dimensional features capable of fitting nonlinear relationships. This is a preliminary feature processing method for fitting complex functional relationships between variables in regression analysis. OLS regression: The most commonly used linear regression method, it solves for the coefficients of the regression model by minimizing the sum of squared errors between predicted and actual values, ultimately obtaining the functional relationship between variables. Understandably, intra-variable temporal change constraints specifically refer to the rate of change, acceleration, and fluctuation amplitude of a single variable over time, such as "heart rate changes no more than 10 times per minute." Inter-variable temporal monotonic constraints specifically refer to the monotonically positive / negative correlation between multiple variables over time, such as "when body temperature rises, the number of white blood cells also increases monotonically." Inter-variable negation constraints specifically refer to the constraint rules corresponding to the combined values ​​of multiple variables that do not exist in normal data, such as "when blood glucose is <3.9 mmol / L, the insulin injection dose cannot be >20 units." It is important to emphasize that after obtaining candidate negation intervals, each candidate negation interval needs to be verified. Inter-variable negation constraints are obtained based on candidate negation intervals that appear less than or equal to a preset number of times in historical multivariate time-series data. The preset number of times is flexibly set according to the scale of historical multivariate time-series data. In this embodiment, by conducting multi-dimensional mining of data constraints, the completeness and comprehensiveness of constraint mining are effectively improved, thereby significantly enhancing the scenario adaptability and practical application value of the constraint violation rate calculation results.

[0026] S2: The first action space is based on the processing priority and repair type of data missing, data anomaly and constraint violation. The first reward function is constructed by the improvement of the overall data quality index. The second action space is based on the cleaning operator. The second reward function is constructed by the improvement of individual data quality index.

[0027] S3: Construct a global quality control agent based on the first action space and the first reward function, and construct a local cleaning agent based on the second action space and the second reward function.

[0028] Understandably, the first reward function serves as the basis for the global quality control agent's decision-making reward and punishment. It is constructed with the overall data quality index improvement as its core and is a quantitative standard for measuring the quality of the global quality control agent's decisions. The reward and punishment logic is as follows: the higher the overall data quality index improvement, the greater the reward value of the first reward function, and vice versa. The purpose is to guide the global quality control agent to learn and select the cleaning priority and repair type that can optimize the overall data quality, avoiding the upper-level decision-making from focusing only on the repair of a single problem and ignoring the overall optimization of the quality of the entire data. This avoids the technical problem in existing technologies where fragmented processing ignores the mutual influence between multiple quality defects, making it difficult to improve the quality of data cleaning. The second reward function serves as the basis for rewarding and penalizing the local cleaning agent. It is constructed with the improvement of a single data quality indicator as its core and is a quantitative standard for measuring the quality of the cleaning operator selected by the local cleaning agent. The reward and penalty logic is as follows: the higher the improvement of a single data quality indicator, the greater the reward value of the second reward function, and vice versa. The purpose is to guide the local cleaning agent to learn and match the optimal cleaning operator for each type of repair, thereby maximizing the cleaning effect under a single repair type.

[0029] By constructing a global-local hierarchical intelligent agent collaborative cleaning architecture, the high-dimensional cleaning task is decoupled into "priority decision-making" and "operator scheduling", which effectively reduces the complexity of the cleaning strategy decision space and improves the efficiency and stability of cleaning strategy search in multivariate scenarios.

[0030] As an optional embodiment, the construction of a global quality control agent based on a first action space and a first reward function, and the construction of a local cleaning agent based on a second action space and a second reward function, includes: An initial global quality control agent is constructed based on the first action space and the first reward function, and an initial local cleaning agent is constructed based on the second action space and the second reward function. Input the historical data missing rate, historical data anomaly rate and historical constraint violation rate of historical multivariate time series data into the initial global quality control agent to obtain the historical processing priority and historical repair type of data missing, data anomaly and constraint violation; Input the historical repair type into the initial local cleaning agent to obtain the historical cleaning operator. Based on the historical processing priority of missing data, abnormal data, and constraint violation, the historical multivariate time series data is cleaned sequentially through the historical cleaning operator to obtain the cleaned historical multivariate time series data, and the reference index value of the cleaned historical multivariate time series data is obtained. Based on the reference index values, the state-action value functions in the initial global quality control agent and the initial local cleaning agent are updated to obtain the global quality control agent and the local cleaning agent.

[0031] In this embodiment, the historical repair types include missing value repair, outlier value repair, and constraint violation repair.

[0032] Specifically, based on the historical processing priorities of missing data, data anomalies, and constraint violations, historical cleaning operators are used sequentially to clean historical multivariate time-series data to obtain cleaned historical multivariate time-series data, including: Based on the historical processing priorities of missing data, data anomalies, and constraint violations, historical cleaning operators are used to clean historical multivariate time series data to obtain the first cleaned historical multivariate time series data. Based on the historical processing priorities of missing data, data anomalies, and constraint violations, the first cleaned historical multivariate time series data is cleaned using historical cleaning operators to obtain the second cleaned historical multivariate time series data. Based on the historical processing priorities of missing data, data anomalies, and constraint violations, the second-cleaned historical multivariate time series data is cleaned using historical cleaning operators to obtain the third-cleaned historical multivariate time series data.

[0033] Taking the historical processing priority of missing data, data anomalies, and constraint violations as data missing > data anomalies > constraint violations as an example, the historical multivariate time series data is first cleaned according to the historical cleaning operator corresponding to missing data to obtain the first cleaned historical multivariate time series data. Then, the first cleaned historical multivariate time series data is cleaned according to the historical cleaning operator corresponding to data anomalies to obtain the second cleaned historical multivariate time series data. Finally, the second cleaned historical multivariate time series data is cleaned according to the historical cleaning operator corresponding to constraint violations to obtain the third cleaned historical multivariate time series data.

[0034] Specifically, the step of updating the state-action value functions of the initial global quality control agent and the initial local cleaning agent based on reference index values ​​to obtain the global quality control agent and the local cleaning agent includes: The first reward value and the second reward value are obtained based on the reference index value of the third cleaning historical multivariate time series data. The state-action value function in the initial global quality control agent is updated based on the first reward value to obtain the global quality control agent. The third reward value is obtained based on the reference index value of the first cleaning historical multivariate time series data, and the fourth reward value is obtained based on the reference index value of the second cleaning historical multivariate time series data. The state-action value function in the initial local cleaning agent is updated based on the second, third, and fourth reward values ​​to obtain the local cleaning agent.

[0035] In this embodiment, obtaining the first reward value and the second reward value based on the reference index values ​​of the third-cleaned historical multivariate time series data specifically refers to: obtaining the overall data quality index improvement degree based on multiple reference index values ​​in the reference index values ​​of the third-cleaned historical multivariate time series data and the historical data missing rate, historical data anomaly rate, and historical constraint violation rate of the historical multivariate time series data, and inputting the overall data quality index improvement degree into the first reward function to obtain the first reward value; obtaining the first single data quality index improvement degree based on the single reference index value in the reference index values ​​of the third-cleaned historical multivariate time series data and the historical data missing rate, historical data anomaly rate, or historical constraint violation rate of the historical multivariate time series data, and inputting the first single data quality index improvement degree into the second reward function to obtain the second reward value; wherein, the multiple reference index values ​​include the historical data missing rate, historical data anomaly rate, and historical constraint violation rate of the third-cleaned historical multivariate time series data. If the historical processing priority for missing data, data anomalies, and constraint violations is missing data > data anomalies > constraint violations, then the single reference indicator value is the historical constraint violation rate of the third-cleaned historical multivariate time series data. Specifically, it refers to obtaining the improvement degree of the first single data quality indicator based on the historical constraint violation rate of the third-cleaned historical multivariate time series data and the historical constraint violation rate of the historical multivariate time series data, and inputting the improvement degree of the first single data quality indicator into the second reward function to obtain the second reward value. At this point, the reference indicator value for the first cleaning of historical multivariate time series data specifically refers to the historical data missing rate of the first cleaning of historical multivariate time series data. Based on the historical data missing rate of the first cleaning of historical multivariate time series data and the historical data missing rate of historical multivariate time series data, the improvement degree of the second single data quality indicator is obtained. The improvement degree of the second single data quality indicator is input into the second reward function to obtain the third reward value. The reference indicator value for the second cleaning of historical multivariate time series data specifically refers to the historical data anomaly rate of the second cleaning of historical multivariate time series data. Based on the historical data anomaly rate of the second cleaning of historical multivariate time series data and the historical data anomaly rate of historical multivariate time series data, the improvement degree of the third single data quality indicator is obtained. The improvement degree of the third single data quality indicator is input into the second reward function to obtain the fourth reward value.

[0036] In this embodiment, when the updated global quality control agent outputs a post-cleaning reference index value corresponding to the processing priority and repair type for the same batch of historical multivariate time-series data, and the fluctuation range is ≤ a preset value (e.g., ±1%), it indicates that the state-action value function in the initial global quality control agent has been updated. Similarly, when the updated local cleaning agent outputs a post-cleaning reference index value corresponding to the cleaning operator for the same batch of historical multivariate time-series data, and the improvement variance under different batches of historical multivariate time-series data tests is ≤ a preset value (e.g., 0.01), it indicates that the state-action value function in the initial local cleaning agent has been successfully updated.

[0037] S4: Input the data missing rate, data anomaly rate, and constraint violation rate into the global quality control agent to obtain the actual processing priority and actual repair type of data missing, data anomaly, and constraint violation.

[0038] S5: Based on the actual processing priority of missing data, data anomalies, and constraint violations, the actual repair type is input into the local cleaning agent to obtain the actual cleaning operator, and the real-time multivariate time series data is cleaned through the actual cleaning operator.

[0039] In this embodiment, by constructing a global-local hierarchical intelligent agent collaborative cleaning architecture and combining a precise design of a multi-dimensional action space and reward function, efficient and high-quality cleaning of real-time multivariate time-series data is achieved. This effectively solves the technical problems in the prior art where fragmented processing ignores the mutual influence between various quality defects and the dimensionality curse caused by high-dimensional decision space, making it difficult to improve the quality and efficiency of data cleaning.

[0040] Example 2: The difference from Example 1 is that: The construction of the first reward function based on the improvement of overall data quality indicators includes: The first reward function is constructed by weighting and summing the overall data quality improvement, the time cost penalty during the cleaning process, and the penalty for invalid actions.

[0041] The construction of the second reward function based on the improvement of individual data quality indicators includes: The second reward function is constructed by weighting and summing the improvement of individual data quality indicators, the smoothness of data structure, and the data disturbance indicators.

[0042] In this embodiment, the first reward function incorporates the improvement of overall data quality indicators, time cost penalties, and invalid action penalties, while the second reward function incorporates the improvement of individual data quality indicators, data structure smoothness, and data disturbance indicators. This allows the global quality control agent to focus on overall quality and efficiency, and the local cleaning agent to focus on precise repair and data pattern preservation. This avoids over-cleaning and pattern destruction guided by a single quality indicator, achieving a triple improvement in cleaning quality, cleaning efficiency, and data pattern preservation, thereby enhancing the feasibility of the cleaned data.

[0043] In this embodiment, obtaining the first reward value and the second reward value based on the reference index values ​​of the third historical multivariate time series data specifically refers to: obtaining the overall data quality index improvement degree based on multiple reference index values ​​in the reference index values ​​of the third historical multivariate time series data and the historical data missing rate, historical data anomaly rate, and historical constraint violation rate of the historical multivariate time series data; obtaining the time cost penalty in the cleaning process based on the acquisition time in the reference index values ​​of the third historical multivariate time series data; obtaining the invalid action penalty based on the number of invalid actions in the reference index values ​​of the third historical multivariate time series data; inputting the overall data quality index improvement degree, time cost penalty, and invalid action penalty into the first reward function to obtain the first reward value; and obtaining the first reward value based on the reference index values ​​of the third historical multivariate time series data. The improvement degree of the first single data quality indicator is obtained by comparing the individual reference indicator value in the evaluation index with the historical data missing rate, historical data anomaly rate, or historical constraint violation rate of the historical multivariate time series data. The first data structure smoothness is obtained based on the time series data fluctuation characteristics and trend characteristics in the reference indicator values ​​of the third-cleaned historical multivariate time series data. The first data disturbance indicator is obtained based on the time series data fluctuation amplitude, mutation frequency, and dispersion in the reference indicator values ​​of the third-cleaned historical multivariate time series data. The improvement degree of the first single data quality indicator, the first data structure smoothness, and the first data disturbance indicator are input into the second reward function to obtain the second reward value. The multiple reference indicator values ​​include the historical data missing rate, historical data anomaly rate, and historical constraint violation rate of the third-cleaned historical multivariate time series data. Acquisition time is the total time taken from acquiring the historical multivariate time series data to acquiring the third-cleaned historical multivariate time series data. The number of invalid actions is the number of invalid actions taken during the process from acquiring the historical multivariate time series data to acquiring the third-cleaned historical multivariate time series data.

[0044] If the historical processing priority for missing data, data anomalies, and constraint violations is data missing > data anomalies > constraint violations, then the single reference indicator value is the historical constraint violation rate of the third-cleaned historical multivariate time series data. Specifically, this refers to obtaining the improvement degree of the first single data quality indicator based on the historical constraint violation rate of the third-cleaned historical multivariate time series data and the historical constraint violation rate of the historical multivariate time series data. At this time, obtaining the third reward value based on the reference indicator value of the first-cleaned historical multivariate time series data specifically refers to: obtaining the improvement degree of the second single data quality indicator based on the historical data missing rate of the first-cleaned historical multivariate time series data and the historical data missing rate of the historical multivariate time series data; obtaining the second data structure smoothness based on the time series data fluctuation characteristics and trend characteristics in the reference indicator value of the first-cleaned historical multivariate time series data; obtaining the second data disturbance indicator based on the time series data fluctuation amplitude, mutation frequency, and dispersion in the reference indicator value of the first-cleaned historical multivariate time series data; and inputting the second single data quality indicator improvement degree, the second data structure smoothness, and the second data disturbance indicator into the second reward function to obtain the third reward value. The fourth reward value is obtained based on the reference index values ​​of the second-cleaned historical multivariate time series data. Specifically, this involves: obtaining the improvement degree of the third single data quality index based on the historical data anomaly rate of the second-cleaned historical multivariate time series data; obtaining the third data structure smoothness based on the time series data fluctuation characteristics and trend characteristics in the reference index values ​​of the second-cleaned historical multivariate time series data; obtaining the third data disturbance index based on the time series data fluctuation amplitude, mutation frequency, and dispersion in the reference index values ​​of the second-cleaned historical multivariate time series data; and inputting the third single data quality index improvement degree, the third data structure smoothness, and the third data disturbance index into the second reward function to obtain the fourth reward value.

[0045] Example 3: This example also provides a clinical time-series data cleaning system based on hierarchical agent reinforcement learning, applicable to the aforementioned clinical time-series data cleaning method based on hierarchical agent reinforcement learning, including: The data quality inspection module is used to detect missing, abnormal, and default data in real-time multivariate time series data to obtain the data missing rate, data abnormality rate, and constraint violation rate. The intelligent agent configuration module is used to construct a first reward function based on the processing priority and repair type of data missing, data anomaly and constraint violation as the first action space, and the improvement degree of the overall data quality index as the second action space, and to construct a second reward function based on the improvement degree of individual data quality index as the cleaning operator. A hierarchical intelligent agent construction module is used to construct a global quality control intelligent agent based on a first action space and a first reward function, and to construct a local cleaning intelligent agent based on a second action space and a second reward function; The global cleaning strategy decision module is used to input the data missing rate, data anomaly rate and constraint violation rate into the global quality control agent to obtain the actual processing priority and actual repair type of data missing, data anomaly and constraint violation. The local cleaning execution module is used to sequentially input the actual repair type into the local cleaning agent according to the actual processing priority of data missing, data anomaly, and constraint violation to obtain the actual cleaning operator, and then clean the real-time multivariate time series data through the actual cleaning operator.

[0046] like Figure 2 As shown, this system consists of three main modules: a data preprocessing module, a quality inspection module, and an automatic cleaning module. The data preprocessing module and the quality inspection module together constitute the data quality inspection module. The automatic cleaning module consists of an agent configuration module, a hierarchical agent construction module, a global cleaning strategy decision module, and a local cleaning execution module. The upper-level agent specifically refers to the global quality control agent, and the lower-level agent specifically refers to the local cleaning agent.

[0047] The specific embodiments described above are preferred embodiments of the clinical time-series data cleaning method and system based on hierarchical agent reinforcement learning of the present invention, and are not intended to limit the specific scope of the present invention. The scope of the present invention includes but is not limited to the specific embodiments described above. All equivalent changes made in accordance with the shape and structure of the present invention are within the protection scope of the present invention.

Claims

1. A clinical time-series data cleaning method based on hierarchical agent reinforcement learning, characterized in that, Includes the following steps: The missing data rate, anomaly rate, and constraint violation rate are obtained by performing missing, anomaly, and default detection on real-time multivariate time series data. The first action space is defined by the processing priority and repair type of data missing, data anomaly, and constraint violation. The first reward function is constructed by the improvement of the overall data quality index. The second action space is defined by the cleaning operator. The second reward function is constructed by the improvement of individual data quality index. A global quality control agent is constructed based on the first action space and the first reward function, and a local cleaning agent is constructed based on the second action space and the second reward function. Input the data missing rate, data anomaly rate, and constraint violation rate into the global quality control agent to obtain the actual processing priority and actual repair type of data missing, data anomaly, and constraint violation; Based on the actual processing priority of missing data, abnormal data, and constraint violation, the actual repair type is input into the local cleaning agent to obtain the actual cleaning operator, and the real-time multivariate time series data is cleaned through the actual cleaning operator; The process of detecting missing data, anomalies, and violations in real-time multivariate time-series data to obtain data missing rate, data anomaly rate, and constraint violation rate includes: Time alignment is performed on real-time multivariate time series data, and missing data detection is performed on real-time multivariate time series data to obtain the missing data rate; Feature extraction is performed on real-time multivariate time series data to obtain data features. An anomaly detection model is selected based on the data features. Anomaly detection model is used to detect anomalies in real-time multivariate time series data to obtain the data anomaly rate. Obtain historical multivariate time series data, perform constraint mining on the historical multivariate time series data to obtain a constraint set, and obtain the constraint violation rate based on the constraint set; In this process, continuous redundant data in real-time multivariate time series data are marked as missing data. Data features include conventional data features, outlier detection features, and basic model features. The constraint set includes inter-variable functional constraints, intra-variable time series change constraints, inter-variable time series monotonic constraints, and inter-variable negation constraints.

2. The clinical time-series data cleaning method based on hierarchical agent reinforcement learning according to claim 1, characterized in that, The process of extracting features from real-time multivariate time-series data, selecting an anomaly detection model based on these features, and then using the anomaly detection model to detect anomalies in the real-time multivariate time-series data and obtain the data anomaly rate includes: Perform routine feature extraction on real-time multivariate time series data to obtain routine data features; Real-time multivariate time series data is compared with normal data intervals to obtain outlier detection features; Real-time multivariate time series data is input into the candidate anomaly detection model to obtain basic model features; Based on conventional data features, outlier detection features, and basic model features, meta-learning is used to select an anomaly detection model. Real-time multivariate time series data is input into the anomaly detection model to obtain the average anomaly score of each data point in the real-time multivariate time series data. Data with an average anomaly score greater than a preset threshold are judged as anomaly data, thereby obtaining the data anomaly rate.

3. The clinical time-series data cleaning method based on hierarchical agent reinforcement learning according to claim 1, characterized in that, The process of obtaining a constraint set by performing constraint mining on historical multivariate time-series data, and obtaining the constraint violation rate based on the constraint set, includes: Use multinomial features and OLS regression to mine the functional relationships between variables in historical multivariate time series data and obtain functional constraints between variables; Obtain constraints on intra-variable temporal changes based on historical time series data of the same variable in historical multivariate time series data; Obtain temporal monotonic constraints between variables based on the monotonic influence relationships among variables in historical multivariate time series data; The variation range of the same variable is obtained from the historical time series data of the same variable in the historical multivariate time series data. The variation ranges of different variables in the same time period are combined to obtain the candidate negation range. Negation constraints between variables are obtained based on the candidate negation range. Based on inter-variable functional constraints, intra-variable temporal change constraints, inter-variable temporal monotonic constraints, and inter-variable negation constraints, violation data in real-time multivariable time series data is obtained, thereby obtaining the constraint violation rate.

4. The clinical time-series data cleaning method based on hierarchical agent reinforcement learning according to claim 1, characterized in that, The construction of the first reward function based on the improvement of overall data quality indicators includes: The first reward function is constructed by weighting and summing the overall data quality improvement, the time cost penalty during the cleaning process, and the penalty for invalid actions.

5. The clinical time-series data cleaning method based on hierarchical agent reinforcement learning according to claim 4, characterized in that, The construction of the second reward function based on the improvement of individual data quality indicators includes: The second reward function is constructed by weighting and summing the improvement of individual data quality indicators, the smoothness of data structure, and the data disturbance indicators.

6. The clinical time-series data cleaning method based on hierarchical agent reinforcement learning according to claim 1 or 5, characterized in that, The construction of a global quality control agent based on a first action space and a first reward function, and a local cleaning agent based on a second action space and a second reward function, includes: An initial global quality control agent is constructed based on the first action space and the first reward function, and an initial local cleaning agent is constructed based on the second action space and the second reward function. Input the historical data missing rate, historical data anomaly rate and historical constraint violation rate of historical multivariate time series data into the initial global quality control agent to obtain the historical processing priority and historical repair type of data missing, data anomaly and constraint violation; Input the historical repair type into the initial local cleaning agent to obtain the historical cleaning operator. Based on the historical processing priority of missing data, abnormal data, and constraint violation, the historical multivariate time series data is cleaned sequentially through the historical cleaning operator to obtain the cleaned historical multivariate time series data, and the reference index value of the cleaned historical multivariate time series data is obtained. Based on the reference index values, the state-action value functions in the initial global quality control agent and the initial local cleaning agent are updated to obtain the global quality control agent and the local cleaning agent.

7. The clinical time-series data cleaning method based on hierarchical agent reinforcement learning according to claim 6, characterized in that, The process involves sequentially cleaning historical multivariate time-series data using historical cleaning operators based on the historical processing priorities of missing data, data anomalies, and constraint violations. This includes: Based on the historical processing priorities of missing data, data anomalies, and constraint violations, historical cleaning operators are used to clean historical multivariate time series data to obtain the first cleaned historical multivariate time series data. Based on the historical processing priorities of missing data, data anomalies, and constraint violations, the first cleaned historical multivariate time series data is cleaned using historical cleaning operators to obtain the second cleaned historical multivariate time series data. Based on the historical processing priorities of missing data, data anomalies, and constraint violations, the second-cleaned historical multivariate time series data is cleaned using historical cleaning operators to obtain the third-cleaned historical multivariate time series data.

8. The clinical time-series data cleaning method based on hierarchical agent reinforcement learning according to claim 7, characterized in that, The process of updating the state-action value functions of the initial global quality control agent and the initial local cleaning agent based on reference index values ​​to obtain the global quality control agent and the local cleaning agent includes: The first reward value and the second reward value are obtained based on the reference index value of the third cleaning historical multivariate time series data. The state-action value function in the initial global quality control agent is updated based on the first reward value to obtain the global quality control agent. The third reward value is obtained based on the reference index value of the first cleaning historical multivariate time series data, and the fourth reward value is obtained based on the reference index value of the second cleaning historical multivariate time series data. The state-action value function in the initial local cleaning agent is updated based on the second, third, and fourth reward values ​​to obtain the local cleaning agent.

9. A clinical time-series data cleaning system based on hierarchical agent reinforcement learning, applicable to the clinical time-series data cleaning method based on hierarchical agent reinforcement learning as described in any one of claims 1-8, characterized in that, include: The data quality inspection module is used to detect missing, abnormal, and default data in real-time multivariate time series data to obtain the data missing rate, data abnormality rate, and constraint violation rate. The intelligent agent configuration module is used to construct a first reward function based on the processing priority and repair type of data missing, data anomaly and constraint violation as the first action space, and the improvement degree of the overall data quality index as the second action space, and to construct a second reward function based on the improvement degree of individual data quality index as the cleaning operator. A hierarchical intelligent agent construction module is used to construct a global quality control intelligent agent based on a first action space and a first reward function, and to construct a local cleaning intelligent agent based on a second action space and a second reward function; The global cleaning strategy decision module is used to input the data missing rate, data anomaly rate and constraint violation rate into the global quality control agent to obtain the actual processing priority and actual repair type of data missing, data anomaly and constraint violation. The local cleaning execution module is used to sequentially input the actual repair type into the local cleaning agent according to the actual processing priority of data missing, data anomaly, and constraint violation to obtain the actual cleaning operator, and then clean the real-time multivariate time series data through the actual cleaning operator.

Images