Data filling and anomaly monitoring method and system based on multi-task learning

Abstract

The invention provides a data filling and abnormity monitoring method and system based on multi-task learning, and relates to the technical field of sensor data processing.The method comprises the steps that multiple types of sensor data are integrated and preprocessed, correlation among the multiple types of sensor data is comprehensively analyzed, and a correlation matrix is obtained; the method comprises the following steps: grouping multiple types of sensors, constructing a multi-task learning prediction model based on a correlation matrix, inputting monitoring data of related sensors at corresponding time of a certain measuring point in the trained model to obtain prediction data and simulation data, performing data filling based on the prediction data, and performing anomaly detection based on the simulation data. Through the multi-task learning prediction model, the inherent correlation among multiple types of monitoring data and the correlation of measuring points in time and space can be fully utilized, and the monitoring data prediction and filling precision is improved.

Description

Technical Field

[0001] The present invention relates to the field of sensor data processing, and in particular to a data filling and anomaly monitoring method and system based on multi-task learning. Background Art

[0002] Modern structural, geological, and other monitoring systems are usually composed of multiple types of sensors. Various types of sensors are deployed at various locations on the object being measured to form a sensor network. The data collected by different sensors are usually correlated, especially sensor data with similar measurement content, sensor data with mutually influential measurement properties, and sensor data that are relatively close in time and space. These data have strong correlations, such as the changing trends of temperature and structural displacement in bridge monitoring, the physical coupling of bridge strain and vibration data, and abnormal blade vibration and gearbox temperature in wind tower monitoring.

[0003] However, traditional data prediction and filling methods usually process various types of monitoring data (such as strain, vibration, temperature, displacement, etc.) separately, and fail to fully utilize the correlation between multiple types of data. That is, traditional data prediction methods for single-type sensors lack comprehensive consideration of information inference on missing data of some sensors, and have limitations in processing high-dimensional and nonlinear data. It is difficult to effectively capture the mutual state influence in complex structures, resulting in low data prediction accuracy.

[0004] In traditional abnormal data detection methods, the monitoring system does not clearly display the data of some fault measurement points in abnormal data alarms. Traditional methods find it difficult to detect data distortion caused by sensor failures, so there are many false alarms and missed alarms.

[0005] Therefore, it is very necessary to design a method and system that can comprehensively consider the correlation between data by performing correlation analysis on multi-type sensor data, and perform data prediction and filling and abnormal data detection based on the correlation. Summary of the Invention

[0006] To solve the above problems, the present invention provides a data filling and anomaly monitoring method and system based on multi-task learning, which can perform correlation analysis between multiple types of sensor data, comprehensively consider the correlation between data, and perform data prediction and filling and abnormal data detection based on the correlation, with high data prediction and anomaly detection accuracy.

[0007] To achieve the above object, the technical solution adopted by the present invention is:

[0008] The present invention provides a data filling and anomaly monitoring method based on multi-task learning, comprising the following steps:

[0009] S1. Integrate data collected by multiple types of sensors and align timestamps, and preprocess the data after timestamp alignment;

[0010] S2. Calculate the correlation coefficient, analyze the statistical correlation between the preprocessed data, perform cross-correlation analysis on the time series data, study the lag relationship between the data, use the Granger causality test to analyze the causal relationship between the data, integrate the analysis results, and compile a preliminary correlation matrix of the monitoring system's measurement points / data;

[0011] S3. Build the topology of the sensor network, group it based on a clustering algorithm, and design shared parameters for each group. Based on the preliminary measurement point / data correlation matrix, build and train a multi-task learning prediction model to extract common features from multiple types of data.

[0012] S4. Select a measuring point and input the monitoring data of other relevant sensors corresponding to the time of the measuring point into the trained multi-task learning prediction model to obtain the predicted data or simulated data of the measuring point during that period. Use the predicted data for data filling and compare and analyze the simulated data with the actual data collected at the measuring point to determine whether there are any anomalies in the data. Combined with the output characteristics of different tasks, the cause of the anomaly is distinguished.

[0013] Preferably, in step S1, a time window-based interpolation algorithm is used to align timestamps, and the sensor data is divided into time windows. If a sensor has no data in the window, the data is interpolated using a cubic spline interpolation method based on the data of the previous and next time points.

[0014] Furthermore, in step S2, the step of analyzing the statistical correlation between the data includes:

[0015] Select an appropriate correlation coefficient;

[0016] For multi-type, multi-point location monitoring data, the selected correlation coefficient is calculated for each pair of variables;

[0017] The strength of the correlation between variables is determined based on the value of the calculated correlation coefficient;

[0018] A significance test was performed to determine whether the correlation was significant.

[0019] Furthermore, in step S2, the step of performing cross-correlation analysis on the data includes:

[0020] Determine the appropriate maximum lag order based on the actual scenario;

[0021] Calculate cross-correlation functions;

[0022] Plot the calculated cross-correlation coefficient as a function of the lag order to form a cross-correlation plot;

[0023] Set a significance level and perform statistical tests to determine which lag orders have cross-correlation functions that are significantly different from zero.

[0024] Furthermore, in step S2, the causal relationship between the data is analyzed using the Granger causality test. The specific process includes:

[0025] Perform a stationary test on the data;

[0026] Construct models without causality and models with causality respectively;

[0027] Determine the lag order according to the criteria;

[0028] Estimate the parameters of the non-causal relationship model and the causal relationship model to obtain the residual sum of squares of the non-causal relationship model and the residual sum of squares of the causal relationship model;

[0029] Calculate test statistics and construct Granger causality test statistics;

[0030] Perform hypothesis testing based on statistics, give a significance level, look up the distribution table to get the critical value, compare the calculated statistics with the critical value, and determine the causal relationship between the data.

[0031] Furthermore, in step S3, the step of constructing a sensor topology map includes:

[0032] Determine the nodes and edges in the topology graph based on the physical deployment locations and network connection relationships of the sensors;

[0033] Assign edge weights based on relevant factors.

[0034] Furthermore, in step S3, the step of grouping the sensors using the spectral clustering algorithm includes:

[0035] Construct a similarity matrix based on the sensor topology graph;

[0036] Calculate the Laplacian matrix from the similarity matrix;

[0037] Perform eigenvalue decomposition on the Laplace matrix to obtain the eigenvalues ​​and corresponding eigenvectors;

[0038] Map the data points of each sensor node into the low-dimensional space spanned by the selected feature vectors to obtain a new representation;

[0039] In low-dimensional space, the K-means algorithm is used to cluster the mapped points, and sensors with close spatiotemporal locations and strong correlation are divided into the same group.

[0040] Furthermore, in step S3, based on the preliminary measurement point / data correlation matrix, a multi-task learning prediction model is constructed using the Transformer architecture. The specific process includes:

[0041] Determine the dependencies of task pieces based on the dependency matrix and design a shared-private parameter architecture;

[0042] Design parameter sharing strategy based on the correlation matrix and sensor grouping results;

[0043] Extract common features of multi-type data through shared layers;

[0044] Design loss functions by combining multi-task objectives and physical constraints;

[0045] Dynamically adjust weights based on sensor network changes and task importance;

[0046] Adopt a time segment division strategy to alternate between training and validation;

[0047] Common feature visualization and analysis.

[0048] On the other hand, the present invention provides a data filling and anomaly monitoring system based on multi-task learning, including a data acquisition and processing module, a correlation analysis module, a model building and optimization module, and a data prediction filling and anomaly detection module connected to each other;

[0049] The data acquisition and processing module is used to integrate data collected by multiple types of sensors and align timestamps, and pre-process the data after timestamp alignment;

[0050] The correlation analysis module is used to calculate the correlation coefficient, analyze the statistical correlation between pre-processed data, perform cross-correlation analysis on time series data, study the lag relationship between data, use Granger causality test to analyze the causal relationship between data, integrate the analysis results, and compile a preliminary correlation matrix of measurement points / data of the monitoring system;

[0051] The model building and optimization module is used to build the topology of the sensor network, grouping based on a clustering algorithm and designing shared parameters for each group. Based on the preliminary measurement point / data correlation matrix, a multi-task learning prediction model is built and trained to extract common features of multiple types of data.

[0052] The data prediction and filling and anomaly detection module is used to select a certain measuring point, input the monitoring data of other relevant sensors corresponding to the time of the measuring point into the trained multi-task learning prediction model, obtain the predicted data or simulated data of the measuring point during the period, use the predicted data to fill in the data, compare and analyze the simulated data with the actual data collected at the measuring point, determine whether there are anomalies in the data, and distinguish the causes of the anomalies by combining the output characteristics of different tasks.

[0053] Furthermore, the data prediction filling and anomaly detection module includes a data prediction filling unit and an abnormal data detection unit;

[0054] The data prediction and filling unit is used to select a certain measuring point, input the monitoring data of other relevant sensors corresponding to the time of the measuring point into the trained multi-task learning prediction model, obtain the predicted data of the measuring point during the period, and use the predicted data to perform data filling;

[0055] The abnormal data detection unit is used to select a certain measuring point, input the monitoring data of other relevant sensors corresponding to the time of the measuring point into the trained multi-task learning prediction model, obtain simulated data of the measuring point during the period, compare and analyze the simulated data with the data actually collected at the measuring point, determine whether there is an abnormality in the data, and distinguish the cause of the abnormality by combining the output characteristics of different tasks.

[0056] The beneficial effects of the present invention are:

[0057] By constructing a multi-task learning prediction model, the present invention can fully utilize the inherent correlation between multiple types of monitoring data and the correlation between measurement points in time and space to improve the accuracy of monitoring data prediction and filling. Through correlation analysis between multiple types of sensor data, not only can the performance of traditional single-type data models be optimized, but real-time detection and isolation of faulty measurement points can also be achieved, thereby significantly improving the robustness of the monitoring system. It is capable of identifying abnormal sensors and dynamically detecting faulty measurement points that deviate from group characteristics by utilizing the statistical correlation of multi-sensor data. It is capable of adaptive data repair. When a sensor is determined to be faulty, the system automatically enables interpolation or prediction models based on other related sensor data to ensure monitoring continuity. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is a flow chart of a data filling and anomaly monitoring method based on multi-task learning in the present invention.

[0059] Figure 2 This is a block diagram of a data filling and anomaly monitoring system based on multi-task learning in the present invention. DETAILED DESCRIPTION

[0060] See also Figures 1 to 2 As shown, the present invention relates to a data filling and anomaly monitoring method and system based on multi-task learning.

[0061] Specifically, the present invention provides a data filling and anomaly monitoring method based on multi-task learning, comprising the following steps:

[0062] S1. Integrate data collected by multiple types of sensors and align timestamps, and preprocess the data after timestamp alignment;

[0063] Multiple sensors, such as temperature sensors (DS18B20), pressure sensors (MPX5010DP), and vibration sensors (ADXL345), are deployed throughout the monitoring area. Each sensor transmits data to a gateway via IoT communications. The gateway then sends data to the data center via network protocols such as 5G / NB-IoT.

[0064] Since different sensor data collection frequencies and clocks are different (e.g., temperature sensors collect data once per second, and vibration sensors collect data 10 times per second), in order to unify the time distribution, an interpolation algorithm based on a time window is used to align the timestamps. For example, a unified time window is set (e.g., 1 minute), and the data of each sensor is divided according to the time window. If a sensor has no data within the window, the cubic spline interpolation method is used to interpolate based on the data of the previous and next time points to ensure that all data are accurately matched in the time dimension. The cubic spline interpolation formula is: S(x) = a j +b j (xx j )+c j (xx j ) 2 +d j (xx j ) 3 , where x j is a known data point, a j , b j , c j , d j are coefficients calculated from known data points.

[0065] Preprocess the data after timestamp alignment, including handling missing values ​​and outliers, and standardizing or normalizing the data.

[0066] When dealing with missing values ​​in data, there are two cases to be handled separately.

[0067] For small proportions of missing values, that is, when the missing values ​​account for less than 10%, for numerical data that conforms to the normal distribution (such as temperature data), the formula is used to calculate the mean, and the mean is used to fill the missing values; for numerical data that is not normally distributed (such as pressure data), the median is found by sorting and filled; non-numeric data (such as equipment status identification) is filled with the mode.

[0068] For situations with a large proportion of missing values ​​(i.e., when the missing value ratio exceeds 15%), a missing value prediction model based on random forests is constructed. By building multiple decision trees, the model leverages the advantages of ensemble learning to mine data feature relationships to predict missing values. For example, to predict missing data for a pressure sensor, the model uses data from temperature and vibration sensors on the same device, as well as features such as time and device operating status, as input. The random forest model outputs a predicted value through a voting mechanism among multiple decision trees, completing the missing value filling.

[0069] The isolation forest algorithm is used for outlier detection. When constructing an isolation tree, data is randomly sampled. The shorter the path length of each sample point in the tree, the more likely it is an outlier. The path length threshold is set to 1.5 times the standard deviation of the average path length. Data exceeding this threshold is considered an outlier.

[0070] For a small number of outliers (accounting for less than 5%), they are directly deleted; for a large number of outliers, the local weighted regression model (LWLR) is used for correction, that is, weighted regression is performed based on the data points around the target data point, and the weights are determined by the Gaussian kernel function. The corrected normal data is obtained by minimizing the weighted error sum of squares to fit the curve.

[0071] For most numerical data, such as continuous variables like temperature and pressure, Z-score normalization is used. For example, for temperature data, assume that the mean temperature over a certain period of time is μ = 25°C and the standard deviation is σ = 3°C. After normalizing the original temperature value x = 28°C to x' = 1, the data is converted to a standard normal distribution with a mean of 0 and a standard deviation of 1.

[0072] For data with a specific range, such as sensor voltage output (0-5V), Min-Max normalization is used. For example, if a sensor voltage output value x = 3V, min(x) = 0V, and max(x) = 5V, then after normalization, x' = 0.6, mapping the data to the interval [0, 1].

[0073] S2. Calculate the correlation coefficient, analyze the statistical correlation between the preprocessed data, perform cross-correlation analysis on the time series data, study the lag relationship between the data, use the Granger causality test to analyze the causal relationship between the data, integrate the analysis results, and compile a preliminary correlation matrix of the monitoring system's measurement points / data;

[0074] The steps of analyzing the statistical correlation between data include:

[0075] When analyzing the statistical correlation between data, the first thing to do is to choose an appropriate correlation coefficient;

[0076] The Pearson correlation coefficient measures the linear correlation between two variables and is applicable when the data is normally distributed. The Spearman correlation coefficient is a nonparametric rank correlation coefficient that measures the correlation between the ranks of two variables and does not strictly require the distribution of the data. First, the data must be converted to ranks, and then the Pearson correlation coefficient between the ranks is calculated. The Kendall correlation coefficient is a nonparametric rank correlation coefficient that measures correlation based on the ordering information of the data pairs.

[0077] For multi-type, multi-point location monitoring data, the selected correlation coefficient is calculated for each pair of variables;

[0078] After selecting the correlation coefficient, you need to calculate it for each pair of variables (i.e., different measurement points or different types of data). You can use the relevant libraries in your programming language or statistical software to perform the calculation. For example, in Python, you can use the numpy and pandas libraries to calculate the Pearson correlation coefficient, and the scipy.stats library to calculate the Spearman and Kendall correlation coefficients.

[0079] The strength of the correlation between variables is determined based on the value of the calculated correlation coefficient;

[0080] Generally speaking, the closer the absolute value of the correlation coefficient is to 1, the stronger the correlation; the closer it is to 0, the weaker the correlation. A positive correlation coefficient indicates a positive correlation between two variables; that is, when one variable increases, the other tends to increase as well. A negative correlation coefficient indicates a negative correlation between two variables; that is, when one variable increases, the other tends to decrease. For example, a Pearson correlation coefficient of r = 0.8 indicates a strong positive linear correlation between two variables, r = -0.6 indicates a strong negative linear correlation, and r = 0.2 indicates a weak linear correlation.

[0081] A significance test was performed to determine whether the correlation was significant.

[0082] To determine whether a correlation is statistically significant, a significance test is required. This can be achieved by calculating a p-value, which is a probability value calculated based on sample data in a hypothesis test, indicating the likelihood that the observed difference would occur under the null hypothesis. If the p-value is less than a pre-set significance level (usually 0.05 or 0.01), the correlation is considered significant, meaning that the correlation between the two variables is not due to random factors.

[0083] The steps of performing cross-correlation analysis on the data include:

[0084] First, determine the appropriate maximum lag order k based on the actual scenario;

[0085] Calculate the cross-correlation function (CCF):

[0086] For two time series X t and Y t , calculate their cross covariance C at different lag orders l (l = -k, ..., 0, ..., k) XY (l):

[0087]

[0088] Where n is the length of the time series, and They are X t and Y t The mean of .

[0089] Calculate the cross-correlation coefficient r XY (l):

[0090]

[0091] Among them, C XX (0) and C YY (0) are X t and Y t The autocovariance of .

[0092] Plot the calculated cross-correlation coefficient as a function of the lag order to form a cross-correlation plot;

[0093] The calculated cross-correlation coefficient r XY (l) is plotted as a function of the lag order l to form a cross-correlation diagram. By observing the graph, we can intuitively understand the lag relationship between the two time series.

[0094] Set a significance level (such as 0.05) and use statistical tests (such as hypothesis testing using asymptotic normality) to determine which lag orders have cross-correlation coefficients that are significantly different from zero. A significant cross-correlation coefficient indicates that there is a meaningful correlation between the two time series at the corresponding lag order. Specifically, if the lag order l>0, r XY (l) is significantly different from zero, indicating that X t The change of Y after l time steps t If the lag order l<0, r XY (l) is significantly different from zero, which means that Y t The change of X|l| time steps ago t For example, if r XY (3) Significant, which means that the change of X sequence will affect Y sequence after 3 time steps, that is, X t With Y t+3 Related.

[0095] In addition, when it is necessary to analyze the cross-correlation between multiple time series simultaneously, multiple comparison corrections, such as Bonferroni correction and Benjamini-Hochberg correction, are required to control the false discovery rate (FDR). These methods adjust the significance level based on the number of comparisons to avoid excessive false rejection of the null hypothesis.

[0096] Cross-correlation analysis was performed on time series data, and the cross-correlation function (CCF) was calculated at different lag orders. Taking the impact of temperature changes on humidity changes as an example, the cross-correlation function (CCF) was calculated using the Python statsmodels library and plotted. By observing the cross-correlation function curve, the lag relationship between time series data was investigated. Assuming that the cross-correlation function value reaches a peak at a lag of three time steps, indicating that temperature changes will have the greatest impact on humidity after three time steps, the optimal lag time between temperature and humidity data is determined to be three time steps, providing a time parameter basis for subsequent model design.

[0097] Granger causality test is a statistical method used to test the causal relationship between variables in time series data. The Granger causality test is used to analyze the causal relationship between data. The specific process includes:

[0098] For two time series variables X t and Y t, t = 1, 2, …, T, where T is the sample size. First, the data needs to be tested for stationarity. Common methods include unit root tests (such as the ADF test). If the data is non-stationary, it may be necessary to perform differencing or other transformations to make it a stationary series to meet the prerequisites for Granger causality testing.

[0099] The following two regression models are constructed respectively:

[0100] No causality model (null hypothesis H0: X is not the Granger cause of Y):

[0101]

[0102] Among them, β i is the regression coefficient, p is the lag order, ∈ t is the error term.

[0103] There is a causal relationship model (alternative hypothesis H1: X is the Granger cause of Y):

[0104]

[0105] Among them, α j is the lag coefficient of X, and q is the lag order of X.

[0106] The lag orders p and q can usually be determined based on criteria such as the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC). These criteria balance the goodness of fit and complexity of the model, and the lag order that minimizes the criterion value is selected as the optimal order.

[0107] The parameters of the non-causal relationship model and the causal relationship model are estimated using a method such as the least square method to obtain the residual sum of squares RSS0 of the non-causal relationship model and the residual sum of squares RSS1 of the causal relationship model.

[0108] Calculate the test statistic and construct the Granger causality test statistic F:

[0109]

[0110] The statistic follows an F distribution with (q,Tpq-1) degrees of freedom.

[0111] Perform hypothesis testing based on the statistic F, give a significance level α (such as α = 0.05), and look up the F distribution table to obtain the critical value F α (q,Tpq-1), the calculated statistic F and the critical value F α Compare and judge the causal relationship between the data. If the calculated F value is greater than the critical value F α, then reject the null hypothesis H0, and believe that X is the Granger cause of Y, that is, there is a causal relationship between X and Y; if the F value is less than or equal to the critical value F α , then we cannot reject the null hypothesis and conclude that X is not the Granger cause of Y.

[0112] Similarly, we can test whether Y is the Granger cause of X, thereby comprehensively analyzing the causal relationship between the two variables.

[0113] For each pair of measurement point data, if there is a correlation between the two, the calculated correlation coefficient is filled in the corresponding position of the correlation matrix. For example, if there are measurement points A and B, their correlation coefficient is r AB , then r AB Fill in the corresponding positions in the matrix, row A, column B and row B, column A (because the correlation coefficient is symmetrical).

[0114] When performing cross-correlation analysis, the correlation between the data of two measurement points at different lag times will be obtained. For each pair of measurement points, the lag time τ that makes the correlation reach the maximum and the corresponding maximum correlation coefficient r are determined. max For example, if there are measuring points A and B, when the lag time is τ, their maximum correlation coefficient is r max , then r max (τ) is filled in the corresponding position in the matrix.

[0115] After performing the Granger causality test on each pair of measurement points, the judgment result on the causal relationship will be obtained, and a specific symbol can be used to represent the Granger causality. For example, for measurement points A and B, if the Granger causality test shows that measurement point A is the Granger cause of measurement point B, then G + Fill in the corresponding position in the matrix; if measurement point B is the Granger cause of measurement point A, then G - Fill in the corresponding position in the matrix; if there is no Granger cause between the two, then G 0 Fill in the corresponding position in the matrix.

[0116] The results of correlation coefficient calculation, cross-correlation analysis, and Granger causality test are combined to organize the relationships between all variables into a matrix, which is the initial correlation matrix of the monitoring system's measurement points / data. This matrix allows for intuitive observation of the correlations between variables, facilitating subsequent analysis and processing.

[0117] S3. Build the topology of the sensor network, group it using a clustering algorithm, and design shared parameters for each group. Based on the preliminary measurement point / data correlation matrix, build and train a multi-task learning prediction model to extract common features from multiple types of data.

[0118] The steps to construct a sensor topology map include:

[0119] Determine nodes and edges: Determine the nodes and edges in the topology graph based on the sensors' physical locations and network connectivity. Consider each sensor as a node. If two sensors are physically adjacent or directly connected in the network, add an edge between them.

[0120] Assigning edge weights: Edge weights can be determined based on the distance between sensors, signal strength, or other relevant factors. For example, edges between sensors that are closer together can have larger weights, indicating a closer connection. Edge weights for connections with stronger signal strength can also be increased accordingly.

[0121] The steps to group sensors using the spectral clustering algorithm include:

[0122] Construct similarity matrix: Construct similarity matrix S according to sensor topology. If there is an edge between sensor i and sensor j, then S ij The value of can be set as the weight of the edge; if there is no edge connected, then S ij =0.

[0123] Calculate the Laplace matrix: Calculate the Laplace matrix L from the similarity matrix S, such as the Laplace matrix L = DS of the undirected graph, where D is a diagonal matrix with diagonal elements D ii =∑ i S ij , which is the sum of the desired edge weights of node i.

[0124] Calculate eigenvalues ​​and eigenvectors: Perform eigenvalue decomposition on the Laplace matrix L and obtain eigenvalues ​​λ1≤λ2≤…≤λ n and the corresponding eigenvectors v1,v2,…,v n .

[0125] Data point mapping: Map the data points of each sensor node to the low-dimensional space spanned by the selected feature vectors to obtain a new representation. For example, for sensor i, its representation in the low-dimensional space is

[0126] y i =V T x i , where x i is the original characteristic line of sensor i (which can be its connection feature in the topology diagram, etc.).

[0127] Clustering: In low-dimensional space, the K-means algorithm is used to cluster the mapped points, and sensors with close spatiotemporal locations and strong correlation are divided into the same group.

[0128] Design shared parameters for each group of divided sensors:

[0129] Determine parameter types: Based on the characteristics of the sensor data and application requirements, determine the shared parameters that need to be designed. For example, for a temperature sensor group, you can design a shared temperature threshold parameter; for a pressure sensor group, you can design a shared pressure change rate parameter.

[0130] Parameter estimation: For each cluster group, the shared parameters are estimated using the data from the sensors within the group.

[0131] Parameter Adjustment and Optimization: Through further analysis of sensor data and feedback from actual applications, the initially estimated shared parameters are adjusted and optimized. For example, if deviations or anomalies are found in the sensor data within a cluster, the shared parameters can be adjusted accordingly to better adapt to the overall characteristics of the sensors within the cluster.

[0132] For example, consider an industrial device equipped with 10 sensors. Sensors with close proximity and related functions are grouped together. First, a similarity matrix is ​​calculated between the sensors. A graph Laplacian matrix is ​​constructed based on this similarity matrix. By performing eigendecomposition of the Laplacian matrix, the top k eigenvectors are selected for clustering, and the 10 sensors are divided into three groups. Shared parameters are designed for each group. During model training, sensors within the same group share some layer parameters of the Transformer encoder, reducing parameter redundancy and improving training efficiency.

[0133] Based on the preliminary measurement point / data correlation matrix, a multi-task learning prediction model is constructed using the Transformer architecture. The specific process includes:

[0134] Determine the dependencies of task pieces based on the dependency matrix and design a shared-private parameter architecture;

[0135] The shared layer uses a Transformer encoder to extract common features across all measurement points. The shared layer parameters are learned jointly by all tasks (measurement point prediction). For example, the input layer is used to map the time series data of each measurement point into a high-dimensional feature space. The shared Transformer layer uses a self-attention mechanism to capture temporal dependencies and spatial correlations between different measurement points.

[0136] A separate output layer, also known as a private layer, is designed for each measurement point to predict the target value of that measurement point. For example, the decoder layer of each measurement point is used to generate the prediction result of that measurement point based on the shared features.

[0137] Design parameter sharing strategy based on the correlation matrix and sensor grouping results;

[0138] Hard parameter sharing: All tasks share the underlying Transformer layers, with each task having its own unique output layer above them. The parameters of the shared layers are updated jointly using the gradients of all tasks.

[0139] Soft parameter sharing: allows the parameters of different tasks to differ, but constrains the parameter similarity through regularization terms (such as weight decay).

[0140] Extract common features of multi-type data through shared layers;

[0141] Temporal features: The Transformer's self-attention mechanism captures dependencies between different time steps and extracts common patterns such as seasonality and trends.

[0142] Spatial features: Based on the correlation matrix, the model learns the spatial associations between different measurement points (such as the coupling relationship between temperature and pressure).

[0143] Physical Constraint Embedding: Incorporating physical knowledge into shared layers, such as encoding the constraints of the heat conduction equation into the attention weights, to enhance the physical interpretability of features.

[0144] Design loss functions by combining multi-task objectives and physical constraints;

[0145] Basic loss term: For each measurement point i, the mean square error (MSE) is used to calculate the difference between the predicted value and the true value:

[0146] Among them, y i , t is the true value, is the predicted value, and T is the time step.

[0147] Physical constraint term: add physical law constraint residual L phy , for example the residual of the heat conduction equation:

[0148]

[0149] Where T is the predicted temperature and α is the thermal diffusion coefficient.

[0150] Total loss function:

[0151] Among them, ω i is the task weight (dynamically adjustable), and λ is the physical constraint weight.

[0152] Dynamically adjust weights based on sensor network changes and task importance;

[0153] Uncertainty-based weights: Use the predicted variance of a task to estimate its uncertainty, with higher uncertainty tasks receiving higher weights:

[0154] in, is the prediction variance of task i.

[0155] Adaptive weight updates: During training, weights are dynamically adjusted based on the gradient norm of the task or the validation set performance.

[0156] Adopt a time segment division strategy to alternate between training and validation;

[0157] Divide the data into K non-overlapping segments (e.g., K = 10) in chronological order and train using the leave-one-out method. This method uses K-1 segments to train the model each time, and the remaining segment for validation. Repeat this cycle K times to assess model stability. Stop training when the validation set performance stops improving to prevent overfitting.

[0158] The specific training process is as follows: Data from each measurement point is normalized and time-aligned, and input features are filtered according to the correlation matrix. The parameters of the shared and private layers are randomly initialized. A forward propagation algorithm is used to extract common features through the shared layer, and predictions for each measurement point are generated through the private layer. A loss function is calculated by combining multi-task losses and physical constraints. Backpropagation is used to update parameters in the shared and private layers, dynamically adjusting task weights. The prediction performance of the model at each measurement point is evaluated using a validation set.

[0159] In addition, finite element analysis (FEA) simulation data can be used as an auxiliary task to train the model together with actual monitoring data to enhance model interpretability. An anti-interference loss function can also be introduced into model training. When abnormal data (such as sensor failure data) is detected, different weights are assigned based on the normality of the data, with normal data weighted to 1 and abnormal data weighted to 0.1. This automatically reduces the impact of abnormal data in loss calculations and prevents abnormal data from contaminating the model.

[0160] Common feature visualization and analysis.

[0161] By visualizing attention weights, we analyze the contribution of different measurement points to shared features. We cluster the extracted common features to verify their consistency with the sensor topology grouping results. We compare the common features with known physical laws to verify that the model has learned patterns that are consistent with physical intuition.

[0162] S4. Select a measuring point and input the monitoring data of other relevant sensors corresponding to the time of the measuring point into the trained multi-task learning prediction model to obtain the predicted data or simulated data of the measuring point during that period. Use the predicted data for data filling, compare and analyze the simulated data with the actual data collected at the measuring point to determine whether there are any anomalies in the data, and distinguish the causes of the anomalies based on the output characteristics of different tasks.

[0163] Specifically, a target measuring point A (e.g., a temperature sensor at a key location) is selected, and its related sensors (e.g., nearby pressure sensors and vibration sensors) are identified based on the correlation matrix. The trained multi-task learning prediction model then inputs the monitoring data from other related sensors corresponding to the time period of measuring point A. Common features are extracted through the shared parameter layer, and the output layer corresponding to measuring point A outputs the predicted data for that time period. The predicted data is used to fill in the gaps in the data for measuring point A. For example, if measuring point A is missing data for 10 time points, the model outputs 10 predicted values, completing the data prediction and filling function.

[0164] Select target measurement point B and determine its relevant sensors based on the correlation matrix. During the real-time data collection process at measurement point B, input the monitoring data of other relevant sensors corresponding to the time period of measurement point B into the trained multi-task learning prediction model. Common features are extracted through the shared parameter layer, and simulated data for that period is output through the output layer corresponding to measurement point B. The simulated data is compared and analyzed with the actual collected data to determine whether the data is abnormal. For example, thresholds for mean square error (MSE) and relative error (RE) are set, such as MSE = 0.1 and RE = 5%. The mean square error (MSE) and relative error (RE) of the simulated data and the actual collected data are calculated. If MSE > 0.1 or RE > 5%, the data is considered abnormal.

[0165] Furthermore, through a multi-task learning prediction model, the output characteristics of different tasks are combined to distinguish the cause of the anomaly. For example, if the temperature data is abnormal and the output characteristics of the temperature-related pressure and vibration tasks also show abnormal patterns, it may be that the actual monitored object is damaged. If only the temperature task output is abnormal, while other related tasks are normal, it may be that the temperature sensor is faulty, providing users with accurate anomaly diagnosis results.

[0166] In addition, the present invention provides a data filling and anomaly monitoring system based on multi-task learning, which includes a data acquisition and processing module, a correlation analysis module, a model building and optimization module, and a data prediction filling and anomaly detection module. Each module works in coordination to achieve efficient processing and in-depth mining of multi-type sensor data.

[0167] The data acquisition and processing module is used to integrate data collected by multiple types of sensors and align timestamps, and pre-process the data after timestamp alignment;

[0168] The correlation analysis module is used to calculate the correlation coefficient, analyze the statistical correlation between pre-processed data, perform cross-correlation analysis on time series data, study the lag relationship between data, use Granger causality test to analyze the causal relationship between data, integrate the analysis results, and compile a preliminary correlation matrix of measurement points / data of the monitoring system;

[0169] The model building and optimization module is used to build the topology of the sensor network, grouping based on a clustering algorithm and designing shared parameters for each group. Based on the preliminary measurement point / data correlation matrix, a multi-task learning prediction model is built and trained to extract common features of multiple types of data.

[0170] The data prediction and filling and anomaly detection module is used to select a certain measuring point, input the monitoring data of other relevant sensors corresponding to the time of the measuring point into the trained multi-task learning prediction model, obtain the predicted data or simulated data of the measuring point during the period, use the predicted data to fill in the data, compare and analyze the simulated data with the actual data collected at the measuring point, determine whether there are anomalies in the data, and distinguish the causes of the anomalies by combining the output characteristics of different tasks.

[0171] The data prediction filling and anomaly detection module includes a data prediction filling unit and an abnormal data detection unit;

[0172] The data prediction and filling unit is used to select a certain measuring point, input the monitoring data of other relevant sensors corresponding to the time of the measuring point into the trained multi-task learning prediction model, obtain the predicted data of the measuring point during the period, and use the predicted data to perform data filling;

[0173] The abnormal data detection unit is used to select a certain measuring point, input the monitoring data of other relevant sensors corresponding to the time of the measuring point into the trained multi-task learning prediction model, obtain simulated data of the measuring point during the period, compare and analyze the simulated data with the actual data collected at the measuring point, determine whether there is an anomaly in the data, and distinguish the cause of the anomaly by combining the output characteristics of different tasks.

[0174] The system is deployed on a cloud server and uses Docker container technology for environmental isolation and rapid deployment. The front-end uses the Vue.js framework to develop a visual interface, allowing users to view monitoring data, prediction results, and anomaly detection information in real time. In industrial equipment monitoring scenarios, the system can predict equipment failures in advance, reducing downtime; in environmental monitoring scenarios, it can promptly detect anomalies in environmental parameters, supporting environmental decision-making. Furthermore, the system can flexibly adjust model parameters and thresholds based on the needs of different scenarios, continuously optimizing performance to meet diverse monitoring needs.

[0175] The above embodiments are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary engineering technicians in this field should fall within the scope of protection determined by the claims of the present invention.

Claims

1. A data filling and anomaly monitoring method based on multi-task learning, characterized in that: The following steps are involved: S1. Integrate data collected by multiple types of sensors and align timestamps, and preprocess the data after timestamp alignment; S2. Calculate the correlation coefficient, analyze the statistical correlation between the preprocessed data, perform cross-correlation analysis on the time series data, study the lag relationship between the data, use the Granger causality test to analyze the causal relationship between the data, integrate the analysis results, and compile a preliminary correlation matrix of the monitoring system's measurement points / data; S3. Build the topology of the sensor network, group it based on a clustering algorithm, and design shared parameters for each group. Based on the preliminary measurement point / data correlation matrix, build and train a multi-task learning prediction model to extract common features from multiple types of data. S4. Select a measuring point and input the monitoring data of other relevant sensors corresponding to the time of the measuring point into the trained multi-task learning prediction model to obtain the predicted data or simulated data of the measuring point during that period. Use the predicted data for data filling and compare and analyze the simulated data with the actual data collected at the measuring point to determine whether there are any anomalies in the data. Combined with the output characteristics of different tasks, the cause of the anomaly is distinguished.

2. The method for data filling and anomaly monitoring based on multi-task learning according to claim 1, characterized in that: In step S1, the time stamps are aligned using a time window-based interpolation algorithm, and the sensor data are divided into time windows. If a sensor has no data within the window, the data is interpolated using the cubic spline interpolation method based on the data of the previous and next time points.

3. The method for data filling and anomaly monitoring based on multi-task learning according to claim 1, characterized in that: In step S2, the step of analyzing the statistical correlation between the data includes: Select an appropriate correlation coefficient; For multi-type, multi-point location monitoring data, the selected correlation coefficient is calculated for each pair of variables; The strength of the correlation between variables is determined based on the value of the calculated correlation coefficient; A significance test was performed to determine whether the correlation was significant.

4. The method for data filling and anomaly monitoring based on multi-task learning according to claim 3 is characterized in that: In step S2, the steps of performing cross-correlation analysis on the data include: Determine the appropriate maximum lag order based on the actual scenario; Calculate cross-correlation functions; Plot the calculated cross-correlation coefficient as a function of the lag order to form a cross-correlation plot; Set a significance level and perform statistical tests to determine which lag orders have cross-correlation functions that are significantly different from zero.

5. The method for data filling and anomaly monitoring based on multi-task learning according to claim 4 is characterized in that: In step S2, the Granger causality test is used to analyze the causal relationship between the data. The specific process includes: Perform a stationary test on the data; Construct models without causality and models with causality respectively; Determine the lag order according to the criteria; Estimate the parameters of the non-causal relationship model and the causal relationship model to obtain the residual sum of squares of the non-causal relationship model and the residual sum of squares of the causal relationship model; Calculate test statistics and construct Granger causality test statistics; Perform hypothesis testing based on statistics, give a significance level, look up the distribution table to get the critical value, compare the calculated statistics with the critical value, and determine the causal relationship between the data.

6. The method for data filling and anomaly monitoring based on multi-task learning according to claim 1, characterized in that: In step S3, the steps of constructing the sensor topology map include: Determine the nodes and edges in the topology graph based on the physical deployment locations and network connection relationships of the sensors; Assign edge weights based on relevant factors.

7. The method for data filling and anomaly monitoring based on multi-task learning according to claim 6, characterized in that: In step S3, the step of grouping sensors using the spectral clustering algorithm includes: Construct a similarity matrix based on the sensor topology map; Calculate the Laplacian matrix from the similarity matrix; Perform eigenvalue decomposition on the Laplace matrix to obtain the eigenvalues ​​and corresponding eigenvectors; Map the data points of each sensor node into the low-dimensional space spanned by the selected feature vectors to obtain a new representation; In low-dimensional space, the K-means algorithm is used to cluster the mapped points, and sensors with close spatiotemporal locations and strong correlation are divided into the same group.

8. The method for data filling and anomaly monitoring based on multi-task learning according to claim 7, characterized in that: In step S3, based on the preliminary measurement point / data correlation matrix, a multi-task learning prediction model is constructed using the Transformer architecture. The specific process includes: Determine the dependencies of task pieces based on the dependency matrix and design a shared-private parameter architecture; Design parameter sharing strategy based on the correlation matrix and sensor grouping results; Extract common features of multi-type data through shared layers; Design loss functions by combining multi-task objectives and physical constraints; Dynamically adjust weights based on sensor network changes and task importance; Adopt a time segment division strategy to alternate between training and validation; Common feature visualization and analysis.

9. A data filling and anomaly monitoring system based on multi-task learning, used to execute the data filling and anomaly monitoring method based on multi-task learning according to any one of claims 1 to 8, characterized in that: It includes a data acquisition and processing module with communication connection, a correlation analysis module, a model building and optimization module, and a data prediction, filling and anomaly detection module; The data acquisition and processing module is used to integrate data collected by multiple types of sensors and align timestamps, and pre-process the data after timestamp alignment; The correlation analysis module is used to calculate the correlation coefficient, analyze the statistical correlation between pre-processed data, perform cross-correlation analysis on time series data, study the lag relationship between data, use Granger causality test to analyze the causal relationship between data, integrate the analysis results, and compile a preliminary correlation matrix of measurement points / data of the monitoring system; The model building and optimization module is used to build the topology of the sensor network, grouping based on a clustering algorithm and designing shared parameters for each group. Based on the preliminary measurement point / data correlation matrix, a multi-task learning prediction model is built and trained to extract common features of multiple types of data. The data prediction and filling and anomaly detection module is used to select a certain measuring point, input the monitoring data of other relevant sensors corresponding to the time of the measuring point into the trained multi-task learning prediction model, obtain the predicted data or simulated data of the measuring point during the period, use the predicted data to fill in the data, compare and analyze the simulated data with the actual data collected at the measuring point, determine whether there are anomalies in the data, and distinguish the causes of the anomalies by combining the output characteristics of different tasks.

10. The data filling and anomaly monitoring system based on multi-task learning according to claim 9, characterized in that: The data prediction filling and anomaly detection module includes a data prediction filling unit and an abnormal data detection unit; The data prediction and filling unit is used to select a certain measuring point, input the monitoring data of other relevant sensors corresponding to the time of the measuring point into the trained multi-task learning prediction model, obtain the predicted data of the measuring point during the period, and use the predicted data to perform data filling; The abnormal data detection unit is used to select a certain measuring point, input the monitoring data of other relevant sensors corresponding to the time of the measuring point into the trained multi-task learning prediction model, obtain simulated data of the measuring point during the period, compare and analyze the simulated data with the actual data collected at the measuring point, determine whether there is an anomaly in the data, and distinguish the cause of the anomaly by combining the output characteristics of different tasks.

Images