Method and related product for detecting and repairing abnormal data based on multi-constraint collaboration

Abstract

The application discloses a method and related products for abnormal data detection and repair based on multi-constraint cooperation, which can be applied to the technical field of data repair. The method comprises the following steps: determining a to-be-repaired time interval in a multivariate time series; establishing a dependency network according to the to-be-repaired time interval; obtaining a variable feature prediction order and a corresponding prediction model dictionary according to the dependency network; determining candidate repair values corresponding to the variable features according to the variable feature prediction order and the prediction model dictionary; and updating the multivariate time series according to the candidate repair values to realize repair of time series data. Therefore, the application divides the multivariate time series and uses an algorithm of the same class constraint to repair the multivariate time series, so that no new constraint violation is introduced in the repair process, the accuracy and reliability of the error data repair result are improved, and the error data repair speed is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data repair, in particular to a method for detecting and repairing abnormal data based on multi-constraint collaboration and related products. BACKGROUND

[0002] In the industrial field, the collection of time series data is mainly completed by sensors, and different sensors distributed on different machines can monitor the running status of the machines in real time. These sensors usually collect data at a frequency of seconds, so the amount of data collected is quite large.

[0003] The quality of the data stream is unstable due to the influence of many factors on the sensors. Therefore, there are a large number of error data in the multi-time series, which brings great difficulty to subsequent data management and analysis, so it is necessary to repair the time series data in time and effectively. In industrial big data, different sensor data belonging to the same system have strong correlation, that is, each different feature in the multi-time series has strong correlation, and these time series data are usually continuous, so the repair method based on functional dependence in the traditional relational database has reduced applicability, and the continuity of the data increases the difficulty of repairing poor data.

[0004] Therefore, how to improve the repair speed of error data under the premise of improving the accuracy and reliability of the repair result of error data is a problem that needs to be solved by those skilled in the art. SUMMARY

[0005] Based on the above problems, the present application provides a method for detecting and repairing abnormal data based on multi-constraint collaboration and related products, which divides the multi-time series and uses the algorithm of the upper class constraint to repair the multi-time series, thereby ensuring that no new constraint violation is introduced in the repair process, and solves the problems of low accuracy and reliability of the repair result and slow repair speed of the existing repair method.

[0006] In a first aspect, the embodiments of the present application provide a time series data repair method based on multi-constraint collaboration, comprising:

[0007] determining a to-be-repaired time interval in a multi-time series;

[0008] establishing a dependency network according to the to-be-repaired time interval;

[0009] obtaining a variable feature prediction order and a corresponding prediction model dictionary according to the dependency network;

[0010] determining a candidate repair value corresponding to a variable feature according to the variable feature prediction order and the prediction model dictionary;

[0011] updating the multivariate time series according to the candidate repair value, to realize repair of the time series data.

[0012] Optionally, the determining of the time interval to be repaired in the multivariate time series comprises:

[0013] performing compliance checking on the constraints in the constraint set to determine a rule-violating constraint;

[0014] determining an abnormal time interval in the multivariate time series according to the rule-violating constraint;

[0015] determining the time interval to be repaired in the multivariate time series according to the abnormal time interval;

[0016] The time interval to be repaired is greater than the abnormal time interval.

[0017] Optionally, the establishing of the dependency network according to the time interval to be repaired comprises:

[0018] obtaining a set of variable features in the time interval to be repaired;

[0019] establishing a variable feature dependency network directed graph according to a dependency relationship between variable features in the set of variable features, to realize the establishing of the dependency network.

[0020] Optionally, the obtaining of the variable feature prediction order and the corresponding prediction model dictionary according to the dependency network comprises:

[0021] analyzing the variable feature dependency network directed graph;

[0022] obtaining a variable feature prediction order according to the variable feature dependency network directed graph;

[0023] learning a variable feature prediction model according to the variable feature dependency network directed graph, to obtain a prediction model dictionary corresponding to the variable feature prediction order.

[0024] Optionally, the determining of the candidate repair value corresponding to the variable feature according to the variable feature prediction order and the prediction model dictionary comprises:

[0025] obtaining machine learning model information corresponding to the variable feature prediction order from the prediction model dictionary;

[0026] determining a candidate repair value corresponding to the variable feature by using the machine learning model information and the variable feature prediction order.

[0027] Optionally, before the updating of the candidate repair value into the multivariate time series to realize repair of the time series data, the method further comprises:

[0028] Based on the constraint information and the original data information, the candidate repair values ​​corresponding to the variable features are evaluated using the cleaning cost function;

[0029] Based on the evaluation results, the optimal candidate repair value is determined.

[0030] Optionally, updating the multivariate time series based on the candidate repair values ​​to repair the time series data includes:

[0031] Select the best candidate repair value from the candidate repair values;

[0032] The optimal candidate repair value is updated into the multivariate time series to achieve the repair of the time series data.

[0033] Secondly, embodiments of this application provide a time-series data repair apparatus based on multi-constraint collaboration, comprising:

[0034] The first determining module is used to determine the time interval to be repaired in the multivariate time series.

[0035] A module is established to create a dependency network based on the time interval to be repaired.

[0036] The acquisition module is used to obtain the variable feature prediction order and the corresponding prediction model dictionary based on the dependency network;

[0037] The second determining module is used to determine the candidate repair value corresponding to the variable feature based on the variable feature prediction order and the prediction model dictionary;

[0038] The repair module is used to update the candidate repair values ​​into the multivariate time series to achieve the repair of time series data.

[0039] Thirdly, embodiments of this application provide a time-series data repair device based on multi-constraint collaboration, comprising:

[0040] Memory, used to store computer programs;

[0041] A processor, configured to implement the steps of the time-series data repair method based on multi-constraint collaboration as described above when executing the computer program.

[0042] Fourthly, embodiments of this application provide a readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the time-series data repair method based on multi-constraint collaboration as described in any of the preceding claims.

[0043] As can be seen from the above technical solutions, compared with the prior art, this application has the following advantages:

[0044] In summary, the application first determines the time interval to be repaired in the multivariate time series, then establishes a dependency network according to the time interval to be repaired, and obtains a variable feature prediction order and a corresponding prediction model dictionary according to the dependency network. Then, the candidate repair values corresponding to the variable features are determined according to the variable feature prediction order and the prediction model dictionary. Finally, the multivariate time series is updated according to the candidate repair values to realize the repair of time series data. In this way, the multivariate time series is divided, and the algorithm of the upper class constraint is used to repair the multivariate time series, so as to ensure that no new constraint violation is introduced in the repair process, improve the accuracy and reliability of the error data repair result, and improve the error data repair speed. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 A flowchart of a method for detecting and repairing abnormal data based on multi-constraint collaboration provided by the application;

[0046] Figure 2 A structural schematic diagram of an apparatus for detecting and repairing abnormal data based on multi-constraint collaboration provided by the application. DETAILED DESCRIPTION

[0047] As described above, the existing time series data repair method cannot guarantee the accuracy and reliability of the error data repair result while guaranteeing the error data repair speed. Specifically, in industrial big data, different sensor data belonging to the same system have a strong correlation, that is, each different feature in the time series data in the multivariate time series has a strong correlation, and these time series data are usually continuous, so the repair method based on functional dependence in the traditional relational database has reduced applicability, and the continuity of data increases the difficulty of repairing poor data.

[0048] To solve the above problems, the embodiment of the application provides a time series data repair method based on multi-constraint collaboration, which comprises the following steps: first, determining the time interval to be repaired in the multivariate time series, then establishing a dependency network according to the time interval to be repaired, and obtaining a variable feature prediction order and a corresponding prediction model dictionary according to the dependency network. Then, the candidate repair values corresponding to the variable features are determined according to the variable feature prediction order and the prediction model dictionary. Finally, the multivariate time series is updated according to the candidate repair values to realize the repair of time series data.

[0049] In this way, the multivariate time series is divided, and the algorithm of the upper class constraint is used to repair the multivariate time series, so as to ensure that no new constraint violation is introduced in the repair process, improve the accuracy and reliability of the error data repair result, and improve the error data repair speed.

[0050] It should be noted that the multi-constraint collaborative time series data repair method and related products provided by the present application can be applied to the field of data repair technology. The above is only an example and does not limit the application field of the multi-constraint collaborative time series data repair method and related products provided by the present application.

[0051] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the present application.

[0052] Figure 1 The flowchart of the multi-constraint collaborative abnormal data detection and repair method provided by the embodiments of the present application is shown in FIG. 1. In combination with FIG. 1, the multi-constraint collaborative time series data repair method provided by the embodiments of the present application can include the following steps. Figure 1

[0053] S101: Determine the time interval to be repaired in the multivariate time series.

[0054] In actual application, the multivariate time series X is a sequence composed of n univariate time series. Each univariate time series x f is uniquely determined by the superscript f, where f represents a feature. The feature set of X is F={f1, f2,..., f n}. The timestamp set in the multivariate time series where K is a positive integer, and the timestamp set refers to the timestamp set of all data points in the multivariate time series X. When the timestamp is t and t∈T, the observation value of X is where represents the data value of feature f i when the timestamp is t. When the time interval T i,j is a continuous timestamp set T i,j ={t i ,....,t j}, and , the data point set of the multivariate time series X in the time interval T i,j is as follows:

[0055]

[0056] ​The above formula is called a subsequence of the multivariate time series X, and contains n univariate time series subsequences with the same time start and end points. In this application, in order to reduce the amount of data processing and improve processing speed, it is necessary to first divide the multivariate time series X into multiple repair blocks. A repair block B can be represented by a triple: B = <T B ,T abnormal ,F F >, where T B is the time interval of the repair block B, T abnormal is the abnormal time interval of the repair block B, and F F is the variable feature set of the repair block B. The features in F F are called variable features. Since any timestamp belonging to the abnormal time interval will cause the observation value corresponding to the timestamp in the multivariate time series to violate a certain constraint. Therefore, the multivariate time series can be divided into multiple repair blocks according to the time interval, that is, the to-be-repaired time interval in the multivariate time series is determined.

[0057] In addition, since the way to determine the to-be-repaired time interval in the multivariate time series is different, this application can describe the possible determination methods.

[0058] In one case, how to determine the to-be-repaired time interval. Accordingly, S101: determining the to-be-repaired time interval in the multivariate time series, specifically comprising:

[0059] checking the compliance of the constraints in the constraint set to determine the violated constraints;

[0060] determining the abnormal time interval in the multivariate time series according to the violated constraints;

[0061] determining the to-be-repaired time interval in the multivariate time series according to the abnormal time interval;

[0062] The to-be-repaired time interval is greater than the abnormal time interval.

[0063] In actual application, the compliance of the constraints in the constraint set can be checked to determine the violated constraints. For any timestamp t ∈ T abnormal , the observation value Xt of the multivariate time series X will cause the violation of a certain constraint, and for any feature f ∈ F F , there exists a timestamp t ∈ T abnormal , and the data point participates in causing the violation of the constraint. Therefore, the abnormal time interval in the multivariate time series can be determined according to the relationship between the abnormal time interval T abnormal and the violated constraint. It should be noted that in order to facilitate calculation, it is necessary to make the selected to-be-repaired time interval contain a reliable timestamp set, and the reliable timestamp set is defined as TR = T B -T abnormal That is, the value of the to-be-repaired time interval should be slightly larger than the abnormal time interval but not too large in order to ensure the calculation speed.

[0064] S102: establishing a dependency network according to the to-be-repaired time interval.

[0065] In actual applications, in order to repair the time series data, a dependency network needs to be established according to the to-be-repaired time interval after the to-be-repaired time interval in the multivariate time series is determined. It should be noted that this step is mainly to establish the directed graph part of the dependency network.

[0066] In addition, since the ways of establishing the dependency network are not the same, the present application can describe the possible establishment ways.

[0067] In one case, how to establish the dependency network. Accordingly, S102: establishing a dependency network according to the to-be-repaired time interval, specifically comprising:

[0068] obtaining a variable feature set in the to-be-repaired time interval;

[0069] establishing a variable feature dependency network directed graph according to the dependency relationship between the variable features in the variable feature set, so as to realize the establishment of the dependency network.

[0070] In actual applications, F F is the variable feature set of the repair block B, and the features in F F are called variable features. Since there is a dependency relationship between the variable features, the dependency relationship between the variable features in the repair block B is obtained first, that is, the dependency relationship between the variable features in the variable feature set in the to-be-repaired time interval, and then a variable feature dependency network directed graph is established according to the dependency relationship, so as to realize the establishment of the dependency network.

[0071] S103: obtaining a variable feature prediction order and a corresponding prediction model dictionary according to the dependency network.

[0072] In actual applications, after the dependency network is established, a variable feature prediction order and a corresponding prediction model dictionary need to be obtained according to the dependency network, so as to realize the subsequent repair of the time series data. For example, a global model dictionary can be created, all variable features can be looped and the corresponding model can be trained, a node including the variable features and the model dictionary can be created, and a recursive algorithm can be called to realize the obtaining of the variable feature prediction order and the corresponding prediction model dictionary according to the dependency network.

[0073] In addition, since the ways of obtaining the variable feature prediction order and the corresponding prediction model dictionary are different, the present application can describe the possible obtaining ways.

[0074] In one case, how to obtain the variable feature prediction order and the corresponding prediction model dictionary is described. Accordingly, S103: obtaining the variable feature prediction order and the corresponding prediction model dictionary according to the dependency network, specifically comprising:

[0075] analyzing the variable feature dependency network directed graph;

[0076] obtaining the variable feature prediction order according to the variable feature dependency network directed graph;

[0077] learning the variable feature prediction model according to the variable feature dependency network directed graph to obtain the prediction model dictionary corresponding to the variable feature prediction order.

[0078] In practical applications, the variable feature dependency network directed graph can be analyzed first, then the variable feature prediction order can be obtained and the variable feature prediction model can be learned according to the variable feature dependency network directed graph, and finally the prediction model dictionary corresponding to the variable feature prediction order can be obtained. The specific algorithm is as follows:

[0079] 1.

[0080] 2.F F ←Obtain the variable feature set of the repair block B

[0081] 3.T B ←Obtain the time interval of the repair block B

[0082] 4.T abnormal ←Obtain the abnormal time interval of the repair block B

[0083] 5.T R =T B -T abnormal

[0084] 6.for f in F F

[0085] 7.

[0086] 8.root←newNode(NULL,globalModelDict)

[0087] 9.PredictionOrderTree(root,G,T R )

[0088] 10. orderList, modelDictList <- root.GetTreePath()

[0089] 11. return(orderList, modelDictList)

[0090] In summary, a global model dictionary can be established first, and the variable feature set F of the repair block B is input F , time interval T B , abnormal time interval T abnormal , and reliable timestamp set T R = T B -T abnormal . Through the first input variable feature dependent network G, the relevant feature dictionary RF, and the repair block B, the relevant machine learning model is trained. Then the root node including the variable feature and the model dictionary is created and the recursive algorithm is called to realize the acquisition of the variable feature prediction order and the corresponding prediction model dictionary.

[0091] S104: Determine the candidate repair value corresponding to the variable feature according to the variable feature prediction order and the prediction model dictionary.

[0092] In actual application, in order to realize the repair of time series data, after acquiring the variable feature prediction order and the corresponding prediction model dictionary, the variable feature value search tree is searched using the variable feature prediction order and the corresponding prediction model dictionary, so as to determine the candidate repair value corresponding to the variable feature.

[0093] In addition, since the ways of determining the candidate repair value corresponding to the variable feature are different, the present application can describe the possible determination methods.

[0094] In one case, how to determine the candidate repair value corresponding to the variable feature. Correspondingly, S104: Determine the candidate repair value corresponding to the variable feature according to the variable feature prediction order and the prediction model dictionary, specifically comprising:

[0095] Obtain the machine learning model information corresponding to the variable feature prediction order from the prediction model dictionary;

[0096] Determine the candidate repair value corresponding to the variable feature using the machine learning model information and the variable feature prediction order.

[0097] In practical applications, the prediction model dictionary records all the machine learning model information that may be used by the variable feature prediction order corresponding to the position. Therefore, the application can obtain the machine learning model information corresponding to the variable feature prediction order from the prediction model dictionary, and then determine the candidate repair value corresponding to the variable feature by using the machine learning model information and the variable feature prediction order. The specific algorithm is as follows:

[0098] 1. if i = order.size()

[0099] 2.

[0100] 3. return

[0101] 4. models <- modelDict[order[i]]

[0102] 5.

[0103] 6. for model F，f in models

[0104] 7.

[0105] 8.

[0106] 9.

[0107] 10.

[0108] In summary, a variable feature prediction order order, a corresponding prediction model dictionary modeDict, a multivariate time subsequence and the i-th variable feature in the variable feature prediction order currently traversed can be given to perform prediction, obtain the variable feature candidate repair value and retain the variable feature original information.

[0109] S105: updating the multivariate time sequence according to the candidate repair value to realize the repair of time series data.

[0110] In practical applications, after obtaining the candidate repair value, the multivariate time sequence can be updated according to the candidate repair value to realize the repair of the error time series data in the multivariate time sequence.

[0111] In addition, since the less data that needs to be repaired, the shorter the repair time becomes, therefore, the application can be described in the way of possibly shortening the repair time.

[0112] In one scenario, regarding how to shorten the repair time for time-series data, the process includes, prior to updating the candidate repair values ​​to the multivariate time series to repair the time-series data, the following steps are also included:

[0113] Based on the constraint information and the original data information, the candidate repair values ​​corresponding to the variable features are evaluated using the cleaning cost function;

[0114] Based on the evaluation results, the optimal candidate repair value is determined.

[0115] In practical applications, if we simply predict the target feature values ​​using the corresponding prediction model dictionary according to the prediction order of variable features (i.e., predict candidate repair values), and then use these candidate repair values ​​to update the time series, then all variable feature values ​​in the resulting new time series will be modified. However, such repair results are not ideal. The original data information of the variable features may be correct, so we cannot modify the data value of every variable feature. Therefore, this application considers both the benefits and costs of candidate repair values. Based on the constraint information and the original data information, a cleaning cost function is used to evaluate the candidate repair values ​​corresponding to the variable features, and the optimal candidate repair value is determined based on the evaluation results. It is important to note that the fewer constraints are violated by the candidate repair value, the greater the benefit; the greater the modification of the original data information by the candidate repair value, the higher the cost. Therefore, we search for a candidate repair value among all candidate repairs as the repair result to update the multivariate time series, which has both greater benefits and lower costs. This candidate repair value is the optimal candidate repair value. The specific cleaning cost function is as follows:

[0116]

[0117] Violation rate utilizes constraint information, and Repair distance utilizes original data information. k is a constant greater than 0, used to control the dominance of Violation rate and Repair distance. To make the constraints more accurate, the value of k is generally set to be large, and the smaller the value of the cleaning cost function VRRD, the better the candidate repair value.

[0118] Furthermore, since there are different ways to update multivariate time series, this application can describe the possible update methods.

[0119] In one scenario, regarding how to update a multivariate time series to repair the time series data, S105: Update the multivariate time series according to the candidate repair values ​​to repair the time series data, specifically including:

[0120] Select the best candidate repair value from the candidate repair values;

[0121] updating the best candidate repair value into the multivariate time series to achieve repair of the time series data.

[0122] In practical applications, in order to reduce costs and improve the benefits of candidate repair, the candidate repair value corresponding to the variable feature is evaluated by using the cleaning cost function. The candidate repair value with the minimum VRRD value is selected, that is, the best candidate repair value in the candidate repair value is selected. The best candidate repair value is updated into the multivariate time series to achieve repair of the time series data.

[0123] To sum up, the application first determines the to-be-repaired time interval in the multivariate time series, then establishes a dependency network according to the to-be-repaired time interval, and obtains a variable feature prediction order and a corresponding prediction model dictionary according to the dependency network. Then, the candidate repair value corresponding to the variable feature is determined according to the variable feature prediction order and the prediction model dictionary. Finally, the multivariate time series is updated according to the candidate repair value to achieve repair of the time series data. In this way, the application divides the multivariate time series and repairs the multivariate time series by using the algorithm of the upper class constraint, thereby ensuring that no new constraint violation is introduced in the repair process, improving the repair speed of the error data on the premise of improving the accuracy and reliability of the error data repair result.

[0124] Based on the method for detecting and repairing abnormal data based on multi-constraint cooperation provided in the above embodiment, the application further provides a device for detecting and repairing abnormal data based on multi-constraint cooperation. The device for detecting and repairing abnormal data based on multi-constraint cooperation is described below in combination with embodiments and drawings.

[0125] Figure 2 A structure diagram of the device for detecting and repairing abnormal data based on multi-constraint cooperation provided in the embodiments of the application is shown in the figure. In combination with the above description, Figure 2 The device for detecting and repairing abnormal data based on multi-constraint cooperation provided in the embodiments of the application 200 comprises:

[0126] A first determination module 201 is configured to determine a to-be-repaired time interval in a multivariate time series.

[0127] An establishment module 202 is configured to establish a dependency network according to the to-be-repaired time interval.

[0128] An acquisition module 203 is configured to acquire a variable feature prediction order and a corresponding prediction model dictionary according to the dependency network.

[0129] A second determination module 204 is configured to determine a candidate repair value corresponding to the variable feature according to the variable feature prediction order and the prediction model dictionary.

[0130] The repair module 205 is configured to update the candidate repair value to the multivariate time series, so as to realize repair of the time series data.

[0131] As an implementation, the first determination module 201 is configured to determine the to-be-repaired time interval in the multivariate time series in the following manner.

[0132] Conducting compliance checking on the constraints in the constraint set to determine a rule-violating constraint;

[0133] Determining an abnormal time interval in the multivariate time series according to the rule-violating constraint;

[0134] Determining the to-be-repaired time interval in the multivariate time series according to the abnormal time interval;

[0135] The to-be-repaired time interval is greater than the abnormal time interval.

[0136] As an implementation, the establishment module 202 is configured to establish the dependency network according to the to-be-repaired time interval in the following manner.

[0137] Obtaining a set of variable features in the to-be-repaired time interval;

[0138] Establishing a variable feature dependency network directed graph according to dependency relationships between variable features in the set of variable features, so as to realize establishment of the dependency network.

[0139] As an implementation, the obtaining module 203 is configured to obtain the variable feature prediction order and the corresponding prediction model dictionary according to the dependency network in the following manner.

[0140] Analyzing the variable feature dependency network directed graph;

[0141] Obtaining a variable feature prediction order according to the variable feature dependency network directed graph;

[0142] Learning a variable feature prediction model according to the variable feature dependency network directed graph, so as to obtain a prediction model dictionary corresponding to the variable feature prediction order.

[0143] As an implementation, the second determination module 204 is configured to determine the candidate repair value corresponding to the variable feature according to the variable feature prediction order and the prediction model dictionary in the following manner.

[0144] Obtaining machine learning model information corresponding to the variable feature prediction order from the prediction model dictionary;

[0145] Determining the candidate repair value corresponding to the variable feature by using the machine learning model information and the variable feature prediction order.

[0146] As an implementation form, in order to shorten the repair time of the time series data, the device 200 for detecting and repairing abnormal data based on multi-constraint collaboration further comprises an evaluation module.

[0147] The evaluation module is configured to evaluate the candidate repair value corresponding to the variable feature by using a cleaning cost function according to the constraint information and the original data information.

[0148] According to the evaluation result, the best candidate repair value is determined.

[0149] As an implementation form, in order to update the multi-element time series according to the candidate repair value and realize the repair of the time series data, the repair module 205 is specifically configured to:

[0150] select the best candidate repair value from the candidate repair values;

[0151] update the best candidate repair value into the multi-element time series to realize the repair of the time series data.

[0152] In summary, the application first determines the time interval to be repaired in the multi-element time series, then establishes a dependency network according to the time interval to be repaired, and obtains a variable feature prediction order and a corresponding prediction model dictionary according to the dependency network. Then, the candidate repair value corresponding to the variable feature is determined according to the variable feature prediction order and the prediction model dictionary. Finally, the multi-element time series is updated according to the candidate repair value to realize the repair of the time series data. In this way, the application divides the multi-element time series and repairs the multi-element time series by using the algorithm of the above constraint to ensure that no new constraint violation is introduced in the repair process, thereby improving the error data repair speed on the premise of improving the accuracy and reliability of the error data repair result.

[0153] In addition, the application further provides a time series data repair device based on multi-constraint collaboration, characterized in that it comprises:

[0154] a memory configured to store a computer program;

[0155] a processor configured to execute the computer program to realize the steps of the time series data repair method based on multi-constraint collaboration according to any one of the above.

[0156] In addition, the application further provides a readable storage medium, wherein the readable storage medium stores a computer program, and the computer program is executed by a processor to realize the steps of the time series data repair method based on multi-constraint collaboration according to any one of the above.

[0157] The foregoing description of the disclosed embodiments enables a person skilled in the art to make or use the application. Modifications of these embodiments will occur to persons of skill in the art, and that the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Therefore, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting and repairing abnormal data based on multi-constraint collaboration, characterized in that, The method includes: Identify the time intervals to be repaired in the multivariate time series; Establish a dependency network based on the time interval to be repaired; The variable feature prediction order and the corresponding prediction model dictionary are obtained based on the dependent network. The candidate repair values ​​corresponding to the variable features are determined based on the variable feature prediction order and the prediction model dictionary; The multivariate time series is updated based on the candidate repair values ​​to achieve the repair of time series data; The step of establishing a dependency network based on the time interval to be repaired includes: Obtain the set of variable features within the time interval to be repaired; Based on the dependencies between the variable features in the variable feature set, a directed graph of the variable feature dependency network is established to realize the establishment of the dependency network; The step of obtaining the variable feature prediction order and the corresponding prediction model dictionary based on the dependency network includes: Analyze the directed graph of the variable feature-dependent network; The prediction order of variable features is obtained based on the directed graph of the variable feature dependency network; Learn a variable feature prediction model based on the directed graph of the variable feature dependency network, and obtain a prediction model dictionary corresponding to the variable feature prediction order; The step of determining the candidate repair value corresponding to the variable feature based on the variable feature prediction order and the prediction model dictionary includes: Obtain the machine learning model information corresponding to the variable feature prediction order from the prediction model dictionary; The candidate repair values ​​corresponding to the variable features are determined using the machine learning model information and the variable feature prediction order.

2. The method according to claim 1, characterized in that, The process of determining the time interval to be repaired in the multivariate time series includes: Perform compliance checks on the constraints in the constraint union to identify non-compliant constraints; Based on the aforementioned violation constraints, abnormal time intervals in the multivariate time series are determined; The time intervals to be repaired in the multivariate time series are determined based on the abnormal time intervals; The time interval to be repaired is greater than the abnormal time interval.

3. The method according to claim 1, characterized in that, Before updating the multivariate time series based on the candidate repair values ​​to repair the time series data, the method further includes: Based on the constraint information and the original data information, the candidate repair values ​​corresponding to the variable features are evaluated using the cleaning cost function; Based on the evaluation results, the optimal candidate repair value is determined.

4. The method according to claim 3, characterized in that, The step of updating the multivariate time series based on the candidate repair values ​​to repair the time series data includes: Select the best candidate repair value from the candidate repair values; The optimal candidate repair value is updated into the multivariate time series to achieve the repair of the time series data.

5. A device for detecting and repairing abnormal data based on multi-constraint collaboration, characterized in that, include: The first determining module is used to determine the time interval to be repaired in the multivariate time series. A module is established to create a dependency network based on the time interval to be repaired. The acquisition module is used to obtain the variable feature prediction order and the corresponding prediction model dictionary based on the dependency network; The second determining module is used to determine the candidate repair value corresponding to the variable feature based on the variable feature prediction order and the prediction model dictionary; The repair module is used to update the candidate repair values ​​into the multivariate time series to achieve the repair of time series data; The establishment module is specifically used for: Obtain the set of variable features within the time interval to be repaired; Based on the dependencies between the variable features in the variable feature set, a directed graph of the variable feature dependency network is established to realize the establishment of the dependency network; The acquisition module is specifically used for: Analyze the directed graph of the variable feature-dependent network; The prediction order of variable features is obtained based on the directed graph of the variable feature dependency network; Learn a variable feature prediction model based on the directed graph of the variable feature dependency network, and obtain a prediction model dictionary corresponding to the variable feature prediction order; The second determining module is specifically used for: Obtain the machine learning model information corresponding to the variable feature prediction order from the prediction model dictionary; The candidate repair values ​​corresponding to the variable features are determined using the machine learning model information and the variable feature prediction order.

6. A device for detecting and repairing abnormal data based on multi-constraint collaboration, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the method for detecting and repairing abnormal data based on multi-constraint collaboration as described in any one of claims 1 to 4 when executing the computer program.

7. A readable storage medium, characterized in that, The readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method for detecting and repairing abnormal data based on multi-constraint collaboration as described in any one of claims 1 to 4.

Images