Abnormal data detection and identification method for time series

Abstract

The invention relates to the technical field of time sequence anomaly detection, and discloses a time sequence abnormal data detection and identification method, which comprises the following steps of: filtering historical data by adopting a single-side anomaly detection algorithm; inputting the historical abnormal data into a zero sample matching strategy algorithm by adopting the zero sample matching strategy algorithm; collecting to-be-identified data, and filtering the to-be-identified data by using a unilateral anomaly detection algorithm; collecting expert experience data, and performing whole-in and whole-out data modeling according to the historical matching data or the expert experience data; generating an integer-in and integer-out model matching algorithm according to the identified abnormal data and the integer-in and integer-out model pool, and obtaining an integer-in and integer-out behavior matching result; and obtaining a model pool updating algorithm according to the integral-in and integral-out model pool and the integral-in and integral-out behavior matching result. According to the method, a unilateral anomaly detection algorithm, a zero sample matching strategy algorithm and expert empirical data modeling are combined, so that the abnormal data in the time sequence are comprehensively detected and identified.

Description

Technical Field

[0001] The present invention relates to the technical field of time series anomaly detection, and in particular to a method for detecting and identifying abnormal data in a time series. Background Art

[0002] In the business of identifying the risk characteristics of "entry and exit", data is recorded and processed in the form of unlabeled logs. And due to the different categories of customs declaration goods, the specific morphological characteristics of "entry and exit" behavior will be different, such as the one-time exit mentioned above, multiple exits on the same day, and periodic split exits. In addition, due to the actual situation of unlabeled data, there is only empirical knowledge about the mode of entry and exit.

[0003] Therefore, it is particularly important to develop a method that can automatically identify and detect abnormal data in time series. Summary of the invention

[0004] In view of this, the present invention proposes a method for detecting and identifying abnormal data in time series, aiming to solve the problem of inaccurate detection and identification of abnormal data in the current technology of "whole-in and whole-out" risk feature identification business due to unlabeled data, diverse patterns and insufficient empirical knowledge.

[0005] The present invention proposes a method for detecting and identifying abnormal data in a time series, comprising:

[0006] S100: Filter the historical data using a unilateral anomaly detection algorithm to obtain historical anomaly data;

[0007] S200: adopting a zero-sample matching strategy algorithm, inputting the historical abnormal data into the zero-sample matching strategy algorithm, and obtaining historical matching data;

[0008] S300: collecting data to be identified, and filtering the data to be identified using the unilateral anomaly detection algorithm to obtain identified anomaly data;

[0009] S400: Collect expert experience data, perform whole-in-whole-out data modeling according to the historical matching data or the expert experience data, and obtain a whole-in-whole-out model pool;

[0010] S500: generating an entire-in-an-out model matching algorithm according to the identified abnormal data and the entire-in-an-out model pool, and obtaining an entire-in-an-out behavior matching result;

[0011] S600: Obtaining a model pool update algorithm according to the whole-in-whole-out model pool and the whole-in-whole-out behavior matching result.

[0012] Furthermore, the historical data includes historical input data and historical output data.

[0013] Furthermore, a unilateral anomaly detection algorithm is used to filter historical data to obtain historical anomaly data, including:

[0014] Acquire the historical input data and the historical output data, arrange the historical input data and the historical output data in chronological order, and generate a historical input time series and a historical output time series;

[0015] A first empirical cumulative probability density function is calculated based on the historical integrated time series and the historical integrated time series, and a first differential function of the inverse function of the first empirical cumulative probability density function is calculated based on the first empirical cumulative probability density function, wherein the first differential function is recorded as Δ inv,eCDF (q1);

[0016] A first hypothesis testing algorithm is constructed based on the first differential function, and a first abnormal data threshold is calculated according to the first hypothesis testing algorithm; the data in the historical integral-in and integral-out time series are compared with the first abnormal data threshold one by one, and the historical abnormal data is filtered out according to the comparison result.

[0017] Further, when calculating the first abnormal data threshold according to the first hypothesis testing algorithm, it includes:

[0018] The first hypothesis testing method was constructed based on 99.5% significance, where the first null hypothesis was set as

[0019] Take the maximum score that satisfies the first null hypothesis as value q 1max As the first abnormal data threshold.

[0020] Furthermore, a zero-sample matching strategy algorithm is used, and the historical abnormal data is input into the zero-sample matching strategy algorithm to obtain the historical matching data, including:

[0021] Set up several whole-in and whole-out strategies;

[0022] Based on the selected round-in and round-out strategy, each of the historical round-in data within the time window is matched with the historical round-out data to obtain the historical matching data.

[0023] Further, the data to be identified is obtained, and the data to be identified is arranged in chronological order to generate a time series of the data to be identified;

[0024] A second empirical cumulative probability density function is calculated based on the time series of the data to be identified, and an inverse function of the second empirical cumulative probability density function is calculated based on the second empirical cumulative probability density function, wherein the first differential function is recorded as Δ inv,eCDF(q2);

[0025] A second hypothesis testing algorithm is constructed based on the second differential function, and a second abnormal data threshold is calculated according to the second hypothesis testing algorithm; the data in the to-be-identified data time series are compared with the second abnormal data threshold one by one, and the identified abnormal data are filtered out according to the comparison result; wherein the identified abnormal data includes identifying abnormal input data and identifying abnormal output data.

[0026] Further, when calculating the second abnormal data threshold according to the second hypothesis testing algorithm, it includes:

[0027] The second hypothesis testing method was constructed based on 99.5% significance, in which the second null hypothesis was set as

[0028] Take the maximum score that satisfies the second null hypothesis as value q 2max as the second abnormal data threshold.

[0029] Furthermore, collecting expert experience data, performing whole-in-whole-out data modeling according to the historical matching data or the expert experience data, and obtaining a whole-in-whole-out model pool includes:

[0030] The historical matching data or the expert experience data is fitted using a monotonically increasing function model to obtain a monotonically increasing function of the time lag and the cumulative round-off percentage, denoted as P i (dt), i = 1, 2, ..., N, the domain of the monotonically increasing function of the time lag and the cumulative integer percentage is from negative infinity to positive infinity, and the range is [0, 1];

[0031] Using the monotonically increasing function of the time lag and the cumulative round-off percentage as a round-in and round-off template function;

[0032] All the whole-in-whole-out template functions are collected to construct a whole-in-whole-out template function set, and the whole-in-whole-out template function set is the whole-in-whole-out model pool.

[0033] Further, when generating an entire-in-an-out model matching algorithm according to the identified abnormal data and the entire-in-an-out model pool, and obtaining an entire-in-an-out behavior matching result, it includes:

[0034] Calculate the matching pattern with the minimum distance between the identified abnormal data and the whole-in and whole-out template function set by an optimization algorithm;

[0035] The whole-in and whole-out behavior matching result is obtained according to the minimum distance matching mode.

[0036] Further, when obtaining the model pool update algorithm according to the whole-in-whole-out model pool and the whole-in-whole-out behavior matching result, it includes:

[0037] Finding the whole-in-whole-out template function with the shortest distance from the whole-in-whole-out behavior matching result in the whole-in-whole-out model pool;

[0038] Re-sampling the whole-in and whole-out template function, and adding the data of the whole-in and whole-out behavior matching result to form a new whole-in and whole-out template function set;

[0039] The new set of integral-in and integral-out template functions is refitted to obtain updated integral-in and integral-out template functions.

[0040] Compared with the prior art, the beneficial effect of the present invention is that the present invention realizes comprehensive detection and identification of abnormal data in time series by combining the unilateral anomaly detection algorithm, the zero-sample matching strategy algorithm and the expert experience data modeling. Among them, the unilateral anomaly detection algorithm can effectively filter out the non-abnormal part of the historical data and retain the real abnormal data for subsequent analysis. The zero-sample matching strategy algorithm uses these historical abnormal data to further extract representative historical matching data through the matching of the whole-in and whole-out strategy, providing strong support for subsequent modeling.

[0041] The present invention also fully considers the importance of expert experience data. By collecting and integrating expert experience data, and carrying out whole-in-whole-out data modeling together with historical matching data, a rich whole-in-whole-out model pool is constructed. This model pool not only contains the rules automatically learned from the data, but also incorporates the professional knowledge and experience of experts, making the model more comprehensive and accurate. After obtaining the identification abnormal data and the whole-in-whole-out model pool, the method further generates a whole-in-whole-out model matching algorithm, which is used to match the identification abnormal data with the template function in the model pool, so as to obtain the whole-in-whole-out behavior matching result. This process not only realizes the in-depth analysis and understanding of the abnormal data, but also provides a basis for subsequent model updating and optimization. Finally, according to the whole-in-whole-out model pool and the whole-in-whole-out behavior matching results, the method also obtains a model pool update algorithm. The algorithm can dynamically update and optimize the model pool according to the new matching results, ensuring that the model always maintains the adaptability and accuracy to the latest data. This feature makes the method more flexible and scalable in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0043] Figure 1A flowchart of a method for detecting and identifying abnormal data in a time series provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features described in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0045] See also Figure 1 As shown, in some embodiments of the present application, this embodiment provides a method for detecting and identifying abnormal data in a time series, including:

[0046] S100: Filter the historical data using a unilateral anomaly detection algorithm to obtain historical anomaly data;

[0047] S200: adopting a zero-sample matching strategy algorithm, inputting the historical abnormal data into the zero-sample matching strategy algorithm, and obtaining historical matching data;

[0048] S300: collecting data to be identified, and filtering the data to be identified using the unilateral anomaly detection algorithm to obtain identified anomaly data;

[0049] S400: Collect expert experience data, perform whole-in-whole-out data modeling according to the historical matching data or the expert experience data, and obtain a whole-in-whole-out model pool;

[0050] S500: generating an entire-in-an-out model matching algorithm according to the identified abnormal data and the entire-in-an-out model pool, and obtaining an entire-in-an-out behavior matching result;

[0051] S600: Obtaining a model pool update algorithm according to the whole-in-whole-out model pool and the whole-in-whole-out behavior matching result.

[0052] In this embodiment, a method for detecting and identifying abnormal data in a time series realizes comprehensive detection and identification of abnormal data in a time series by combining a unilateral anomaly detection algorithm, a zero-sample matching strategy algorithm, and expert experience data modeling. Among them, the unilateral anomaly detection algorithm can effectively filter out the non-abnormal part of the historical data and retain the real abnormal data for subsequent analysis. The zero-sample matching strategy algorithm uses these historical abnormal data to further extract representative historical matching data through the matching of the whole-in and whole-out strategy, providing strong support for subsequent modeling.

[0053] In this embodiment, the abnormal data detection and identification method also fully considers the importance of expert experience data. By collecting and integrating expert experience data, and carrying out whole-in-whole-out data modeling together with historical matching data, a rich whole-in-whole-out model pool is constructed. This model pool not only contains the rules automatically learned from the data, but also incorporates the professional knowledge and experience of experts, making the model more comprehensive and accurate. After obtaining the identified abnormal data and the whole-in-whole-out model pool, the method further generates a whole-in-whole-out model matching algorithm for matching the identified abnormal data with the template function in the model pool, thereby obtaining the whole-in-whole-out behavior matching result. This process not only realizes the in-depth analysis and understanding of the abnormal data, but also provides a basis for subsequent model updating and optimization. Finally, according to the whole-in-whole-out model pool and the whole-in-whole-out behavior matching results, the method also obtains a model pool update algorithm. The algorithm can dynamically update and optimize the model pool according to the new matching results, ensuring that the model always maintains the adaptability and accuracy of the latest data. This feature makes the method more flexible and scalable in practical applications.

[0054] Specifically, the historical data includes historical input data and historical output data.

[0055] Specifically, the unilateral anomaly detection algorithm is used to filter historical data to obtain historical anomaly data, including:

[0056] Acquire the historical input data and the historical output data, arrange the historical input data and the historical output data in chronological order, and generate a historical input time series and a historical output time series;

[0057] A first empirical cumulative probability density function is calculated based on the historical integrated time series and the historical integrated time series, and a first differential function of the inverse function of the first empirical cumulative probability density function is calculated based on the first empirical cumulative probability density function, wherein the first differential function is recorded as Δ inv,eCDF (q1);

[0058] A first hypothesis testing algorithm is constructed based on the first differential function, and a first abnormal data threshold is calculated according to the first hypothesis testing algorithm; the data in the historical integral-in and integral-out time series are compared with the first abnormal data threshold one by one, and the historical abnormal data is filtered out according to the comparison result.

[0059] In this embodiment, Δ inv,eCDF (q1) is the difference function of the inverse function of the first empirical cumulative probability density function.

[0060] In this embodiment, the first empirical cumulative probability density function is calculated based on the historical integrated time series and the historical integrated time series. This can be done by statistically analyzing the distribution of the historical integrated data and the historical integrated data at different time points to obtain the probability density function of the data, which reflects the probability of occurrence of the data at different values. Further, by calculating the differential function of the inverse function of the probability density function, the sensitivity of the data change, that is, the influence of the slight change of the data at different values ​​on the overall probability density, is obtained, which is crucial for constructing an effective hypothesis testing algorithm. Next, based on this differential function, a first hypothesis testing algorithm is constructed, which determines whether the data belongs to the abnormal category by comparing the difference in statistical characteristics between the actual data and the historical data. Specifically, a threshold is calculated based on the statistical characteristics of the historical data, which can reflect the normal fluctuation range of the data. Then, the data in the historical integrated time series and the historical integrated time series are compared with this threshold one by one. If the data exceeds the threshold range, it is considered to be abnormal data and filtered out. In this way, the historical data can be effectively filtered using the unilateral anomaly detection algorithm, and the real abnormal data is retained for subsequent analysis and processing.

[0061] Specifically, when calculating the first abnormal data threshold according to the first hypothesis testing algorithm, it includes:

[0062] The first hypothesis testing method was constructed based on 99.5% significance, where the first null hypothesis was set as

[0063] Take the maximum score that satisfies the first null hypothesis as value q 1max As the first abnormal data threshold.

[0064] In this embodiment, 99.5% significance is a commonly used statistical significance level, which is used to determine the rejection region in hypothesis testing.

[0065] In this embodiment, by setting the first null hypothesis as "for all time points T1 in the interval [0,1], and all q1 values ​​less than or equal to q0, the inverse function of the first empirical cumulative probability density function (i.e., Δinv, eCDF(q1)) is less than T1", q0 and q1 represent different quantiles of the historical data. This setting means that we expect that in most cases, that is, at a confidence level of 99.5%, the actual change in historical data will not exceed the threshold of this hypothesis. By constructing a rigorous hypothesis testing method, it is ensured that data will only be identified as abnormal data when it is abnormal to a significant enough degree. Taking the maximum q1 value that meets this hypothesis as the first abnormal data threshold further ensures the rationality and effectiveness of the threshold. This enables the unilateral anomaly detection algorithm to retain the true abnormal data while avoiding misjudging normal data as abnormal data when filtering historical data, thereby improving the accuracy and reliability of data processing.

[0066] Specifically, the zero-sample matching strategy algorithm is adopted, the historical abnormal data is input into the zero-sample matching strategy algorithm, and when the historical matching data is obtained, it includes:

[0067] Set up several whole-in and whole-out strategies;

[0068] Based on the selected round-in and round-out strategy, each of the historical round-in data within the time window is matched with the historical round-out data to obtain the historical matching data.

[0069] In this embodiment, given a set of historical integrated time series BI:=(TI i , QI i ) i , i = 1, 2, ..., N I , and a set of historical time data series BO: = (TO j , QO j ) j , j = 1, 2, ..., N O TI and TO represent timestamps, QI and QO represent dimensionless quantities. Construct the final matching result matrix [M i，j ], whose size is N I Row N O Column. Element M i，j Indicates the number of matches between the jth output data and the ith input data. Sort BI and BO in ascending order according to T. For each input data BI in each BI i , traverse the data in BO, for the kth BO data, if ∑ j M i，j <QI i , then another M i，k =min(QOk , QI i -∑ j M i，j Finally, another QO k =QO k -M i，k . Thus, the final matching result matrix [M i，j ], this matching result matrix is ​​the historical matching data.

[0070] In this embodiment, the zero-sample matching strategy algorithm sets multiple round-in and round-out strategies to match the historical round-in data with the historical round-out data within the time window, thereby obtaining the matching round-out data corresponding to each round-in data. This matching strategy takes into account the dimension and time sequence of the data, ensuring the accuracy of the matching.

[0071] Specifically, the data to be identified is obtained, and the data to be identified is arranged in chronological order to generate a time series of the data to be identified;

[0072] A second empirical cumulative probability density function is calculated based on the time series of the data to be identified, and an inverse function of the second empirical cumulative probability density function is calculated based on the second empirical cumulative probability density function, wherein the first differential function is recorded as Δ inv , e CDF (q2);

[0073] A second hypothesis testing algorithm is constructed based on the second differential function, and a second abnormal data threshold is calculated according to the second hypothesis testing algorithm; the data in the to-be-identified data time series are compared with the second abnormal data threshold one by one, and the identified abnormal data are filtered out according to the comparison result; wherein the identified abnormal data includes identifying abnormal input data and identifying abnormal output data.

[0074] In this embodiment, for the in-and-out time series data to be identified, they are first arranged in chronological order to generate a time series of data to be identified. Subsequently, the second empirical cumulative probability density function is calculated based on these data to be identified, in order to understand the overall distribution characteristics of the data. Next, the second differential function of the inverse function of the second empirical cumulative probability density function is calculated. This function will be used to evaluate the impact of small changes in data at different values ​​on the overall probability density, which is a key step in building an effective hypothesis testing algorithm. Based on this second differential function, a second hypothesis testing algorithm is constructed, which determines whether the data belongs to the abnormal category by comparing the statistical characteristics of the data to be identified with a preset threshold. Specifically, a second abnormal data threshold is calculated based on the statistical characteristics of historical data, and this threshold can reflect the normal fluctuation range of the data to be identified. Then, the data in the time series of the data to be identified are compared with this threshold one by one. If the data exceeds the threshold range, it is considered to be abnormal data, including identifying abnormal in-and-out data and identifying abnormal in-and-out data. These abnormal data will be filtered out for subsequent analysis and processing. Such a processing flow ensures that when identifying abnormal data, the real abnormal data can be retained and normal data can be avoided from being misjudged as abnormal data, thereby improving the accuracy and reliability of data processing.

[0075] Specifically, when calculating the second abnormal data threshold according to the second hypothesis testing algorithm, it includes:

[0076] The second hypothesis testing method was constructed based on 99.5% significance, in which the second null hypothesis was set as

[0077] Take the maximum score that satisfies the second null hypothesis as value q 2max as the second abnormal data threshold.

[0078] In this embodiment, in order to determine the second abnormal data threshold, a 99.5% significance level is used to construct the second hypothesis testing method. The second null hypothesis here sets a condition that for all possible values ​​T2 within the time range, and all q2 values ​​less than or equal to a certain initial threshold q0, the result of the inverse function Δinv,eCDF(q2) of the second empirical cumulative probability density function is less than T1. When the whole-in and whole-out time series data to be identified is compared with this threshold, those truly abnormal data points can be accurately identified. These abnormal data points may represent mutations, noise or errors in the data. By filtering them out, the quality and reliability of the data can be further improved, providing strong support for subsequent data analysis and processing.

[0079] Specifically, collecting expert experience data, performing whole-in-whole-out data modeling according to the historical matching data or the expert experience data, and obtaining a whole-in-whole-out model pool includes:

[0080] The historical matching data or the expert experience data are simulated using a monotonically increasing function model to obtain a monotonically increasing function of the time lag and the cumulative percentage, denoted as P i (dt), i=1, 2, ..., N, the domain of the monotonically increasing function of the time lag and the cumulative integer percentage is from negative infinity to positive infinity, and the range is [0, 1];

[0081] Using the monotonically increasing function of the time lag and the cumulative round-off percentage as a round-in and round-off template function;

[0082] All the whole-in-whole-out template functions are collected to construct a whole-in-whole-out template function set, and the whole-in-whole-out template function set is the whole-in-whole-out model pool.

[0083] In this embodiment, given a historical input data Bi: = (TI, QI), a historical output time series BO: = (TO j , QO j ) j , j = 1, 2, ..., N O , given the whole-in and whole-out matching result matrix [M j ]. Calculate the time delay matrix [DT j ], DT j :=TO j -TI. Calculate the cumulative percentage matrix [PC j ], Use IsotonicModel (monotonic function fitting model) to fit (DT, PC) to obtain function IMi. Finally, define the whole-in and whole-out template function Pi as

[0084]

[0085] Specifically, when generating a whole-in-whole-out model matching algorithm according to the identified abnormal data and the whole-in-whole-out model pool, and obtaining a whole-in-whole-out behavior matching result, it includes:

[0086] Calculate the matching pattern with the minimum distance between the identified abnormal data and the whole-in and whole-out template function set by an optimization algorithm;

[0087] The whole-in and whole-out behavior matching result is obtained according to the minimum distance matching mode.

[0088] In this embodiment, given the whole-in and whole-out model pool P: = {P i}, i = 1, 2, 3, ..., N P, given a set of historical integrated time series BI: = (TI i , QI i ) i , i = 1, 2, ..., N I , and a set of historical time series BO: = (TO j , QO j ) j , j = 1, 2, ..., N O . Calculate the time delay matrix [DT i，j ], DT i，j :=TO j -TI i . Initialize the matching result matrix [M i，k ], size is N I Row N O A zero matrix of columns.

[0089] For a matching guess g i :=([DT i，j ],[M i，j ]), use the same method as step S400 to obtain the guessed matching function For a guess function and a template function P i ,

[0090] Defining metrics

[0091] Define the loss function

[0092] The whole input and output behavior is the matching result matrix That’s what you want.

[0093] Specifically, when the model pool update algorithm is obtained according to the whole-in-whole-out model pool and the whole-in-whole-out behavior matching result, it includes:

[0094] Finding the whole-in-whole-out template function with the shortest distance from the whole-in-whole-out behavior matching result in the whole-in-whole-out model pool;

[0095] Re-sampling the whole-in and whole-out template function, and adding the data of the whole-in and whole-out behavior matching result to form a new whole-in and whole-out template function set;

[0096] The new set of integral-in and integral-out template functions is refitted to obtain updated integral-in and integral-out template functions.

[0097] In this embodiment, for a matching result g i :=([DT i，j ],[M i，j]), find the minimum distance in the whole-in and whole-out template function in the whole-in and whole-out model pool P Take ([t l ],[P * (t l )]), where t l is a uniform sampling point in [0, QI]. Construct a new set of integer-input and integer-output template functions ([[t l ],[DT i，j ]],[[P * (t l )],[M i，j ]]), the whole-in-whole-out template function obtained from this new data set is the original whole-in-whole-out template function P * Updates.

[0098] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0099] The present application is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0100] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0101] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for detecting and identifying abnormal data in a time series, characterized in that: include: Use a unilateral anomaly detection algorithm to filter historical data and obtain historical anomaly data; Using a zero-sample matching strategy algorithm, inputting the historical abnormal data into the zero-sample matching strategy algorithm to obtain historical matching data; Collecting data to be identified, and filtering the data to be identified using the unilateral anomaly detection algorithm to obtain identified anomaly data; Collect expert experience data, perform whole-in-whole-out data modeling based on the historical matching data or the expert experience data, and obtain a whole-in-whole-out model pool; Generate an entire-in-an-out model matching algorithm according to the identified abnormal data and the entire-in-an-out model pool, and obtain an entire-in-an-out behavior matching result; A model pool updating algorithm is obtained according to the whole-in-whole-out model pool and the whole-in-whole-out behavior matching results.

2. The method for detecting and identifying abnormal data in time series according to claim 1, characterized in that: The historical data includes historical input data and historical output data.

3. The method for detecting and identifying abnormal data in time series according to claim 2, characterized in that: The unilateral anomaly detection algorithm is used to filter historical data to obtain historical anomaly data, including: Acquire the historical input data and the historical output data, arrange the historical input data and the historical output data in chronological order, and generate a historical input time series and a historical output time series; A first empirical cumulative probability density function is calculated based on the historical integrated time series and the historical integrated time series, and a first differential function of the inverse function of the first empirical cumulative probability density function is calculated based on the first empirical cumulative probability density function, wherein the first differential function is recorded as Δ inv,eCDF (q1); A first hypothesis testing algorithm is constructed based on the first differential function, and a first abnormal data threshold is calculated according to the first hypothesis testing algorithm; the data in the historical integral-in and integral-out time series are compared with the first abnormal data threshold one by one, and the historical abnormal data is filtered out according to the comparison result.

4. The method for detecting and identifying abnormal data in time series according to claim 3, characterized in that: When calculating the first abnormal data threshold according to the first hypothesis testing algorithm, it includes: The first hypothesis testing method was constructed based on 99.5% significance, where the first null hypothesis was set as Take the maximum score that satisfies the first null hypothesis as value q 1max As the first abnormal data threshold.

5. The method for detecting and identifying abnormal data in time series according to claim 4, characterized in that: The zero-sample matching strategy algorithm is adopted, and the historical abnormal data is input into the zero-sample matching strategy algorithm to obtain the historical matching data, including: Set up several whole-in and whole-out strategies; Based on the selected round-in and round-out strategy, each of the historical round-in data within the time window is matched with the historical round-out data to obtain the historical matching data.

6. The method for detecting and identifying abnormal data in time series according to claim 5, characterized in that: Acquire the data to be identified, arrange the data to be identified in chronological order, and generate a time series of the data to be identified; A second empirical cumulative probability density function is calculated based on the time series of the data to be identified, and an inverse function of the second empirical cumulative probability density function is calculated based on the second empirical cumulative probability density function, wherein the first differential function is recorded as Δ inv,eCDF (q2); constructing a second hypothesis testing algorithm based on the second difference function, and calculating a second abnormal data threshold according to the second hypothesis testing algorithm; The data in the time series of the data to be identified are compared one by one with the second abnormal data threshold, and the identified abnormal data are filtered out according to the comparison result; wherein the identified abnormal data includes identified abnormal input data and identified abnormal output data.

7. The method for detecting and identifying abnormal data in time series according to claim 6, characterized in that: When calculating the second abnormal data threshold according to the second hypothesis testing algorithm, it includes: The second hypothesis testing method was constructed based on 99.5% significance, in which the second null hypothesis was set as Take the maximum score that satisfies the second null hypothesis as value q 2max as the second abnormal data threshold.

8. The method for detecting and identifying abnormal data in time series according to claim 7, characterized in that: Collecting expert experience data, performing whole-in-whole-out data modeling based on the historical matching data or the expert experience data, and obtaining a whole-in-whole-out model pool includes: The historical matching data or the expert experience data is fitted using a monotonically increasing function model to obtain a monotonically increasing function of the time lag and the cumulative round-off percentage, denoted as P i (dt), i = 1, 2, ..., N, the domain of the monotonically increasing function of the time lag and the cumulative integer percentage is from negative infinity to positive infinity, and the range is [0, 1]; Using the monotonically increasing function of the time lag and the cumulative round-off percentage as a round-in and round-off template function; All the whole-in-whole-out template functions are collected to construct a whole-in-whole-out template function set, and the whole-in-whole-out template function set is the whole-in-whole-out model pool.

9. The method for detecting and identifying abnormal data in time series according to claim 8, characterized in that: When generating an entire-in-an-out model matching algorithm according to the identified abnormal data and the entire-in-an-out model pool, and obtaining an entire-in-an-out behavior matching result, it includes: Calculate the matching pattern with the minimum distance between the identified abnormal data and the whole-in and whole-out template function set by an optimization algorithm; The whole-in and whole-out behavior matching result is obtained according to the minimum distance matching mode.

10. The method for detecting and identifying abnormal data in time series according to claim 9, characterized in that: When the model pool update algorithm is obtained according to the whole-in-whole-out model pool and the whole-in-whole-out behavior matching result, it includes: Finding the whole-in-whole-out template function with the shortest distance from the whole-in-whole-out behavior matching result in the whole-in-whole-out model pool; Re-sampling the whole-in and whole-out template function, and adding the data of the whole-in and whole-out behavior matching result to form a new whole-in and whole-out template function set; The new set of integral-in and integral-out template functions is refitted to obtain updated integral-in and integral-out template functions.

Images