Abnormal data analysis method and device, computer equipment and readable storage medium

Abstract

The invention relates to an abnormal data analysis method and device, computer equipment and a readable storage medium. The method comprises the steps of obtaining multi-dimensional time sequence data at different time points, converting the multi-dimensional time sequence data into a multi-dimensional time sequence vector, determining the distance between the multi-dimensional time sequence vector and a reference distribution vector at each time point, constructing a one-dimensional time sequence matrix based on the distance based on the distance corresponding to each time point, and obtaining a one-dimensional time sequence matrix based on the one-dimensional time sequence matrix. And determining a Z score value of each data point in the one-dimensional time sequence matrix, and identifying an abnormal value in the data points based on the Z score value of each data point. By adopting the method, the abnormal value in the multi-dimensional time sequence data can be efficiently and accurately identified.

Description

Technical Field

[0001] The present application relates to the field of data detection technology, and in particular to an abnormal data analysis method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the rapid development of information technology, society has entered the era of big data, which not only marks the exponential growth of data volume, but also means that the frequency and efficiency of data acquisition have reached new heights. At present, in the business environment, data collection is no longer limited to traditional data collection methods, but can capture and update various types of data in near real time, thereby more accurately reflecting the dynamic changes of real business.

[0003] Since there may be a lot of noise and outliers in multidimensional time series, these data will affect the efficiency and accuracy of data analysis. At present, the processing and analysis of multidimensional time series data usually relies on the analyst's rich experience and professional knowledge, and requires the user to have high professional knowledge and skills. In the face of the analysis of massive multidimensional time series data, the above solutions seem to be inadequate and it is difficult to achieve efficient and accurate data analysis. Summary of the invention

[0004] Based on this, it is necessary to provide an efficient abnormal data analysis method, device, computer equipment, computer readable storage medium and computer program product to address the above technical problems.

[0005] In a first aspect, the present application provides a method for analyzing abnormal data, comprising:

[0006] Acquire multidimensional time series data at different time points, and convert the multidimensional time series data into a multidimensional time series vector;

[0007] Determine the distance between the multidimensional time series vector and the reference distribution vector at each time point, and construct a distance-based one-dimensional time series matrix based on the distance corresponding to each time point;

[0008] Determine a Z score value for each data point in the one-dimensional time series matrix;

[0009] Based on the Z-score value of each data point, outliers among the data points are identified.

[0010] In one embodiment, determining the distance between the multidimensional time series vector and the reference distribution vector at each time point, and constructing a distance-based one-dimensional time series matrix based on the distance corresponding to each time point, includes:

[0011] Determine the Wasserstein distance between the multidimensional time series vector and a benchmark distribution vector at each time point;

[0012] Based on the Wasserstein distance corresponding to each time point, a one-dimensional time series matrix based on Wasserstein distance is constructed.

[0013] In one embodiment, determining the Z score value of each data point in the one-dimensional time series matrix includes:

[0014] For each time point, the local mean and local standard deviation of the data points in the window are determined according to the preset sliding window size;

[0015] A Z score value of each data point is determined based on the local mean and the local standard deviation.

[0016] In one embodiment, the identifying of outliers in the multidimensional time series data based on the Z-score value of each data point includes:

[0017] Compare the Z score value of each data point with the preset score threshold;

[0018] If the Z score value of the data point is greater than the preset score threshold, it is determined that the data point is abnormal.

[0019] In one embodiment, after converting the multidimensional time series data into a multidimensional time series vector, the method further comprises:

[0020] Normalizing the multidimensional time series vector to obtain a target multidimensional time series vector;

[0021] The step of determining the distance between the multidimensional time series vector and the reference distribution vector at each time point, and constructing a distance-based one-dimensional time series matrix based on the distance corresponding to each time point, comprises:

[0022] The distance between the target multidimensional time series vector and the reference distribution vector at each time point is determined, and a one-dimensional time series matrix based on the distance is constructed based on the distance corresponding to each time point.

[0023] In one embodiment, after identifying the outliers in the data points, the method further comprises:

[0024] Get the smoothing coefficient;

[0025] The abnormal value is smoothed according to the smoothing coefficient.

[0026] In a second aspect, the present application also provides an abnormal data analysis device, comprising:

[0027] A data conversion module is used to obtain multidimensional time series data at different time points and convert the multidimensional time series data into a multidimensional time series vector;

[0028] The data dimension reduction module is used to determine the distance between the multidimensional time series vector and the benchmark distribution vector at each time point, and to construct a distance-based one-dimensional time series matrix based on the distance corresponding to each time point;

[0029] A score value determination module is used to determine the Z score value of each data point in the one-dimensional time series matrix;

[0030] The abnormal data identification module is used to identify abnormal values ​​in the data points based on the Z score value of each data point.

[0031] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0032] Acquire multidimensional time series data at different time points, and convert the multidimensional time series data into a multidimensional time series vector;

[0033] Determine the distance between the multidimensional time series vector and the reference distribution vector at each time point, and construct a distance-based one-dimensional time series matrix based on the distance corresponding to each time point;

[0034] Determine a Z score value for each data point in the one-dimensional time series matrix;

[0035] Based on the Z-score value of each data point, outliers among the data points are identified.

[0036] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0037] Acquire multidimensional time series data at different time points, and convert the multidimensional time series data into a multidimensional time series vector;

[0038] Determine the distance between the multidimensional time series vector and the reference distribution vector at each time point, and construct a distance-based one-dimensional time series matrix based on the distance corresponding to each time point;

[0039] Determine a Z score value for each data point in the one-dimensional time series matrix;

[0040] Based on the Z-score value of each data point, outliers among the data points are identified.

[0041] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0042] Acquire multidimensional time series data at different time points, and convert the multidimensional time series data into a multidimensional time series vector;

[0043] Determine the distance between the multidimensional time series vector and the reference distribution vector at each time point, and construct a distance-based one-dimensional time series matrix based on the distance corresponding to each time point;

[0044] Determine a Z score value for each data point in the one-dimensional time series matrix;

[0045] Based on the Z-score value of each data point, outliers among the data points are identified.

[0046] The above-mentioned abnormal data analysis method, device, computer equipment, computer-readable storage medium and computer program product, by converting multidimensional time series data into multidimensional time series vectors, and calculating the distance between the multidimensional time series vector and the reference distribution vector at each time point, simplifies the multidimensional time series data into a one-dimensional time series matrix, greatly reduces the complexity of the data, and can more comprehensively measure the difference and distance between discrete distribution and continuous distribution, and take into account the geometric morphological characteristics of the distribution itself, making subsequent analysis easier and more efficient, further, by calculating the Z-Score value of each data point in the one-dimensional time series matrix, it is possible to quantify the degree of deviation of each data point from the local mean, retaining the correlation between the data, thereby more comprehensively discovering the structural changes in the data distribution, so as to accurately identify the outliers in the data points. In summary, by adopting the above-mentioned scheme, outliers in multidimensional time series data can be efficiently and accurately identified. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments of the present application or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 A diagram of an application environment of an abnormal data analysis method in an embodiment;

[0049] Figure 2 A schematic diagram of a process of analyzing abnormal data in one embodiment;

[0050] Figure 3is a flow chart of an abnormal data analysis method in another embodiment;

[0051] Figure 4 is a flowchart of an abnormal data analysis method in yet another embodiment;

[0052] Figure 5 A schematic diagram of a flow chart of an abnormal data analysis method in yet another embodiment;

[0053] Figure 6 A schematic diagram of the result of the Wasserstein distance between a multidimensional time series vector and a reference distribution vector in one embodiment;

[0054] Figure 7 A schematic diagram of the results of normal data distribution and abnormal data analysis in one embodiment;

[0055] Figure 8 is a structural block diagram of an abnormal data analysis device in one embodiment;

[0056] Fig. 9 is a structural block diagram of an abnormal data analysis device in another embodiment;

[0057] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0059] The abnormal data analysis method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the data acquisition terminal 102 communicates with the server 104 through a network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers.

[0060] Specifically, the data acquisition terminal 102 may continuously acquire multidimensional time series data at different time points, and upload the acquired multidimensional time series data to the server 104. After receiving the multidimensional time series data, the server 104 converts the multidimensional time series data into a multidimensional time series vector, and then determines the distance between the multidimensional time series vector and the benchmark distribution vector at each time point, and constructs a distance-based one-dimensional time series matrix based on the distance corresponding to each time point. Next, the Z score value of each data point in the one-dimensional time series matrix is ​​determined, and finally, based on the Z score value of each data point, the outliers in the data points are identified.

[0061] The data collection terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. IoT devices may be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, projection devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices may be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server 104 may be an independent physical server, or a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.

[0062] In an exemplary embodiment, Figure 2 As shown, a method for analyzing abnormal data is provided, which is applied to Figure 1 The server 104 in FIG. 1 is used as an example to illustrate the method, which includes the following steps S200 to S800. Among them:

[0063] S200, acquiring multidimensional time series data at different time points, and converting the multidimensional time series data into a multidimensional time series vector.

[0064] Multidimensional time series data refers to time series data that is affected by multiple factors. These data contain multiple indicators, influencing factors or variables. Multidimensional time series data is usually used to describe the state of a system or object at different time points. Each dimension can represent different characteristics or attributes. A multidimensional time series vector is obtained by representing the multidimensional data at each time point as a vector.

[0065] For example, taking the monitoring data collected by the monitoring system of the cloud host authorized by the user as an example, the data not only covers various performance indicators of the user in the process of using the cloud host, but also contains rich information in the spatial dimension and the time dimension. Specifically, for each cloud host space (such as different servers, data centers or cloud areas) and each user who uses these spaces, their cloud host usage data constitutes a unique time series data. These time series data not only record the usage behavior of each user in each space, but also reveal the trends and patterns of these behaviors over time. For example, the monitored indicators include CPU utilization, memory utilization, disk I / O rate and network bandwidth utilization. The monitoring data at each time point is a multi-dimensional time series data point, containing information in these four dimensions.

[0066] After receiving the multidimensional time series data, the server may regard the data in different spaces at each time point as multidimensional vectors, and convert the multidimensional time series data into multidimensional time series vectors.

[0067] Specifically, the original multidimensional time series data can be converted into vector form. , where n represents the number of different spaces in the spatial dimension, t represents different time dates, and m represents the maximum value of the observation time. The multidimensional time series vector obtained by transformation is as follows:

[0068]

[0069] Obtained by the above method is the final multidimensional time series vector at different time points.

[0070] S400, determining the distance between the multidimensional time series vector and the reference distribution vector at each time point, and constructing a distance-based one-dimensional time series matrix based on the distance corresponding to each time point.

[0071] The reference distribution vector may also be referred to as a reference distribution, which is usually a reference point or a reference distribution for comparison with the multidimensional time series vector at each time point. In this embodiment, the reference distribution vector may be the average distribution of the sequence data at all time points. It is understood that in other embodiments, the reference distribution may also be the distribution at a specific time point, which is not limited here. The one-dimensional time series matrix is ​​a new time series formed by the distance between each time point and the reference distribution vector.

[0072] In specific implementation, the average distribution of all time points can be determined in advance and determined as the reference distribution vector Then, the distance between the multidimensional time series vector and the benchmark distribution vector at each time point is measured and quantified by a preset distance calculation formula. Specifically, the distance calculation formula may include but is not limited to Manhattan distance, Euclidean distance, etc. Then, based on the distance, a distance-based time series matrix is ​​constructed. . Each element in the matrix represents the distance between the multidimensional time series vector and the benchmark distribution vector.

[0073] S600, determining the Z score value of each data point in the one-dimensional time series matrix.

[0074] The Z-score is a standardized score that indicates the degree of deviation of a data point from the local mean. The Z-score can be calculated by dividing the difference between the data point and the local mean by the local standard deviation.

[0075] In specific implementation, after constructing a one-dimensional time series matrix, the Z score value of each data point in the one-dimensional time series matrix can be determined through a sliding window.

[0076] S800, identifying outliers among the data points based on the Z-score value of each data point.

[0077] In practical applications, a score threshold can be pre-set based on data characteristics and actual conditions to determine whether a data point is an outlier. After determining the Z score value of each data point in the one-dimensional time series matrix, the data point can be judged as an outlier based on the preset score threshold, thereby identifying outliers among all data points.

[0078] The above abnormal data analysis method, by converting the multidimensional time series data into a multidimensional time series vector and calculating the distance between the multidimensional time series vector and the reference distribution vector at each time point, simplifies the multidimensional time series data into a one-dimensional time series matrix, greatly reduces the complexity of the data, and can more comprehensively measure the difference and distance between discrete distribution and continuous distribution, and takes into account the geometric morphological characteristics of the distribution itself, making subsequent analysis easier and more efficient. Furthermore, by calculating the Z-Score value of each data point in the one-dimensional time series matrix, the degree of deviation of each data point from the local mean can be quantified, and the correlation between the data is retained, so as to more comprehensively discover the structural changes in the data distribution, so as to accurately identify the outliers in the data points. In summary, by adopting the above scheme, outliers in multidimensional time series data can be efficiently and accurately identified.

[0079] In an exemplary embodiment, Figure 3 As shown, S400, including:

[0080] S420, determining the Wasserstein distance between the multidimensional time series vector and the benchmark distribution vector at each time point, and constructing a one-dimensional time series matrix based on the Wasserstein distance based on the Wasserstein distance corresponding to each time point.

[0081] Wasserstein distance is a method to measure the distance between two probability distributions. It can be used to compare the similarity or difference between a multidimensional time series vector and a benchmark distribution vector.

[0082] In this embodiment, the distance between the multidimensional time series vector and the reference distribution vector at each time point may be measured by the Wasserstein distance. Specifically, the distance between the multidimensional time series vector and the reference distribution vector at each time point may be measured according to the calculation formula of the Wasserstein distance. The calculation formula of the Wasserstein distance is as follows:

[0083]

[0084] Through the above distance calculation formula, the distance between the multidimensional time series vector and the benchmark distribution vector at each time point is obtained. Then, the Wasserstein distance corresponding to each time point is integrated to construct a time series matrix based on distance. , that is, each element in the matrix represents the distance between the multidimensional time series vector at a time point and the reference distribution vector. For example, if the time points include time point 1, time point 2, time 3, and time point 4, the distances between the multidimensional time series vector at time point 1, time point 2, time 3, and time point 4 and the reference distribution vector are calculated by the Wasserstein distance formula to be 0.5, 0.581, 0.2, and 0.1, respectively. Then, the distance-based time series matrix constructed is: [0.5, 0.581, 0.2, 0.1]. It can be understood that in other embodiments, other distance metrics such as the Euclidean distance formula can be used to measure the distance between the multidimensional time series vector at each time point and the reference distribution vector.

[0085] In this embodiment, by using the Wasserstein distance to measure the distance between the multidimensional time series vector and the benchmark distribution vector at each time point, the multidimensional time series is reduced to a one-dimensional time series. This can more comprehensively measure the difference and distance between discrete distribution and continuous distribution, and take into account the geometric morphological characteristics of the distribution itself, which is conducive to the detection of abnormal data.

[0086] like Figure 3 As shown, in an exemplary embodiment, S600 includes: S620, for each time point, according to a preset sliding window size, determining the local mean and local standard deviation of the data points in the window, and determining the Z score value of each data point according to the local mean and local standard deviation.

[0087] The sliding window size is the window size used to calculate the local mean and standard deviation. In this embodiment, the sliding window size T can be set to 7. It is understandable that in other embodiments, the sliding window size can also be 3 or other values, which can be determined according to actual conditions and is not limited here.

[0088] In specific implementation, for a one-dimensional time series matrix, the local mean and local standard deviation of the data points in the window are calculated according to the sliding window size, and then the Z-Score value of each time point is calculated based on the local mean and local standard deviation. Specifically, the determination of the Z-Score value can be to first determine the difference between the value of the data point and the local mean, and then determine the ratio of the difference to the local standard deviation as the Z-Score value. In other embodiments, for the current data point, the deviation between the next time point and the mean of the previous window can be used as the Z-Score value.

[0089] For example, if the sliding window size T=3 and the data in the sliding window is [0,15.81,20], the local mean is approximately equal to 11.94, the local standard deviation is approximately equal to 9.12, and for the data point at time point 3, the Z-Score value is (20−11.94) / 9.12≈0.88.

[0090] In this embodiment, by calculating the local mean and the local standard deviation within the sliding window, the changing trend of the data in the local range can be reflected, so that the Z-Score value can better adapt to the dynamic changes of the data, especially when there are trends or seasonal changes in the data, and can capture subtle changes in the data in the local range, so that the Z-Score value can more sensitively reflect the abnormal points and reduce false positives (misjudging normal points as abnormal points) and false negatives (misjudging abnormal points as normal points).

[0091] like Figure 3 As shown, in an exemplary embodiment, S800 includes: S820, comparing the Z score value of each data point with a preset score threshold, if the Z score value of the data point is greater than the preset score threshold, it is determined that the data point is abnormal.

[0092] In practical applications, in order to efficiently and accurately identify abnormal points in data points, a specified score threshold can be pre-set. In specific implementation, after determining the Z-Score value of each data point, the Z-Score value of each data point can be compared with the pre-set score threshold. If the Z-Score value of the data point is greater than the preset score threshold, it is determined that the data point is abnormal.

[0093] Specifically, the process of identifying abnormal data points can be:

[0094] 1) Let the data in the window for: ;

[0095] 2) Calculate the mean ;

[0096] 3) Calculate standard deviation ;

[0097] 4) Cycle T+1 to n for k;

[0098] 5) If , then calculate:

[0099]

[0100]

[0101]

[0102] like ,but ,otherwise ;

[0103] 6) If , then calculate:

[0104]

[0105]

[0106]

[0107]

[0108] 7) Identify outliers ;

[0109] in, is the smoothing coefficient, is the sliding window size, is the score threshold.

[0110] Exemplarily, the score threshold may be set to 3. It is understandable that in other embodiments, the score threshold may also be 2, 4 or other values, which may be determined according to actual conditions and are not limited here.

[0111] In this embodiment, by determining the Z score of each data point, the original data is converted into a value with standard normal distribution characteristics, so that different data sets or data of different dimensions can be directly compared, and it is easy to identify which data points are outliers. In addition, the score threshold can be flexibly set according to actual conditions, which is more applicable.

[0112] Since each feature (dimension) in multidimensional time series data may have a different value range, in order to make each feature comparable and analyzed on the same scale, the multidimensional time series vector can be normalized. Figure 4 As shown, in an exemplary embodiment, after converting the multidimensional time series data into a multidimensional time series vector, the method further includes:

[0113] S300, normalizing the multidimensional time series vector to obtain a target multidimensional time series vector.

[0114] S420, including: S422, determining the Wasserstein distance between the target multidimensional time series vector and the benchmark distribution vector at each time point, and constructing a one-dimensional time series matrix based on the Wasserstein distance based on the Wasserstein distance corresponding to each time point.

[0115] In practical applications, the multidimensional time series vector can be normalized by using minimum-maximum normalization, Z-score normalization or other normalization methods. That is, the selected normalization method converts the value of each feature into a normalized value.

[0116] Taking minimum-maximum normalization as an example, the value of each feature in the multidimensional time series vector can be scaled to a fixed interval, usually [0,1]. Taking Z-score normalization as an example, the value of each feature is converted to a standard normal distribution, that is, the mean is 0 and the standard deviation is 1. Then, based on the mean and standard deviation of each feature, the normalization formula is applied to normalize each feature to the same scale. Next, the Wasserstein distance between the target multidimensional time series vector and the benchmark distribution vector at each time point is determined, and based on the Wasserstein distance corresponding to each time point, a one-dimensional time series matrix based on the Wasserstein distance is constructed.

[0117] In this embodiment, by normalizing the multidimensional time series vector, the value of each feature in the vector can be normalized to the same scale, which is beneficial to the subsequent analysis and detection of abnormal data.

[0118] like Figure 5 As shown, in an exemplary embodiment, after S820, the method further includes:

[0119] S900, obtaining a smoothing coefficient, and performing smoothing processing on the abnormal value according to the smoothing coefficient.

[0120] The smoothing coefficient is a parameter used to adjust the degree of smoothing. It is usually used to smooth data in time series analysis to reduce noise and fluctuations and make trends more obvious. The smoothing coefficient is usually represented by the word α, and the value range is between 0 and 1. Different smoothing coefficients will result in different degrees of smoothing effect. Specifically, when α is close to 0, the smoothing effect is stronger, the historical data has a greater impact on the smoothed value, and the new data points have a smaller impact on the smoothed value. When α is close to 1, the smoothing effect is weaker, the new data points have a greater impact on the smoothed value, and the historical data have a smaller impact on the smoothed value.

[0121] In practical applications, a suitable smoothing coefficient α may be set in advance according to the application scenario and requirements. In this embodiment, the smoothing coefficient may be set to 0.5. It is understood that in other embodiments, the smoothing coefficient may also be set to other values ​​such as 0.55 according to specific circumstances, and this is not a sole limitation.

[0122] For example, the exponential smoothing method is used to calculate the smoothed value at each time point. The exponential smoothing formula can be as follows:

[0123]

[0124] Among them, St is the smoothed value at the tth time point, Y t is the actual observed value at the tth time point, It is In the smoothing process, if a data point at a certain time point is detected to be an outlier, the outlier can be replaced by the smoothed value at that time point, and then the smoothing calculation can be performed. In some other embodiments, the outlier can be skipped and the previous smoothed value can be used. as the smoothed value at that time point.

[0125] For example, if there is a time series data Y=[10,12,15,20,100,18,22], where Y5=100 is considered an outlier. Set the smoothing coefficient α=0.5 and use the method of replacing outliers for smoothing. The smoothing process is as follows:

[0126] Initialize the smoothed values ​​S1=Y1=10, S2=0.5×12+0.5×10=11, S3=0.5×15+0.5×11=13, S4=0.5×20+0.5×13=16.5. Process the abnormal values ​​Y5=S4=16.5, S5=0.5×16.5+0.5×16.5=16.5, and continue to calculate the smoothed values, S6=0.5×18+0.5×16.5=17.25, S7=0.5×22+0.5×17.25=19.625, so the smoothed value sequence obtained after smoothing is: S=[10,11,13,16.5,16.5,17.25,19.625].

[0127] In this embodiment, by smoothing the outliers, the correlation between the data can be retained, so that the structural changes in the data distribution can be more comprehensively discovered, and the impact on subsequent outlier detection can be reduced.

[0128] In order to explain the abnormal data analysis method provided by the present application more clearly, a specific embodiment is described below, and the specific embodiment includes the following steps:

[0129] Step 1: Obtain multidimensional time series data at different time points, and convert the multidimensional time series data into a multidimensional time series vector.

[0130] Step 2: Normalize the multidimensional time series vector to obtain the target multidimensional time series vector.

[0131] Step 3: Determine the Wasserstein distance between the target multidimensional time series vector and the benchmark distribution vector at each time point, and construct a one-dimensional time series matrix based on the Wasserstein distance based on the Wasserstein distance corresponding to each time point.

[0132] Step 4: For each time point, determine the local mean and local standard deviation of the data points in the window according to the preset sliding window size, and determine the Z score value of each data point according to the local mean and local standard deviation.

[0133] Step 5: Compare the Z score value of each data point with the preset score threshold. If the Z score value of the data point is greater than the preset score threshold, it is determined that the data point is abnormal.

[0134] Specifically, the target multidimensional time series vector is obtained by treating the data at the same time point in different spaces as a multidimensional time series vector and normalizing it. The Wasserstein distance between the target multidimensional time series vector and the benchmark vector at each time point is calculated to form a new one-dimensional time series, such as Figure 6 shown.

[0135] Then, for the newly generated one-dimensional time series, set appropriate parameters. Specifically, the sliding window size T can be set to T=7, and the score threshold , smoothing coefficient α=0.5. According to the Smoothed Z-Score anomaly detection method, it can be seen that the detected anomalies have obvious characteristics from the graphical structure. The normal data analysis and abnormal data distribution can be shown as follows: Figure 7 shown.

[0136] As can be seen from the figure, the anomaly detection method based on Wasserstein distance and Smoothed Z-Score method is effective and interpretable in business.

[0137] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0138] Based on the same inventive concept, the embodiment of the present application also provides an abnormal data analysis device for implementing the abnormal data analysis method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more abnormal data analysis device embodiments provided below can refer to the limitations on the abnormal data analysis method above, and will not be repeated here.

[0139] In an exemplary embodiment, Figure 8 As shown, an abnormal data analysis device 800 is provided, comprising: a data conversion module 810, a data dimension reduction module 820, a score value determination module 830 and an abnormal data identification module 840, wherein:

[0140] The data conversion module 810 is used to obtain multidimensional time series data at different time points and convert the multidimensional time series data into a multidimensional time series vector.

[0141] The data dimension reduction module 820 is used to determine the distance between the multidimensional time series vector and the reference distribution vector at each time point, and to construct a distance-based one-dimensional time series matrix based on the distance corresponding to each time point.

[0142] The score value determination module 830 is used to determine the Z score value of each data point in the one-dimensional time series matrix.

[0143] The abnormal data identification module 840 is used to identify abnormal values ​​in the data points based on the Z score value of each data point.

[0144] The above-mentioned abnormal data analysis device, by converting the multidimensional time series data into a multidimensional time series vector and calculating the distance between the multidimensional time series vector and the reference distribution vector at each time point, simplifies the multidimensional time series data into a one-dimensional time series matrix, greatly reduces the complexity of the data, and can more comprehensively measure the difference and distance between discrete distribution and continuous distribution, and takes into account the geometric morphological characteristics of the distribution itself, making subsequent analysis easier and more efficient. Furthermore, by calculating the Z-Score value of each data point in the one-dimensional time series matrix, the degree of deviation of each data point from the local mean can be quantified, and the correlation between the data is retained, thereby more comprehensively discovering the structural changes in the data distribution, so as to accurately identify the abnormal values ​​in the data points. In summary, the above-mentioned device can efficiently and accurately identify abnormal values ​​in multidimensional time series data.

[0145] In an exemplary embodiment, the data dimension reduction module 820 is also used to determine the Wasserstein distance between the multidimensional time series vector and the benchmark distribution vector at each time point, and to construct a one-dimensional time series matrix based on the Wasserstein distance based on the Wasserstein distance corresponding to each time point.

[0146] In an exemplary embodiment, the score value determination module 830 is also used to determine, for each time point, the local mean and local standard deviation of the data points in the window according to a preset sliding window size, and determine the Z score value of each data point according to the local mean and local standard deviation.

[0147] In an exemplary embodiment, the abnormal data identification module 840 is further used to compare the Z score value of each data point with a preset score threshold. If the Z score value of the data point is greater than the preset score threshold, it is determined that the data point is abnormal.

[0148] like Fig. 9 As shown, in an exemplary embodiment, the device also includes a normalization processing module 812, which is used to normalize the multidimensional time series vector to obtain a target multidimensional time series vector; the data dimension reduction module 820 is also used to determine the distance between the target multidimensional time series vector and the reference distribution vector at each time point, and construct a distance-based one-dimensional time series matrix based on the distance corresponding to each time point.

[0149] In an exemplary embodiment, the apparatus further includes a smoothing processing module 850 for acquiring a smoothing coefficient and performing smoothing processing on the abnormal value according to the smoothing coefficient.

[0150] Each module in the above abnormal data analysis device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0151] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig.10As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as multidimensional time series data and multidimensional time series vectors. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an abnormal data analysis method is implemented.

[0152] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0153] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps in any one of the above-mentioned abnormal data analysis method embodiments are implemented.

[0154] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in any one of the above-mentioned abnormal data analysis method embodiments are implemented.

[0155] In one embodiment, a computer program product is provided, including a computer program, which, when executed by a processor, implements the steps in any one of the above-mentioned abnormal data analysis method embodiments.

[0156] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0157] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0158] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0159] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for analyzing abnormal data, characterized in that: The method comprises: Acquire multidimensional time series data at different time points, and convert the multidimensional time series data into a multidimensional time series vector; Determine the distance between the multidimensional time series vector and the reference distribution vector at each time point, and construct a distance-based one-dimensional time series matrix based on the distance corresponding to each time point; Determine a Z score value for each data point in the one-dimensional time series matrix; Based on the Z-score value of each data point, outliers among the data points are identified.

2. The method according to claim 1, characterized in that The step of determining the distance between the multidimensional time series vector and the reference distribution vector at each time point, and constructing a distance-based one-dimensional time series matrix based on the distance corresponding to each time point, comprises: Determine the Wasserstein distance between the multidimensional time series vector and a benchmark distribution vector at each time point; Based on the Wasserstein distance corresponding to each time point, a one-dimensional time series matrix based on Wasserstein distance is constructed.

3. The method according to claim 1, characterized in that Determining the Z score value of each data point in the one-dimensional time series matrix includes: For each time point, the local mean and local standard deviation of the data points in the window are determined according to the preset sliding window size; A Z score value of each data point is determined based on the local mean and the local standard deviation.

4. The method according to any one of claims 1 to 3, characterized in that: The identifying of outliers in the multidimensional time series data based on the Z score value of each data point includes: Compare the Z score value of each data point with the preset score threshold; If the Z score value of the data point is greater than the preset score threshold, it is determined that the data point is abnormal.

5. The method according to any one of claims 1 to 3, characterized in that: After converting the multidimensional time series data into a multidimensional time series vector, the method further includes: Normalizing the multidimensional time series vector to obtain a target multidimensional time series vector; The step of determining the distance between the multidimensional time series vector and the reference distribution vector at each time point, and constructing a distance-based one-dimensional time series matrix based on the distance corresponding to each time point, comprises: The distance between the target multidimensional time series vector and the reference distribution vector at each time point is determined, and a one-dimensional time series matrix based on the distance is constructed based on the distance corresponding to each time point.

6. The method according to any one of claims 1 to 3, characterized in that: After identifying the outliers in the data points, the method further includes: Get the smoothing coefficient; The abnormal value is smoothed according to the smoothing coefficient.

7. An abnormal data analysis device, characterized in that: The device comprises: A data conversion module, used to obtain multidimensional time series data at different time points, and convert the multidimensional time series data into a multidimensional time series vector; A data dimension reduction module, used to determine the distance between the multidimensional time series vector and the reference distribution vector at each time point, and to construct a distance-based one-dimensional time series matrix based on the distance corresponding to each time point; A score value determination module, used to determine the Z score value of each data point in the one-dimensional time series matrix; The abnormal data identification module is used to identify abnormal values ​​in the data points based on the Z score value of each data point.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images