A method, apparatus, device, and readable medium for detecting anomalies in monitoring data.

Abstract

The present invention discloses a method for detecting abnormalities in monitoring data, including: collecting monitoring data, and segmenting the monitoring data based on a sliding window to obtain a training set and a test set / data to be detected; extracting wavelet packet decomposition features and statistical features of sample points in the training set to construct a feature set of monitoring data, and selecting features in the feature set based on a chi-square test to exclude redundant features to obtain a final training set; training an integrated learning model based on the final training set to obtain an abnormality detection model; and inputting the test set / data to be detected into the abnormality detection model to obtain an abnormality detection result. The present invention also discloses a detection device, a computer device, and a readable storage medium for abnormal monitoring data. The present invention utilizes wavelet packet decomposition and statistics to extract instantaneous changes and overall changes in time series data, and performs feature selection and enhancement, thereby enhancing the generalization ability of the abnormality detection model and improving the accuracy of abnormality detection of single-dimensional monitoring data.

Description

Technical Field

[0001] The present invention relates to the technical field of cloud computing platforms, and in particular to a method, device, equipment and readable medium for detecting abnormalities in monitoring data. Background Art

[0002] Cloud platform, also known as cloud computing platform, refers to services based on hardware resources and software resources, providing computing, network and storage capabilities. The cloud platform monitoring system continuously collects a large number of time-series KPIs from the platform, such as CPU usage, network throughput, etc., to determine the operating status of the cloud platform.

[0003] As cloud platform technology becomes more and more mature, the scale of cloud platforms has increased from the initial few or dozens of machines to hundreds or even thousands of machines. At the same time, the cloud platform's own services are increasing and the calls are becoming more and more complex, making the monitoring data massive and complex.

[0004] In traditional cloud platforms, abnormal monitoring of KPIs mostly uses a threshold method, that is, the operation and maintenance personnel set the threshold based on experience, and when the KPI data reaches the preset threshold, an abnormal alarm is generated. However, in actual application, it is found that the threshold setting is too dependent on experience, and it is difficult to accurately set the threshold for complex KPIs. If the threshold is too high, abnormal situations will be missed, and quality risks will be difficult to find; if the threshold is too low, it will often trigger an alarm storm and interfere with the judgment of the operation and maintenance personnel. In addition, for some data jitters but below the threshold, it cannot be detected in this way, resulting in missed reports.

[0005] In summary, the method of determining anomalies based solely on thresholds cannot meet the needs of anomaly detection in cloud platforms, which requires us to use machine learning and data mining technologies for automatic anomaly detection. However, in real cloud platform monitoring, the probability of anomalies is relatively low, and it is difficult to accumulate a large number of abnormal samples. This leads to the need for the ability to automatically build samples in the machine learning process framework to generate sufficient data to support the training of anomaly detection models. In addition, due to the complexity of time series monitoring data, there are characteristics of time series data, that is, it depends on time changes and uses numerical values ​​to reflect the degree of change. This change can have multiple modes. Common time series data models include stable, fluctuating, and periodic. In the monitoring system, the data mode changes with the business changes, which requires that in the anomaly detection of time series monitoring data, the detection model has sufficient generalization ability to detect anomalies in different data modes. Summary of the invention

[0006] In view of this, the purpose of the embodiments of the present invention is to propose a method, device, equipment and readable medium for detecting anomalies in monitoring data, use wavelet packet decomposition features and statistical features to extract instantaneous changes and overall changes in time series data, and use chi-square test to select features, thereby eliminating the influence of redundant features, and then use adaptive oversampling method to enhance the feature sample set, thereby enhancing the generalization ability of the anomaly detection model and improving the accuracy of anomaly detection in one-dimensional monitoring data.

[0007] Based on the above purpose, one aspect of an embodiment of the present invention provides a method for detecting anomalies in monitoring data, comprising the following steps: collecting monitoring data, and segmenting the monitoring data based on a sliding window to obtain a training set and a test set / data to be detected; extracting wavelet packet decomposition features and statistical features of sample points in the training set to construct a feature set of the monitoring data, and selecting features in the feature set based on a chi-square test to exclude redundant features to obtain a final training set; training an integrated learning model based on the final training set to obtain an anomaly detection model; and inputting the test set / data to be detected into the anomaly detection model to obtain an anomaly detection result.

[0008] In some embodiments, extracting wavelet packet decomposition features and statistical features of sample points in the training set to construct a feature set of the monitoring data includes: extracting wavelet packet decomposition features of sample points in the training set to describe instantaneous changes in the monitoring data; extracting statistical features of sample points in the training set to describe overall changes in the monitoring data.

[0009] In some embodiments, extracting wavelet packet decomposition features of sample points in the training set to describe instantaneous changes in the monitoring data includes: decomposing the passband of each sample point into eight frequency bands based on three-layer wavelet packet decomposition, and the frequency bands describe instantaneous change information of the monitoring data.

[0010] In some embodiments, extracting statistical features of sample points in the training set to express the overall changes of the monitoring data includes: extracting statistical features of samples in the training set, wherein the statistical features include a seven-dimensional feature vector consisting of maximum value, minimum value, mean, median, standard deviation, mean square value and energy value.

[0011] In some embodiments, selecting features in the feature set based on the chi-square test to exclude redundant features to obtain the final training set also includes: selecting features in the feature set based on the chi-square test to exclude redundant features, and using an adaptive oversampling method to enhance the feature set to balance the ratio of normal samples to abnormal samples to obtain the final training set.

[0012] In some embodiments, collecting monitoring data and segmenting the monitoring data based on a sliding window to obtain a training set and a test set / data to be tested includes: collecting monitoring data from a cloud platform through a collection tool and marking abnormal points; segmenting the monitoring data based on a sliding window to filter out missing points in the monitoring data, and marking normal samples and abnormal samples respectively.

[0013] In some embodiments, training the ensemble learning model based on the final training set to obtain the anomaly detection model includes: training the ensemble learning model based on an extreme gradient boosting algorithm and the final training set to obtain the anomaly detection model.

[0014] On the other hand, an embodiment of the present invention provides a monitoring data anomaly detection device, comprising: a first module, configured to collect monitoring data, and segment the monitoring data based on a sliding window to obtain a training set and a test set / data to be detected; a second module, configured to extract wavelet packet decomposition features and statistical features of sample points in the training set to construct a feature set of the monitoring data, and select features in the feature set based on a chi-square test to exclude redundant features to obtain a final training set; a third module, configured to train an integrated learning model based on the final training set to obtain an anomaly detection model; and a fourth module, configured to input the test set / data to be detected into the anomaly detection model to obtain an anomaly detection result.

[0015] According to another aspect of the embodiments of the present invention, a computer device is provided, comprising: at least one processor; and a memory, wherein the memory stores computer instructions executable on the processor, and the instructions implement the steps of the above method when executed by the processor.

[0016] According to another aspect of the embodiments of the present invention, a computer-readable storage medium is provided, which stores a computer program that implements the above method steps when executed by a processor.

[0017] The present invention has the following beneficial technical effects: using wavelet packet decomposition features and statistical features to extract instantaneous changes and overall changes in time series data, and using the chi-square test to select features, eliminating the influence of redundant features, and then using the adaptive oversampling method to enhance the feature sample set, thereby enhancing the generalization ability of the anomaly detection model and improving the accuracy of anomaly detection in single-dimensional monitoring data. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A schematic diagram of an embodiment of a method for detecting abnormal monitoring data provided by the present invention;

[0020] Figure 2 A schematic diagram of a flow chart of an embodiment of a method for detecting abnormal monitoring data provided by the present invention;

[0021] Figure 3 A schematic diagram of a binary tree of wavelet packet decomposition provided by the present invention;

[0022] Figure 4 A schematic diagram of an embodiment of a device for detecting abnormal monitoring data provided by the present invention;

[0023] Figure 5 A schematic diagram of an embodiment of a computer device provided by the present invention;

[0024] Figure 6 A schematic diagram of an embodiment of a computer-readable storage medium provided by the present invention. DETAILED DESCRIPTION

[0025] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the embodiments of the present invention are further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0026] It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are for distinguishing two non-identical entities with the same name or non-identical parameters. It can be seen that "first" and "second" are only for the convenience of expression and should not be understood as limitations on the embodiments of the present invention. The subsequent embodiments will not explain this one by one.

[0027] Based on the above objectives, a first aspect of an embodiment of the present invention provides an embodiment of a method for detecting abnormalities in monitoring data. Figure 1 FIG. 2 is a schematic diagram of an embodiment of a method for detecting abnormal monitoring data provided by the present invention. Figure 1 As shown, the embodiment of the present invention includes the following steps:

[0028] S01, collecting monitoring data, and segmenting the monitoring data based on a sliding window to obtain a training set and a test set / data to be tested;

[0029] S02, extracting wavelet packet decomposition features and statistical features of sample points in the training set to construct a feature set of monitoring data, and selecting features in the feature set based on the chi-square test to exclude redundant features, to obtain a final training set;

[0030] S03. Training the integrated learning model based on the final training set to obtain an anomaly detection model; and

[0031] S04. Input the test set / data to be detected into the anomaly detection model to obtain anomaly detection results.

[0032] In this embodiment, the performance optimization is applicable to various cloud platform infrastructures, such as x86, arm and mips platforms. Figure 2 The flowchart of an embodiment of the method for detecting abnormal monitoring data provided by the present invention is shown as follows: Figure 2 As shown in the figure, the process includes data feature extraction, feature selection, feature extraction, XGBoost (Extreme Gradient Boosting, extreme gradient boosting) model training and detection. Among them, monitoring data preprocessing includes filling missing points of monitoring data, data segmentation based on sliding windows, and division of training sets and test sets; feature extraction extracts wavelet packet decomposition features and statistical features from sample points to construct feature sets for anomaly detection; feature selection and enhancement use chi-square test to select the extracted features, and use ADASYN (adaptive synthetic sampling, adaptive comprehensive oversampling) algorithm to enhance the feature data set to balance the ratio of normal samples and abnormal samples; XGBoost model training and parameter optimization use training sets to train XGBoost models, and perform grid search on model parameters to determine the optimal parameters of the model and complete model training; XGBoost model anomaly detection extracts wavelet packet decomposition features and statistical features from test sets or online data, inputs the features into the trained model, and outputs anomaly detection results.

[0033] In some embodiments of the present invention, extracting wavelet packet decomposition features and statistical features of sample points in a training set to construct a feature set of monitoring data includes: extracting wavelet packet decomposition features of sample points in the training set to describe instantaneous changes in the monitoring data; extracting statistical features of sample points in the training set to describe the overall changes in the monitoring data.

[0034] In this embodiment, in a time series, the time series data changes randomly, so when performing feature engineering, both the instantaneous changes of the time series and the overall changes of the data must be considered. Wavelet packet transform and statistical features are used to describe the characteristics of time series data. Wavelet packet transform describes the instantaneous changes of data, and statistical features describe the overall discrete characteristics of time series data.

[0035] Wavelet transform is a representative method of time domain analysis, with multi-resolution characteristics. Wavelet transform only further decomposes the low-frequency part of the signal and no longer decomposes the high-frequency part, which is not enough for time series monitoring data containing a large amount of instantaneous information. However, wavelet packet transform can decompose not only the low-frequency part of time series data, but also the high-frequency part of the data, so this decomposition method is a more refined time series data decomposition method, which improves the time domain resolution of time series data.

[0036] In some embodiments of the present invention, extracting wavelet packet decomposition features of sample points in the training set to describe instantaneous changes in monitoring data includes: decomposing the passband of each sample point into eight frequency bands based on three-layer wavelet packet decomposition, and the frequency bands describe the instantaneous change information of the monitoring data.

[0037] In this embodiment, Figure 3 What is shown is a binary tree schematic diagram of the wavelet packet decomposition provided by the present invention, such as Figure 3 As shown in the figure, wavelet packet decomposition can be described by a complete binary tree. In a three-layer wavelet packet decomposition process, each node represents a passband, and the number represents the number of decomposition layers of the wavelet packet. After wavelet packet decomposition, not only can all original signal bands be covered, but also the signal can be decomposed at any multiple scales to obtain comprehensive local information. After three-layer wavelet packet decomposition, 8 frequency bands of time series data can be obtained. The energy feature on each frequency band is used as the time-frequency feature. The feature calculation is expressed as:

[0038] Where j represents the corresponding frequency band after wavelet decomposition, N is the number of data points, and n i Represents the coefficients on each frequency band. Time series data contains instantaneous changes and overall changes. In this paper, wavelet packet analysis is used to extract the instantaneous change information of time series data and describe it. Then, the overall change information of time series data needs to be extracted.

[0039] In some embodiments of the present invention, extracting statistical features of sample points in a training set to express the overall changes in monitoring data includes: extracting statistical features of samples in the training set, the statistical features including a seven-dimensional feature vector consisting of maximum value, minimum value, mean, median, standard deviation, mean square value and energy value.

[0040] In this embodiment, the characteristic of statistics is to describe the quantity and overall nature of data, which can reflect the quantitative characteristics of data rather than focusing on individual data. Therefore, this paper uses statistical characteristics to describe the overall changes of sample data. The statistical feature vectors extracted in this paper include maximum value, minimum value, mean, median, standard deviation, mean square value, and energy value, which constitute a 7-dimensional feature vector with statistical feature dimensions.

[0041] In some embodiments of the present invention, selecting features in the feature set based on the chi-square test to exclude redundant features to obtain the final training set also includes: selecting features in the feature set based on the chi-square test to exclude redundant features, and using an adaptive oversampling method to enhance the feature set to balance the ratio of normal samples to abnormal samples to obtain the final training set.

[0042] In this embodiment, the wavelet packet transform features and statistical features of the data are extracted, and the data samples are described from both instantaneous and overall perspectives. However, it is necessary to know the degree of influence of each dimension in the feature on the classification task, so that redundant features can be eliminated and the feature dimensions can be reduced. The present invention uses the chi-square test to determine the features of each dimension. The chi-square test is a widely used feature selection method, which mainly compares the correlation between two or more samples and two classifications. The chi-square test is used to test the correlation between the extracted wavelet packet decomposition features and statistical features and the classification of normal samples and abnormal samples. The formula is as follows:

[0043] Where A represents the actual value and T represents the theoretical value.

[0044] In this embodiment, the ratio of abnormal samples to normal samples contained in the data used is unbalanced, and the ratio of abnormal samples is too small, which can easily make the model unusable for classification tasks. Studies have shown that the classification effect is best when the ratio of the two types of samples is 1:1, and the ratio of normal samples to abnormal samples in the data in this article is nearly 20:1. Therefore, this article uses the ADASYN algorithm to balance the ratio of normal samples to abnormal samples. The idea of ​​the ADASYN algorithm is to calculate the K nearest neighbors around the abnormal sample, and use the proportion of normal samples in its K nearest neighbors as the weight, calculate the number of synthetic samples based on the weight, and randomly select abnormal samples from the K nearest neighbors of the abnormal sample. The formula is as follows:

[0045] s=x+(x z -x)*λ,λ∈[0,1], where x z is a random sample selected from the K nearest neighbors, x is an abnormal sample, s is a synthetic sample, and λ is a random factor.

[0046] In some embodiments of the present invention, monitoring data is collected, and the monitoring data is segmented based on a sliding window to obtain a training set and a test set / data to be tested, including: collecting monitoring data from a cloud platform through a collection tool, and marking abnormal points; segmenting the monitoring data based on a sliding window to filter out missing points in the monitoring data, and marking normal samples and abnormal samples respectively.

[0047] In this embodiment, the training data used is the monitoring data of the cloud platform for four months as an example. The monitoring data collection tool is telegraf, and its built-in plug-ins cpu, disk, and mem are used for collection, and the collection cycle is 60s. The collected data includes monitoring data of key performance indicators such as CPU and memory. Abnormal points are marked by machine recognition and manual judgment. The data is segmented based on the sliding window method, the missing points in the data are filtered out, and data labels are added. The normal sample label is 0 and the abnormal sample label is 1. A total of 21,542 sample data were extracted, of which 20,074 were normal samples and 1,468 were abnormal samples.

[0048] In some embodiments of the present invention, training the ensemble learning model based on the final training set to obtain the anomaly detection model includes: training the ensemble learning model based on the extreme gradient boosting algorithm and the final training set to obtain the anomaly detection model.

[0049] In this embodiment, the full name of the XGBoost algorithm is the extreme gradient boosting algorithm, which is a weak learning algorithm based on ensemble learning. A weak classifier is a classifier that is slightly better than random guessing but has a low accuracy rate. The core of XGBoost is to continuously add feature splitting trees to the classifier, and each tree is equivalent to a weak classifier. These trees are integrated and combined, so that the XGBoost model has better generalization ability. This tree model can not only improve the fitting effect, but also solve the problem of uneven sample distribution to a certain extent.

[0050] In this embodiment, the data input to the XGBoost model is {(x1,y1),(x2,y2),...,(x m ,y m )}, where x is the feature extracted from the sample, y is the label, the normal sample label is 0, and the abnormal sample label is 1. The output of the XGBoost model can be expressed as:

[0051] Among them, f k (x) represents the kth weak learner. The total number of weak learners in the XGBoost model is K. These K weak classifiers need to minimize the function in order to form a strong classifier:

[0052] Where l(·) is the loss function and Ω(·) is the regularization function. In the regularization term, T is the number of leaf nodes in the tree, w is the weight of the leaf node, and γ and λ are the hyperparameters in the regularization term. In each round of iteration, only the objective function of the tth regression tree is optimized:

[0053] in is the sample x corresponding to the first t-1 trees i The output, f t (x i ) is the output of the current tree. Taylor expansion is performed on the objective function, and the linear and quadratic terms in the formula are retained. The approximate value of the objective is:

[0054]

[0055] in:

[0056] For each sample, the first-order derivative and second-order derivative of the loss function, i∈I j Represents each sample data mapped to the jth leaf node.

[0057] For w j The derivative is equal to 0, and w is obtained j The optimal solution is:

[0058] Will Substitute the original objective function to obtain:

[0059] Where T is the number of leaf nodes.

[0060] Through the above iterations, the optimal splitting variable and cut value of the tree can be found. Find the tree with the best structure and add it to the model, using a greedy algorithm to find the optimal tree structure.

[0061] After feature extraction, feature selection, and XGBoost model training, an anomaly detection model can be generated. Wavelet packet decomposition features and statistical features of the test set or online data can be extracted, and redundant features can be eliminated based on the feature selection results during training. The features are input into the trained model to obtain anomaly detection results.

[0062] It should be pointed out in particular that the various steps in the various embodiments of the above-mentioned monitoring data anomaly detection method can be cross-linked, replaced, added, and deleted. Therefore, these reasonable permutations, combinations and transformations of the monitoring data anomaly detection method should also belong to the protection scope of the present invention, and the protection scope of the present invention should not be limited to the embodiments.

[0063] Based on the above purpose, a second aspect of an embodiment of the present invention provides a detection device for abnormal monitoring data. Figure 4 FIG. 2 is a schematic diagram of an embodiment of a device for detecting abnormal monitoring data provided by the present invention. Figure 4As shown, an embodiment of the present invention includes the following modules: a first module S41, configured to collect monitoring data, and to segment the monitoring data based on a sliding window to obtain a training set and a test set / data to be detected; a second module S42, configured to extract wavelet packet decomposition features and statistical features of sample points in the training set to construct a feature set of monitoring data, and to select features in the feature set based on a chi-square test to exclude redundant features to obtain a final training set; a third module S43, configured to train an integrated learning model based on the final training set to obtain an anomaly detection model; and a fourth module S44, configured to input the test set / data to be detected into the anomaly detection model to obtain an anomaly detection result.

[0064] Based on the above purpose, a third aspect of an embodiment of the present invention provides a computer device. Figure 5 FIG. 2 is a schematic diagram of an embodiment of a computer device provided by the present invention. Figure 5 As shown, the embodiment of the present invention includes the following apparatus: at least one processor S51; and a memory S52, wherein the memory S52 stores computer instructions S53 that can be executed on the processor, and the instructions implement the steps of the above method when executed by the processor.

[0065] The present invention also provides a computer-readable storage medium. Figure 6 FIG. 2 is a schematic diagram of an embodiment of a computer-readable storage medium provided by the present invention. Figure 6 As shown, the computer-readable storage medium stores S61 a computer program S62 for executing the above method when executed by a processor.

[0066] Finally, it should be noted that a person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be implemented by instructing the relevant hardware through a computer program, and the program of the detection method of monitoring data anomalies can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium of the program can be a disk, an optical disk, a read-only storage memory (ROM) or a random access memory (RAM), etc. The above-mentioned computer program embodiment can achieve the same or similar effect as any of the corresponding aforementioned method embodiments.

[0067] In addition, the method disclosed in the embodiment of the present invention can also be implemented as a computer program executed by a processor, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the above functions defined in the method disclosed in the embodiment of the present invention are performed.

[0068] In addition, the above method steps and system units can also be implemented using a controller and a computer-readable storage medium for storing a computer program that enables the controller to implement the above steps or unit functions.

[0069] It will also be appreciated by those skilled in the art that various exemplary logic blocks, modules, circuits and algorithm steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software or a combination of the two. In order to clearly illustrate this interchangeability of hardware and software, a general description has been given to the functions of various schematic components, blocks, modules, circuits and steps. Whether this function is implemented as software or hardware depends on specific applications and the design constraints imposed on the entire system. Those skilled in the art can implement the function in various ways for each specific application, but this implementation decision should not be interpreted as causing a departure from the disclosed scope of the embodiments of the present invention.

[0070] In one or more exemplary designs, the function can be implemented in hardware, software, firmware or any combination thereof. If implemented in software, the function can be stored as one or more instructions or codes on a computer-readable medium or transmitted by a computer-readable medium. Computer-readable media include computer storage media and communication media, and the communication media include any medium that helps to transfer a computer program from one location to another. The storage medium can be any available medium that can be accessed by a general or special computer. As an example and not limiting, the computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, disk storage devices or other magnetic storage devices, or any other medium that can be used to carry or store the required program code in the form of an instruction or data structure and can be accessed by a general or special computer or a general or special processor. In addition, any connection can be appropriately referred to as a computer-readable medium. For example, if a coaxial cable, optical fiber cable, twisted pair, digital subscriber line (DSL) or wireless technologies such as infrared, radio and microwaves are used to send software from a website, server or other remote source, the above-mentioned coaxial cable, optical fiber cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwaves are all included in the definition of the medium. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, Blu-ray disc, wherein disks usually reproduce data magnetically, while optical discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0071] The above are exemplary embodiments disclosed in the present invention, but it should be noted that various changes and modifications may be made without departing from the scope disclosed in the embodiments of the present invention as defined in the claims. The functions, steps and / or actions of the method claims according to the disclosed embodiments described herein do not need to be performed in any particular order. In addition, although the elements disclosed in the embodiments of the present invention may be described or required in individual form, they may also be understood as multiple unless explicitly limited to the singular.

[0072] It should be understood that, as used herein, the singular forms "a", "an" are intended to include the plural forms as well, unless the context clearly supports an exception. It should also be understood that, as used herein, "and / or" is intended to include any and all possible combinations of one or more of the associated listed items.

[0073] The serial numbers of the embodiments disclosed in the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0074] A person skilled in the art will appreciate that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0075] A person skilled in the art should understand that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the disclosure of the embodiments of the present invention (including the claims) is limited to these examples; under the concept of the embodiments of the present invention, the technical features in the above embodiments or different embodiments can also be combined, and there are many other changes in different aspects of the embodiments of the present invention as above, which are not provided in detail for the sake of simplicity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the protection scope of the embodiments of the present invention.

Claims

1. A method for detecting abnormal monitoring data, characterized in that: The following steps are involved: Collect monitoring data, and segment the monitoring data based on a sliding window to obtain a training set and a test set / data to be tested; Extracting wavelet packet decomposition features and statistical features of sample points in the training set to construct a feature set of the monitoring data, and selecting features in the feature set based on a chi-square test to exclude redundant features, thereby obtaining a final training set; Training the ensemble learning model based on the final training set to obtain an anomaly detection model; as well as Inputting the test set / data to be detected into the anomaly detection model to obtain an anomaly detection result; Extracting the wavelet packet decomposition features and statistical features of the sample points in the training set to construct the feature set of the monitoring data includes: Extracting wavelet packet decomposition features of sample points in the training set to express instantaneous changes of the monitoring data; Extracting statistical features of sample points in the training set to describe the overall changes of the monitoring data; Extracting the wavelet packet decomposition features of the sample points in the training set to express the instantaneous changes of the monitoring data includes: Decomposing the passband of each sample point into eight frequency bands based on three-layer wavelet packet decomposition, wherein the frequency bands represent the instantaneous change information of the monitoring data; The characteristic calculation of the frequency band is expressed as: ,in j represents the corresponding frequency band after wavelet decomposition, N is the number of data points, n i Represents the coefficients on each frequency band; Extracting statistical features of sample points in the training set to describe the overall changes of the monitoring data includes: Extracting statistical features of the samples in the training set, wherein the statistical features include a seven-dimensional feature vector consisting of a maximum value, a minimum value, a mean value, a median value, a standard deviation, a mean square value, and an energy value; The features in the feature set are selected based on the chi-square test to exclude redundant features, and the final training set also includes: The features in the feature set are selected based on the chi-square test to exclude redundant features, and the feature set is enhanced using an adaptive oversampling method to balance the ratio of normal samples to abnormal samples, thereby obtaining a final training set.

2. The method for detecting abnormal monitoring data according to claim 1, characterized in that: Collecting monitoring data, and segmenting the monitoring data based on a sliding window to obtain a training set and a test set / data to be tested includes: Collect monitoring data from the cloud platform through collection tools and mark abnormal points; The monitoring data is segmented based on a sliding window to filter out missing points in the monitoring data, and normal samples and abnormal samples are marked respectively.

3. The method for detecting abnormal monitoring data according to claim 1, characterized in that: Training the ensemble learning model based on the final training set to obtain an anomaly detection model includes: The ensemble learning model is trained based on the extreme gradient boosting algorithm and the final training set to obtain an anomaly detection model.

4. A monitoring data anomaly detection device, characterized in that: include: A first module is configured to collect monitoring data and segment the monitoring data based on a sliding window to obtain a training set and a test set / data to be tested; The second module is configured to extract the wavelet packet decomposition features and statistical features of the sample points in the training set to construct a feature set of the monitoring data, and select the features in the feature set based on the chi-square test to exclude redundant features to obtain a final training set; Extracting the wavelet packet decomposition features and statistical features of the sample points in the training set to construct the feature set of the monitoring data includes: extracting the wavelet packet decomposition features of the sample points in the training set to express the instantaneous changes of the monitoring data; extracting the statistical features of the sample points in the training set to express the overall changes of the monitoring data; extracting the wavelet packet decomposition features of the sample points in the training set to express the instantaneous changes of the monitoring data includes: decomposing the passband of each sample point into eight frequency bands based on three-layer wavelet packet decomposition, and the frequency bands express the instantaneous change information of the monitoring data; the characteristic calculation of the frequency band is expressed as: ,in j represents the corresponding frequency band after wavelet decomposition, N is the number of data points, n i Representing coefficients on each frequency band; extracting statistical features of sample points in the training set to express the overall change of the monitoring data includes: extracting statistical features of samples in the training set, wherein the statistical features include a seven-dimensional feature vector consisting of maximum value, minimum value, mean value, median value, standard deviation, mean square value and energy value; selecting features in the feature set based on the chi-square test to exclude redundant features, and obtaining the final training set also includes: selecting features in the feature set based on the chi-square test to exclude redundant features, and using an adaptive oversampling method to enhance the feature set to balance the ratio of normal samples to abnormal samples, and obtaining the final training set; A third module is configured to train the integrated learning model based on the final training set to obtain an anomaly detection model; and The fourth module is configured to input the test set / data to be detected into the anomaly detection model to obtain an anomaly detection result.

5. A computer device, characterized in that: include: at least one processor; as well as A memory, wherein the memory stores computer instructions executable on the processor, wherein the instructions, when executed by the processor, implement the steps of the method according to any one of claims 1 to 3.

6. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.

Images