Abnormality detection method and device, equipment, storage medium and program product

Abstract

The invention relates to an anomaly detection method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the steps of obtaining a to-be-detected data set of a target system, and performing variable partitioning on the to-be-detected data set to obtain a to-be-detected partitioned data set; performing feature extraction on each block data set to be detected to obtain a feature data set to be detected; based on each to-be-detected block data set and each to-be-detected feature data set, detecting a deviation degree between the to-be-detected data set and a preset normal data set; and when it is detected that the deviation degree exceeds a preset detection control limit, it is judged that the target system is abnormal. By adopting the method, the anomaly detection accuracy can be improved.

Description

Technical Field

[0001] The present application relates to the field of cloud computing monitoring technology, and in particular to an anomaly detection method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the rapid development of electronic information technology, sensing and detection technologies are also changing with each passing day. Under the large-scale software and hardware system architecture, it is necessary to implement effective monitoring to ensure the stability of the operation status of each system. With the continuous evolution of database technology, rich and diverse system data can be retained, and system detection and monitoring technologies based on multivariate statistics have emerged.

[0003] Traditional technologies, such as principal component analysis (PCA), partial least squares regression (PLS), and independent components analysis (ICA), usually establish a global model of the data for detection.

[0004] However, there are more and more large-scale multi-unit systems. Anomalies in large-scale multi-unit systems may be caused by only some units. Establishing a global model for detection may cause local feature information to be submerged, resulting in low accuracy of anomaly detection. Summary of the invention

[0005] Based on this, it is necessary to provide an anomaly detection method, apparatus, computer device, computer-readable storage medium and computer program product that can improve the accuracy of anomaly detection in response to the above technical problems.

[0006] In a first aspect, the present application provides an anomaly detection method, comprising:

[0007] Obtain a data set to be inspected of the target system, divide the data set into blocks by variables, and obtain a block data set to be inspected;

[0008] Perform feature extraction on each block data set to be inspected respectively to obtain a feature data set to be inspected;

[0009] Based on each block data set to be inspected and each feature data set to be inspected, detecting the deviation between the data set to be inspected and a preset normal data set;

[0010] When it is detected that the deviation exceeds the preset detection control limit, it is determined that there is an abnormality in the target system.

[0011] In one embodiment, based on each block data set to be checked and each feature data set to be checked, detecting the deviation between the data set to be checked and a preset normal data set includes:

[0012] Performing slow feature analysis on each block data set to be tested and each feature data set to be tested respectively to obtain multiple detection statistics;

[0013] Detecting the target distance between each test statistic and the normal data distribution center corresponding to each test statistic;

[0014] Based on the distances of each target, the deviation between the dataset to be inspected and the preset normal dataset is detected.

[0015] In one embodiment, before detecting the target distance between each detection statistic and the normal data distribution center corresponding to each detection statistic, the method further includes:

[0016] Obtain a normal data set of the target system, divide the normal data set into variable blocks, and obtain a normal block data set;

[0017] Perform feature extraction on each normal block data set to obtain a normal feature data set;

[0018] Perform slow feature analysis on each normal block data set and each normal feature data set to obtain multiple normal data distributions;

[0019] Based on the normal data distribution corresponding to each normal block data set or each normal feature data set, the normal data distribution center and the normal data distribution radius corresponding to each normal block data set or each normal feature data set are detected, and the normal data distribution radius is determined as the detection control limit.

[0020] In one embodiment, the feature data set to be tested includes at least one of a cumulative error feature data set to be tested and a second-order difference feature data set to be tested; feature extraction is performed on each block data set to be tested, and the feature data set to be tested includes at least one of the following:

[0021] Extracting cumulative error features from each block data set to be inspected respectively to obtain a cumulative error feature data set to be inspected;

[0022] The second-order difference feature extraction is performed on each block data set to be tested to obtain the second-order difference feature data set to be tested.

[0023] In one embodiment, the to-be-tested data set includes to-be-tested data at multiple moments; the to-be-tested cumulative error feature data set includes multiple to-be-tested cumulative error features; for each to-be-tested block data set, cumulative error feature extraction is performed on the to-be-tested block data set to obtain the to-be-tested cumulative error feature data set, including:

[0024] Detect the data mean of each data to be tested in the block data set to be tested;

[0025] Selecting a preset number of consecutive data to be inspected in time sequence from the block data set to be inspected as the accumulated data to be inspected;

[0026] According to each accumulated data to be inspected and the data mean, the accumulated error characteristics of the data to be inspected are detected.

[0027] In one embodiment, the data set to be inspected includes a plurality of data subsets to be inspected; the data set to be inspected is variable-blocked to obtain the block data set to be inspected, including:

[0028] By presetting the Gaussian mixture model, each subset of the data to be inspected is classified to obtain the classification result;

[0029] According to the classification result, each to-be-inspected data subset is divided into a plurality of to-be-inspected block data sets, wherein the to-be-inspected data subsets belonging to the same classification are divided into the same to-be-inspected block data set.

[0030] In a second aspect, the present application also provides an abnormality detection device, comprising:

[0031] An acquisition module is used to acquire a data set to be inspected of a target system, divide the data set to be inspected into blocks by variables, and obtain a block data set to be inspected;

[0032] The feature extraction module is used to extract features from each block data set to be inspected to obtain a feature data set to be inspected;

[0033] A detection module, used to detect the deviation between the to-be-detected data set and a preset normal data set based on each to-be-detected block data set and each to-be-detected feature data set;

[0034] The determination module is used to determine that an abnormality exists in the target system when it is detected that the deviation exceeds a preset detection control limit.

[0035] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0036] Obtain a data set to be inspected of the target system, divide the data set into blocks by variables, and obtain a block data set to be inspected;

[0037] Perform feature extraction on each block data set to be inspected respectively to obtain a feature data set to be inspected;

[0038] Based on each block data set to be inspected and each feature data set to be inspected, detecting the deviation between the data set to be inspected and a preset normal data set;

[0039] When it is detected that the deviation exceeds the preset detection control limit, it is determined that there is an abnormality in the target system.

[0040] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented:

[0041] Obtain a data set to be inspected of the target system, divide the data set into blocks by variables, and obtain a block data set to be inspected;

[0042] Perform feature extraction on each block data set to be inspected respectively to obtain a feature data set to be inspected;

[0043] Based on each block data set to be inspected and each feature data set to be inspected, detecting the deviation between the data set to be inspected and a preset normal data set;

[0044] When it is detected that the deviation exceeds the preset detection control limit, it is determined that there is an abnormality in the target system.

[0045] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0046] Obtain a data set to be inspected of the target system, divide the data set into blocks by variables, and obtain a block data set to be inspected;

[0047] Perform feature extraction on each block data set to be inspected respectively to obtain a feature data set to be inspected;

[0048] Based on each block data set to be inspected and each feature data set to be inspected, detecting the deviation between the data set to be inspected and a preset normal data set;

[0049] When it is detected that the deviation exceeds the preset detection control limit, it is determined that there is an abnormality in the target system.

[0050] The above-mentioned anomaly detection method, device, computer equipment, computer-readable storage medium and computer program product first obtain a data set to be tested of the target system, perform variable block division on the data set to be tested, and obtain the block data set to be tested. Each block data set to be tested after variable block division contains a part of system variables. These system variables reflect the local behavior of the target system on the block data set to be tested. Therefore, different block data sets to be tested contain different local feature information of the target system. Furthermore, since each block data set to be tested not only contains the local feature information of the target system, but also contains the global feature information of the data set in the corresponding dimension, by performing feature extraction on each block data set to be tested respectively, a feature data set to be tested is obtained, and the global feature information extraction of the target system can be realized; further, based on each block data set to be tested and each feature data set to be tested, the deviation between the data set to be tested and the preset normal data set is detected. When it is detected that the deviation exceeds the preset detection control limit, it is determined that the target system has an abnormality, and accurate detection of the abnormality of the target system can be achieved. Compared with the method of establishing a global model for detection, the present application captures the local feature information of the target system through variable block, and captures the global feature information of the target system by extracting features from each block data set to be tested, and then combines the local feature information and global feature information of the target system for anomaly detection, which can effectively improve the accuracy of anomaly detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0052] FIG1 is a schematic diagram of a flow chart of an abnormality detection method in one embodiment;

[0053] Figure 2 is a schematic diagram of the structure of a target system in one embodiment;

[0054] Figure 3 A schematic diagram of a process flow of a deviation detection step in an embodiment;

[0055] Figure 4 is a flow chart of an abnormality detection method in another embodiment;

[0056] Figure 5 is a structural block diagram of an abnormality detection device in one embodiment;

[0057] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0059] With the rapid development of electronic information technology, sensing and detection technologies are also changing with each passing day. Under the large-scale software and hardware system architecture, it is necessary to implement effective monitoring to ensure the stability of the operation status of each system. With the continuous evolution of database technology, rich and diverse system data can be retained, and system detection and monitoring technologies based on multivariate statistics have emerged. For example, traditional multivariate statistical monitoring methods such as principal component analysis, partial least squares method and independent component analysis. However, most of these methods belong to a single model framework, which makes it difficult to implement effective detection of anomalies in large-scale multi-unit systems.

[0060] In related technologies, we can first assume that data is missing one by one, and use known data regression to estimate the corresponding principal component prediction value, and then use the principal component estimation error to establish a global model of the data for detection. However, there are more and more systems with large-scale and multi-unit features. The anomalies in such systems may only be caused by some units. If only a global model is established, the local feature information of the system may be submerged, resulting in low accuracy of anomaly detection.

[0061] In an exemplary embodiment, as shown in FIG1 , an anomaly detection method is provided. This embodiment is illustrated by applying the method to a terminal, wherein the terminal may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things devices may be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, and the like. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, and the like. Head-mounted devices may be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, and the like. It is understandable that the method may also be applied to a server, and may also be applied to a system including a terminal and a server, and may be implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps S10-S40. Among them:

[0062] Step S10, obtaining a data set to be inspected of the target system, dividing the data set to be inspected into blocks by variables, and obtaining a divided data set to be inspected.

[0063] Among them, the target system can be a multi-unit system, that is, the target system can include multiple system units. Performing abnormality detection on the target system can refer to the process of collecting data to be tested from each system unit of the target system, and judging whether the target system has an abnormality based on the collected data to be tested. The data set to be tested can refer to the collection of data to be tested collected from each system unit, and the data to be tested collected from each system unit can be the same or different. The data to be tested can refer to the operating data of the system unit used for system abnormality detection. In some feasible implementations, the number of data to be tested collected from each system unit can be multiple, and the multiple data to be tested collected from the same system unit can form a subset of data to be tested, and the subset of data to be tested collected from each system unit constitutes a data set to be tested. As an example, the subset of data to be tested can include multiple data to be tested collected from the same system unit within a collection time range, and there is a certain temporal sequence between the data to be tested belonging to the same subset of data to be tested.

[0064] Variable segmentation may refer to the process of classifying each subset of the data to be tested in the data set to be tested into different segmented data sets to be tested. The segmented data set to be tested may refer to a data set composed of some subsets of the data to be tested with the same or similar local feature information after the data set to be tested is segmented by variables. Similar local feature information may refer to the difference not exceeding a preset difference threshold.

[0065] Exemplarily, during the operation of the target system, the operation data of each system unit in the target system can be collected in real time, periodically or in a triggered manner as the data to be tested. Then, the data to be tested collected within a preset collection time range can be combined into a data set to be tested for abnormality detection of the target system. After the data set to be tested is obtained, the data set to be tested can be firstly segmented into variables, and based on the local feature information of each data subset to be tested in the data set to be tested, each data subset to be tested can be classified into different segmented data sets to be tested. The local feature information contained in the data subsets to be tested belonging to different segmented data sets to be tested is different or the difference exceeds a preset difference threshold, and the local feature information contained in the data subsets to be tested belonging to the same segmented data set to be tested is the same or similar.

[0066] In some feasible implementations, one or more of the dimensions and magnitudes of the to-be-tested data collected from different system units may be different. As an example, refer to Figure 2, the target system may include system modules such as software system and hardware system, and these system modules may be further subdivided into service port system unit, service url (uniform resource locator, uniform resource locator system) system unit, response time system unit, database system unit, CPU (Central Processing Unit, central processing unit) system unit, disk system unit, memory system unit, network system unit, etc., and one or more of the dimensions and magnitudes of the data to be inspected collected from different system units may be different. In order to eliminate the influence of the dimensions and magnitudes, the variable block of the data set to be inspected may include: standardizing the data set to be inspected, and then performing variable block on the data set to be inspected after the standardization process to obtain the block data set to be inspected.

[0067] As an example, the method of standardizing the data set to be tested includes: calculating the mean value and standard deviation of the data set to be tested, and then subtracting the mean value from the data to be tested, dividing the difference obtained by the standard deviation, and obtaining the standardized data to be tested. Data standardization can be expressed as:

[0068]

[0069] Among them, X' is the data to be tested after standardization, X is the data to be tested before standardization, and μ is the average value of the data to be tested before standardization. is the standard deviation of the data before standardization.

[0070] Step S20, performing feature extraction on each block data set to be inspected to obtain a feature data set to be inspected.

[0071] Among them, the feature data set corresponding to each block data set includes feature information corresponding to at least part of the data in the block data set, and the total amount of data in the feature data set is less than or equal to the total amount of data in the block data set. It can be understood that, when the feature information corresponding to the data does not depend on other data in the block data set, the total amount of data in the feature data set can be equal to the total amount of data in the block data set; when the feature information corresponding to the data needs to depend on other data in the block data set, there may be a situation where the feature information corresponding to some data cannot be extracted, resulting in the total amount of data in the feature data set being less than the total amount of data in the block data set. For example, if the feature information is a cumulative error value, the cumulative error value of the data needs to be calculated based on the data and a certain number of other data before the data. Then, for these data that have no other data before, or the amount of other data before is insufficient, the cumulative error value cannot be calculated.

[0072] Exemplarily, after the data set to be inspected is subjected to variable block division, feature extraction can be performed on each of the block data sets to be inspected obtained after the variable block division, respectively, to obtain feature data sets to be inspected that correspond one-to-one to each of the block data sets to be inspected. Wherein, for each block data set to be inspected, the feature extraction method includes: feature extraction is performed on each subset of the data to be inspected in the block data set to be inspected, to obtain a subset of feature data to be inspected that corresponds one-to-one to each subset of the data to be inspected. For each subset of the data to be inspected, the feature extraction method includes: feature extraction is performed on at least part of the data to be inspected in the subset of the data to be inspected, to obtain feature data to be inspected that corresponds one-to-one to each of the data to be inspected.

[0073] In some implementations, the subset of data to be inspected may include multiple data to be inspected collected from the same system unit within a collection time range, and the characteristic information may be dynamic characteristic information, which is used to characterize characteristics of data that change over time.

[0074] Most of the traditional principal component analysis, partial least squares method and independent component analysis belong to the static model framework. However, as the scale of the system continues to expand, the structural relationship between the system units becomes more complex, and the dynamic and time series characteristics between system variables gradually emerge. Static detection methods lack the correlation analysis between real-time data and historical data, and it is difficult to effectively detect anomalies in systems with dynamic characteristics, which may lead to reduced detection accuracy. By extracting dynamic features, the accuracy of anomaly detection for systems with dynamic characteristics can be effectively improved.

[0075] Step S30: Based on each block data set to be checked and each feature data set to be checked, detect the deviation between the data set to be checked and a preset normal data set.

[0076] The deviation degree is used to characterize the degree of deviation between the dataset to be tested and the normal dataset in at least one of the whole or part.

[0077] As an example, before the system abnormality detection, the operation data of the target system during normal operation can be obtained in advance to form a normal data set, and then the normal data set is divided into variable blocks in the same manner as the data set to be detected to obtain a normal block data set, and feature extraction is performed on each normal block data set to obtain a normal feature data set, and then the data distribution center of each normal block data set and the data distribution center of each normal feature data set are detected respectively. Furthermore, in some feasible implementations, the data distribution center of each normal block data set and the data distribution center of each normal feature data set can be fused to obtain the normal data distribution center of the normal data set; in other feasible implementations, the normal data distribution center of the normal data set can include the data distribution center of each normal block data set and the data distribution center of each normal feature data set, that is, the normal data distribution center of the normal data set is the collection of the data distribution center of each normal block data set and the data distribution center of each normal feature data set.

[0078] Exemplarily, the data distribution center of each block data set to be checked and the data distribution center of each feature data set to be checked can be detected respectively. Furthermore, as an example, the data distribution center of each block data set to be checked and the data distribution center of each feature data set to be checked can be fused to obtain the data distribution center of the data set to be checked, and the distance between the data distribution center of the data set to be checked and the normal data distribution center is determined as the deviation between the data set to be checked and the preset normal data set; as another example, the deviation between the data set to be checked and the preset normal data set can include the deviation between each block data set to be checked and the corresponding normal block data set, and the deviation between each feature data set to be checked and the corresponding normal feature data set. Specifically, the distance between the data distribution center of each block data set to be checked and the data distribution center of the corresponding normal block data set can be determined as the deviation between the block data set to be checked and the normal block data set, and the distance between the data distribution center of each feature data set to be checked and the data distribution center of the corresponding normal feature data set can be determined as the deviation between the feature data set to be checked and the normal feature data set.

[0079] Step S40: When it is detected that the deviation exceeds the preset detection control limit, it is determined that an abnormality exists in the target system.

[0080] Among them, the detection control limit defines a set of limits, which can be determined based on the normal data set of the target system and are used to determine whether there are abnormal changes during the operation of the target system. Each deviation can be pre-set with a corresponding detection control limit. It can be understood that in the case of the deviation between the data set to be tested and the preset normal data set, including the deviation between each block data set to be tested and the corresponding normal block data set, and the deviation between each feature data set to be tested and the corresponding normal feature data set, the preset detection control limit includes the detection control limit corresponding to each block data set to be tested, and the detection control limit corresponding to each feature data set to be tested, wherein the detection control limit corresponding to the block data set to be tested is determined based on the normal block data set corresponding to the block data set to be tested, and the detection control limit corresponding to the feature data set to be tested is determined based on the normal feature data set corresponding to the feature data set to be tested.

[0081] Exemplarily, the detected deviation is compared with the detection control limit pre-set for the deviation. When any deviation exceeds the corresponding detection control limit, it can be determined that the target system is abnormal. When it is determined that the target system is abnormal, abnormal alarm information can also be issued. The specific form of the abnormal alarm information can be determined according to the actual situation, and this embodiment does not limit this.

[0082] In some feasible implementations, when the deviations do not exceed the corresponding detection control limits, it can be determined that there is no abnormality in the target system, and the process can be returned to execute the step of obtaining the data set to be tested of the target system to continue monitoring the target system.

[0083] In the above-mentioned anomaly detection method, firstly, a data set to be tested of the target system is obtained, and the data set to be tested is divided into variable blocks to obtain the block data set to be tested. After the variable blocks are divided, each block data set to be tested contains a part of system variables, and these system variables reflect the local behavior of the target system on the block data set to be tested. Therefore, different block data sets to be tested contain different local feature information of the target system. Furthermore, since each block data set to be tested not only contains the local feature information of the target system, but also contains the global feature information of the data set in the corresponding dimension, by performing feature extraction on each block data set to be tested respectively to obtain the feature data set to be tested, the global feature information extraction of the target system can be realized; further, based on each block data set to be tested and each feature data set to be tested, the deviation between the data set to be tested and the preset normal data set is detected. When it is detected that the deviation exceeds the preset detection control limit, it is determined that the target system has an abnormality, and accurate detection of the abnormality of the target system can be achieved. Compared with the method of establishing a global model for detection, the present application captures the local feature information of the target system through variable block, and captures the global feature information of the target system by extracting features from each block data set to be tested, and then combines the local feature information and global feature information of the target system for anomaly detection, which can effectively improve the accuracy of anomaly detection.

[0084] In an exemplary embodiment, Figure 3 As shown, based on each block data set to be checked and each feature data set to be checked, detecting the deviation between the data set to be checked and the preset normal data set includes steps S31 to S33. Among them:

[0085] Step S31 , performing slow feature analysis on each block data set to be inspected and each feature data set to be inspected, respectively, to obtain a plurality of detection statistics.

[0086] Among them, slow feature analysis is an unsupervised learning method that can extract slowly changing features from a set of changing time series data. Slow feature analysis assumes that useful information in natural scenes usually changes slowly, while the rapidly changing parts often represent noise or unimportant details. Through slow feature analysis, high-frequency noise can be effectively filtered out, thereby improving the quality of the data.

[0087] In the case where the data to be tested in the subset of data to be tested has time series, the subset of data to be tested can be subjected to slow feature analysis. The feature data set to be tested includes the feature data subsets to be tested corresponding to each subset of data to be tested. In the case where the data to be tested in the subset of data to be tested has time series, the feature data subset to be tested corresponding to the subset of data to be tested also has time series. Therefore, the feature data subset to be tested can be subjected to slow feature analysis. In this way, slow feature analysis can be performed on each subset of data to be tested and each subset of feature data to be tested, respectively, to construct multiple detection statistics.

[0088] Exemplarily, after obtaining the block data set to be tested and the feature data set to be tested, slow feature analysis can be performed on the data subset to be tested in each block data set to be tested to obtain the first slow feature corresponding to each data subset to be tested, and the first slow features corresponding to each data subset to be tested in each block data set to be tested are fused to obtain the detection statistic corresponding to each block data set to be tested; slow feature analysis can also be performed on the feature data subset to be tested in each feature data set to be tested to obtain the second slow feature corresponding to each feature data subset to be tested, and the second slow features corresponding to each feature data subset to be tested in each feature data set to be tested are fused to obtain the detection statistic corresponding to each feature data set to be tested.

[0089] In some feasible implementations, the process of slow feature analysis includes:

[0090] Through the slow feature model, based on the timing input signal , output a set of output signals that change from slow to fast The slow feature model can be expressed as:

[0091]

[0092] Among them, W is the transformation matrix, which can be obtained by solving the following formula:

[0093]

[0094] The above formula also satisfies the following constraints:

[0095] The first constraint is:

[0096] Second constraint:

[0097] The third constraint is:

[0098] in, for The first derivative of ; is the time average; assuming is the original data, is the covariance matrix, To extract slow features, we must first whiten the data, that is, remove the correlation between data or variables. Perform singular value decomposition on the covariance matrix:

[0099]

[0100] Where U is the eigenvector matrix of R, is a diagonal matrix containing all eigenvalues ​​of R, and the whitening matrix is . Then the whitened matrix is:

[0101]

[0102] joint , we can get the extracted slow features:

[0103]

[0104] in Obviously, and The second constraint and the second constraint are equivalent to:

[0105]

[0106] Then the above optimization problem is further transformed into determining the orthogonal matrix P to make the optimization problem Minimum. Covariance matrix Performing singular value decomposition yields:

[0107]

[0108] So the solution of the transformation matrix W is:

[0109]

[0110] The calculation formula for the slow feature is:

[0111]

[0112] It can be obtained that each slow feature corresponds to each row vector in the transformation matrix.

[0113] After calculating the slow features, the fault monitoring statistics based on slow feature analysis are calculated by the following formula:

[0114]

[0115] Furthermore, by calculating the fault monitoring statistics, we can construct the detection statistics , where b represents the number of data subsets in each data set for slow feature analysis. For example, if y test is the detection statistic corresponding to the block data set A1 to be tested, and b represents the number of the subsets of data to be tested in the block data set A1 to be tested.

[0116] In some feasible implementations, by calculating the fault monitoring statistics, a normal data distribution can also be constructed. , where m represents the number of data subsets in each data set for slow feature analysis. For example, if Y is the normal data distribution corresponding to the normal feature data set A1, then m represents the number of normal feature data subsets in the normal feature data set A1.

[0117] As an example, the support vector data description model can be used to calculate the radius of the hypersphere as the detection control limit. The solution model of the support vector data description model is:

[0118]

[0119]

[0120] Where a is the center of the hypersphere, R is the radius of the hypersphere, and C is the penalty coefficient, which is a compromise between the volume of the hypersphere and the sample misclassification rate. It is a slack variable that controls the proportion of sample misclassification.

[0121] Through kernel transformation, the above optimization problem can be transformed into its dual form:

[0122]

[0123]

[0124] in, is the Lagrange multiplier, is the i-th data in the normal data distribution, is the jth data in the normal data distribution, is the kernel function. The distance from the center to the boundary is the radius R of the hypersphere, which can be calculated by the following formula:

[0125]

[0126] Where n represents the total amount of data in the normal data distribution.

[0127] When performing system anomaly detection, the distance between the detection statistic and the center of the hypersphere can be determined as the deviation of the detection statistic. If the distance between the detection statistic and the center of the hypersphere is greater than the radius of the hypersphere, it is judged as out of limit, indicating that the target system is abnormal; otherwise, it is not out of limit, indicating that the target system is normal. The distance between the detection statistic and the center of the sphere can be calculated by the following formula:

[0128]

[0129] in, is the i-th data in the test statistic.

[0130] Step S32, detecting the target distance between each detection statistic and the normal data distribution center corresponding to each detection statistic.

[0131] The normal data distribution center includes the first normal data distribution center corresponding to each first slow feature, and the second normal data distribution center corresponding to each second slow feature. The target distance includes the first distance between each first slow feature and the corresponding first normal data distribution center, and the second distance between each second slow feature and the corresponding second normal data distribution center.

[0132] Exemplarily, the first normal data distribution center corresponding to each first slow feature and the second normal data distribution center corresponding to each second slow feature are obtained, the first distance between each first slow feature and the first normal data distribution center corresponding to the first slow feature is detected, and the second distance between each second slow feature and the second normal data distribution center corresponding to the second slow feature is detected.

[0133] Step S33: detecting the deviation between the to-be-tested data set and the preset normal data set based on the target distances.

[0134] As an example, each target distance can be aggregated to obtain an aggregated distance to be tested, and the difference between the aggregated distance to be tested and the normal aggregated distance corresponding to the normal data set is compared, and the difference is determined as the deviation between the data set to be tested and the preset normal data set. The normal aggregated distance can be detected based on the normal data set using the same detection method as the aggregated distance to be tested.

[0135] As another example, the normal distance corresponding to each target distance can be queried, and then the difference value between each target distance and the corresponding normal distance can be compared. Then, the obtained multiple difference values ​​can be determined as the deviation between the data set to be tested and the preset normal data set, or the various difference values ​​can be aggregated to obtain an aggregated difference value, and the aggregated difference value can be determined as the deviation between the data set to be tested and the preset normal data set. The normal distance can be detected based on the normal data set using the same detection method as the target distance.

[0136] In this embodiment, compared with the static single-model anomaly detection method in the related art, this embodiment takes into account the dynamic characteristics of the target system while performing block detection. For example, the hardware system in the target system includes a CPU, and the change in CPU usage over time is a dynamic characteristic. The software system in the target system includes a database, and the change in the read and write volume of the database over time is a dynamic characteristic, etc. Therefore, a dynamic model detection strategy is introduced, and slow feature analysis is performed on each block data set to be detected and each feature data set to be detected, dynamic features are extracted, and a multi-block dynamic detection model is constructed, which can effectively improve the anomaly detection accuracy of large-scale target systems containing dynamic characteristics.

[0137] In an exemplary embodiment, before detecting the target distance between each detection statistic and the normal data distribution center corresponding to each detection statistic, the method further includes:

[0138] Step S01, obtaining a normal data set of a target system, performing variable block division on the normal data set, and obtaining a normal block data set.

[0139] It should be noted that before performing system anomaly detection, the normal distribution of each data in the normal operation of the target system can be pre-detected, and then by comparing the distribution of various data corresponding to the data to be detected with the distribution of various data corresponding to normal data, the abnormality of the target system can be accurately identified when the target system is abnormal. It is understandable that in order to reduce errors, various operations and processing methods of the data set to be detected and the normal data set can be the same, for example, the same variable block method, the same feature extraction method, the same slow feature analysis method, etc. can be used.

[0140] Among them, the normal data set may refer to a combination of normal data, and normal data refers to the operation data collected from each system unit during the normal operation of the target system, and the operation data collected from each system unit may be the same or different. In some feasible implementations, the number of normal data collected from each system unit may be multiple, and multiple normal data collected from the same system unit may constitute a normal data subset, and the normal data subsets collected from each system unit constitute a normal data set. The normal data subset may include multiple normal data collected from the same system unit within a collection time range, and there is a certain time sequence between each normal data belonging to the same normal data subset.

[0141] Exemplarily, during the operation of the target system, the operation data of each system unit in the target system can be collected in real time, regularly or in a triggered manner. If it is verified that the target system operates normally in a historical time period, the operation data corresponding to the historical time period can be determined as normal data, and the normal data are collected to form a normal data set. After obtaining the normal data set, the normal data set can be firstly segmented into variables, and based on the local feature information of each normal data subset in the normal data set, each normal data subset is classified into a different normal segmented data set. The local feature information contained in the normal data subsets belonging to different normal segmented data sets is different or the difference exceeds a preset difference threshold, and the local feature information contained in the normal data subsets belonging to the same normal segmented data set is the same or similar.

[0142] In some feasible implementations, in order to eliminate the influence of dimension and order of magnitude, performing variable block division on the data set to be tested may include: performing standardization processing on the data set to be tested, and then performing variable block division on the standardized data set to be tested to obtain the block data set to be tested.

[0143] Step S02, extracting features from each normal block data set to obtain a normal feature data set.

[0144] Exemplarily, after performing variable segmentation on the normal data set, feature extraction can be performed on each normal segmentation data set obtained after the variable segmentation, respectively, to obtain a normal feature data set corresponding to each normal segmentation data set. For each normal segmentation data set, the feature extraction method includes: performing feature extraction on each normal data subset in the normal segmentation data set, to obtain a normal feature data subset corresponding to each normal data subset. For each normal data subset, the feature extraction method includes: performing feature extraction on at least part of the normal data in the normal data subset, to obtain normal feature data corresponding to each normal data.

[0145] The normal data subset may include a plurality of normal data collected from the same system unit within a collection time range, and the characteristic information may be dynamic characteristic information, which is used to characterize the characteristics of data changing over time.

[0146] Step S03, performing slow feature analysis on each normal block data set and each normal feature data set to obtain multiple normal data distributions.

[0147] Exemplarily, after obtaining a normal block data set and a normal feature data set, a slow feature analysis can be performed on the normal data subset in each normal block data set to obtain a third slow feature corresponding to each normal data subset, and the third slow features corresponding to each normal data subset in each normal block data set can be fused to obtain a normal data distribution corresponding to each normal block data set; a slow feature analysis can also be performed on the normal feature data subset in each normal feature data set to obtain a fourth slow feature corresponding to each normal feature data subset, and the fourth slow features corresponding to each normal feature data subset in each normal feature data set can be fused to obtain a normal data distribution corresponding to each normal feature data set.

[0148] Step S04, based on the normal data distribution corresponding to each normal block data set or each normal feature data set, detect the normal data distribution center and normal data distribution radius corresponding to each normal block data set or each normal feature data set, and determine the normal data distribution radius as the detection control limit.

[0149] Exemplarily, the normal data distribution center and the normal data distribution radius of the normal data distribution corresponding to each normal block data set are detected, and the normal data distribution center and the normal data distribution radius of the normal data distribution corresponding to each normal feature data set are detected; then, the normal data distribution radius corresponding to each normal block data set is determined as the detection control limit corresponding to the normal block data set, and the normal data distribution radius corresponding to each normal feature data set is determined as the detection control limit corresponding to the normal feature data set.

[0150] In this embodiment, by collecting normal data generated during the normal operation of the target system, the normal data distribution center and detection control limit corresponding to different detection statistics can be more accurately determined. Compared with the method based on experience, the accuracy of system abnormality detection can be effectively improved.

[0151] In an exemplary embodiment, the feature data set to be tested includes at least one of a cumulative error feature data set to be tested and a second-order difference feature data set to be tested; feature extraction is performed on each block data set to be tested, and the feature data set to be tested includes at least one of the following:

[0152] Extracting cumulative error features from each block data set to be inspected respectively to obtain a cumulative error feature data set to be inspected;

[0153] The second-order difference feature extraction is performed on each block data set to be tested to obtain the second-order difference feature data set to be tested.

[0154] Among them, the cumulative error feature is used to characterize the accumulated error of data within a time window. In a sequence, the cumulative error feature corresponding to each data can be the accumulated error of the data and other data within a period of time before the data. It can be understood that if there is no other data before the data, or the time period before the data is less than the preset time range, the cumulative error feature corresponding to the data cannot be detected. Therefore, the total amount of data in the cumulative error feature data set is less than the total amount of data in the corresponding block data set to be tested. For example, assuming that a block data set includes 100 data collected at intervals of one second from 1 to 100s, and the cumulative error feature corresponding to each data is determined based on the data and other data within 5 seconds before the data, then the five data in the first five seconds of the entire sequence cannot detect the cumulative error feature due to the lack of other data within the previous 5 seconds. The final cumulative error feature data set includes the cumulative error features corresponding to the 6th to 100th data, a total of 95, and the total amount of data is 5 less than the block data set.

[0155] Exemplarily, after the data set to be inspected is divided into variable blocks, the cumulative error feature extraction can be performed on each of the block data sets to be inspected obtained after the variable block, respectively, to obtain the cumulative error feature data set to be inspected corresponding to each block data set to be inspected. Wherein, for each block data set to be inspected, the cumulative error feature extraction method includes: performing cumulative error feature extraction on each subset of the data to be inspected in the block data set to be inspected, to obtain a subset of the cumulative error feature data to be inspected corresponding to each subset of the data to be inspected. For each subset of the data to be inspected, the cumulative error feature extraction method includes: sequentially obtaining the data to be inspected at the target moment from the subset of the data to be inspected, and other data to be inspected within a preset cumulative time range before the target moment, calculating the cumulative error value based on the data to be inspected and other data to be inspected, determining the cumulative error value as the cumulative error feature corresponding to the target moment, determining the next moment of the target moment as the new target moment, returning to the step of obtaining the data to be inspected at the target moment, until the data to be inspected in the subset of the data to be inspected that can detect the cumulative error feature completes the cumulative error feature detection, and the obtained cumulative error features form a cumulative error feature data set.

[0156] Among them, the second-order difference feature can refer to the second-order rate of change obtained by further differencing the data after the first difference, which can further reflect the dynamic characteristics of the target system. When one or some anomalies cause the relevant variables to oscillate rather than just change in amplitude, the observation value information and the accumulated error information cannot detect the anomaly well. After the second-order difference of the data, such anomalies can be effectively observed. By extracting this information for modeling, such anomalies can be better detected.

[0157] Exemplarily, after the data set to be inspected is divided into blocks by variables, the second-order differential feature extraction can be performed on each block data set to be inspected obtained after the variable division, respectively, to obtain a second-order differential feature data set to be inspected corresponding to each block data set to be inspected. Wherein, for each block data set to be inspected, the second-order differential feature extraction method includes: performing second-order differential feature extraction on each subset of the data to be inspected in the block data set to be inspected, to obtain a second-order differential feature data subset to be inspected corresponding to each subset of the data to be inspected. For each subset of data to be inspected, the method of extracting second-order differential features includes: sequentially obtaining the data to be inspected at the target moment and other data to be inspected at two moments before the target moment from the subset of data to be inspected, calculating the second-order differential value based on the data to be inspected and other data to be inspected, determining the second-order differential value as the second-order differential feature corresponding to the target moment, determining the next moment after the target moment as the new target moment, returning to execute the step of obtaining the data to be inspected at the target moment, until all the data to be inspected in the subset of data to be inspected that can detect the second-order differential feature have completed the second-order differential feature detection, and the obtained second-order differential features are combined into a second-order differential feature data set.

[0158] As an example, suppose that any block data set after the standardized data set is block-wise , by taking the second-order difference of the data at each moment, a new second-order difference feature data set to be tested is constructed Among them, the second-order difference information x at time t d (t) is:

[0159]

[0160] Among them, x(t) is the data at time t in the block data set; x(t-1) is the data at time t-1 in the block data set; x(t-2) is the data at time t-2 in the block data set.

[0161] In this embodiment, compared with the static anomaly detection method in the related art, this embodiment takes into account the dynamic characteristics of the target system while performing block detection. For example, the hardware system in the target system includes a CPU, and the change in CPU usage over time belongs to a dynamic characteristic. The software system in the target system includes a database, and the change in the read and write volume of the database over time belongs to a dynamic characteristic, etc. Through cumulative error feature extraction and second-order difference feature extraction, the dynamic characteristics of the target system can be effectively captured, and a multi-block dynamic detection model can be constructed, which can effectively improve the accuracy of anomaly detection of the target system containing dynamic characteristics.

[0162] In an exemplary embodiment, the to-be-tested data set includes to-be-tested data at multiple moments; the to-be-tested cumulative error feature data set includes multiple to-be-tested cumulative error features; for each to-be-tested block data set, cumulative error feature extraction is performed on the to-be-tested block data set to obtain the to-be-tested cumulative error feature data set, including:

[0163] Detect the data mean of each to-be-tested data in the to-be-tested block data set; select a preset number of to-be-tested data that are continuous in time sequence from the to-be-tested block data set as the cumulative to-be-tested data; and detect the cumulative error characteristics of the to-be-tested data based on each cumulative to-be-tested data and the data mean.

[0164] Exemplarily, for any block data set to be inspected, it includes multiple subsets of data to be inspected, and each subset of data to be inspected includes multiple data to be inspected with time series. The average value of all the data to be inspected in the data set to be inspected can be calculated first to obtain the data mean value. Then, for any subset of data to be inspected, a preset number of data to be inspected that are continuous in time series are selected as cumulative data to be inspected, and the error value between each cumulative data to be inspected and the data mean value is calculated. The sum of the error values ​​of each cumulative data to be inspected is determined as the cumulative error feature corresponding to the last data to be inspected in this group of cumulative data to be inspected. Then, the execution of selecting the time series is returned. The above step of taking a preset number of consecutive data to be tested as the step of accumulating the data to be tested, until the data to be tested in the subset of data to be tested that can detect the cumulative error characteristics have all detected the corresponding cumulative error characteristics, and the cumulative error characteristic data set corresponding to the subset of data to be tested is obtained; then the same method can be used to detect the cumulative error characteristic data set corresponding to the next subset of data to be tested, until all the subsets of data to be tested have detected the corresponding cumulative error characteristic data sets, and the cumulative error characteristic extraction process for the block data set to be tested is completed, and the same method can be used to extract the cumulative error characteristics of the next block data set to be tested.

[0165] As an example, suppose that any block data set after the standardized data set is block-wise , whose mean is 0, and the standard value is set as the mean of the variable, then the cumulative error value can be obtained by adding the data values. The cumulative error value of the first T moments is used as the cumulative error feature of the Tth moment to construct a new cumulative error feature data set Among them, the cumulative error value x at the tth moment l (t) is:

[0166]

[0167] Among them, x(tl) is the data at time tl in the block data set. Since the first T data in the block data set are used to construct new features, the new cumulative error feature data set will lose T data.

[0168] In this embodiment, the cumulative error feature extraction can capture the error change trend over time. If the cumulative error increases sharply within a certain period of time, it may indicate that the target system has an abnormality or performance degradation. The cumulative error feature not only reflects the error at a single time point, but also integrates the error information at multiple time points, which can effectively feedback the global information of the block data set.

[0169] In an exemplary embodiment, the data set to be inspected includes a plurality of data subsets to be inspected; the data set to be inspected is variable-blocked to obtain the block data set to be inspected, including:

[0170] By presetting the Gaussian mixture model, each subset of the data to be inspected is classified to obtain the classification result;

[0171] According to the classification result, each to-be-inspected data subset is divided into a plurality of to-be-inspected block data sets, wherein the to-be-inspected data subsets belonging to the same classification are divided into the same to-be-inspected block data set.

[0172] Among them, the Gaussian mixture model is a clustering method based on probability density function, which assumes that each cluster is a mixture distribution composed of multiple Gaussian distributions. The goal of the Gaussian mixture model is to estimate the model parameters by maximizing the likelihood function, including the mean, variance and mixing coefficient of each Gaussian distribution, as well as the probability that a data point belongs to each cluster.

[0173] Exemplarily, each data subset to be tested can be input into a pre-optimized Gaussian mixture model to detect the probability of each data subset to be tested belonging to each preset classification, and the classification with the highest probability corresponding to each data subset to be tested is determined as the target classification corresponding to the data subset to be tested, and the target classification corresponding to each data subset to be tested is taken as the classification result; based on the classification result, each data subset to be tested is divided into different block data sets to be tested, wherein the data subsets to be tested belonging to the same classification are divided into the same block data set to be tested, and the data subsets to be tested belonging to different classifications are divided into different block data sets to be tested.

[0174] As an example, suppose the normalized dataset is , represents m n-dimensional sample data. The Gaussian mixture model assumes that the model is governed by K means. , the variance is A mixed model composed of multivariate Gaussian distributions. When clustering variables, The probability of belonging to each class is , represents the sample data The conditional probability of belonging to the jth category, where ; , the specific expression is as follows:

[0175]

[0176] in, represents the covariance matrix of the j-th sample data set, represents the mean vector of the j-th sample data set. According to the Bayesian principle, the sample data can be derived The probability of belonging to the jth class is as follows:

[0177]

[0178] in, Representation classification The probability in the total sample and meets the condition . Parameters as well as , which can be obtained by the maximum likelihood estimation algorithm. Therefore, the steps of clustering system variables include:

[0179] initialization and number of clusters value, and let the number of iterations ;

[0180] Use the maximum likelihood estimation algorithm to find the sample data in the lth iteration process The posterior probability of belonging to the jth class is , the mean of the j-th sample data in the l+1th iteration , covariance matrix and the posterior probability , as shown below:

[0181]

[0182]

[0183]

[0184] Then, the log-likelihood is calculated using the following formula:

[0185]

[0186] like If the convergence condition is not met, set l=l+1 and return to execute the maximum likelihood estimation algorithm to find the sample data in the lth iteration process. The posterior probability of belonging to the jth class is , the mean of the j-th sample data in the l+1th iteration , covariance matrix and the posterior probability steps; if When convergence is reached, the iterative calculation stops and the obtained , , is the characteristic parameter of the Kth data class.

[0187] For any subset of the data to be tested , the following formula can be used to calculate its category:

[0188]

[0189] Where c represents the subset of data to be inspected , The number of data classes where the maximum value is obtained.

[0190] In this embodiment, by performing variable segmentation on the dataset to be inspected using a Gaussian mixture model, different local feature information in the dataset to be inspected can be effectively captured, and local feature information can be prevented from being submerged in the global feature information extraction process, thereby improving the accuracy of anomaly detection.

[0191] In one possible implementation, if Figure 4 As shown, the anomaly detection method includes the following steps:

[0192] Step 1: Get a normal data set X and normalize it to obtain a normalized normal data set.

[0193] Step 2: For the normal data set after standardization, perform variable block division based on the mixed Gaussian model to obtain k variable sub-blocks, which are also block data sets.

[0194] Step 3: After classification, perform cumulative error feature extraction and second-order difference feature extraction on each variable sub-block, and obtain k cumulative error feature sub-blocks and k second-order difference feature sub-blocks, which together with the variable sub-blocks form information sub-blocks, and finally generate 3×k information sub-blocks. Among them, the cumulative error feature sub-block is also the cumulative error feature data set, and the second-order difference feature sub-block is also the second-order difference feature data set.

[0195] Step 4: For each information sub-block, model it based on slow feature analysis and calculate the sub-block statistics, and construct a normal data distribution based on the sub-block statistics .

[0196] Step 5: Construct a support vector data description model based on the normal data distribution obtained in step 4, and determine the center of the hypersphere and the radius R of the hypersphere.

[0197] Step 6: For a new test sample , process it in the same way as normal data and construct the test statistic And bring in the support vector data description model obtained in step 5 to calculate the test sample The distance d from the center of the hypersphere.

[0198] Step 8: If d>R, it is determined that the target system is abnormal; otherwise, it is determined that the target system is normal.

[0199] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0200] Based on the same inventive concept, the embodiment of the present application also provides an anomaly detection device for implementing the anomaly detection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more anomaly detection device embodiments provided below can refer to the limitations on the anomaly detection method above, and will not be repeated here.

[0201] In an exemplary embodiment, Figure 5 As shown, an abnormality detection device is provided, including: an acquisition module 502, a feature extraction module 504, a detection module 506 and a determination module 508, wherein:

[0202] The acquisition module 502 is used to acquire a data set to be inspected of the target system, and perform variable block division on the data set to be inspected to obtain a block data set to be inspected;

[0203] The feature extraction module 504 is used to extract features from each block data set to be inspected to obtain a feature data set to be inspected;

[0204] A detection module 506, configured to detect the deviation between the to-be-detected data set and a preset normal data set based on each to-be-detected block data set and each to-be-detected feature data set;

[0205] The determination module 508 is used to determine that an abnormality exists in the target system when it is detected that the deviation exceeds a preset detection control limit.

[0206] In an exemplary embodiment, the detection module 506 is further configured to:

[0207] Performing slow feature analysis on each block data set to be tested and each feature data set to be tested respectively to obtain multiple detection statistics;

[0208] Detecting the target distance between each test statistic and the normal data distribution center corresponding to each test statistic;

[0209] Based on the distances of each target, the deviation between the dataset to be inspected and the preset normal dataset is detected.

[0210] In an exemplary embodiment, the anomaly detection device further includes a construction module, which, before detecting the target distance between each detection statistic and the normal data distribution center corresponding to each detection statistic, is used to:

[0211] Obtain a normal data set of the target system, divide the normal data set into variable blocks, and obtain a normal block data set;

[0212] Perform feature extraction on each normal block data set to obtain a normal feature data set;

[0213] Perform slow feature analysis on each normal block data set and each normal feature data set to obtain multiple normal data distributions;

[0214] Based on the normal data distribution corresponding to each normal block data set or each normal feature data set, the normal data distribution center and the normal data distribution radius corresponding to each normal block data set or each normal feature data set are detected, and the normal data distribution radius is determined as the detection control limit.

[0215] In an exemplary embodiment, the feature data set to be checked includes at least one of a cumulative error feature data set to be checked and a second-order difference feature data set to be checked; the feature extraction module 504 is further used to:

[0216] Extracting cumulative error features from each block data set to be inspected respectively to obtain a cumulative error feature data set to be inspected;

[0217] The second-order difference feature extraction is performed on each block data set to be tested to obtain the second-order difference feature data set to be tested.

[0218] In an exemplary embodiment, the to-be-tested data set includes to-be-tested data at multiple moments; the to-be-tested cumulative error feature data set includes multiple to-be-tested cumulative error features; and the feature extraction module 504 for each to-be-tested block data set is further used to:

[0219] Detect the data mean of each data to be tested in the block data set to be tested;

[0220] Selecting a preset number of consecutive data to be inspected in time sequence from the block data set to be inspected as the accumulated data to be inspected;

[0221] According to each accumulated data to be inspected and the data mean, the accumulated error characteristics of the data to be inspected are detected.

[0222] In an exemplary embodiment, the to-be-tested data set includes a plurality of to-be-tested data subsets; the acquisition module 502 is further configured to:

[0223] By presetting the Gaussian mixture model, each subset of the data to be inspected is classified to obtain the classification result;

[0224] According to the classification result, each to-be-inspected data subset is divided into a plurality of to-be-inspected block data sets, wherein the to-be-inspected data subsets belonging to the same classification are divided into the same to-be-inspected block data set.

[0225] Each module in the above-mentioned abnormality detection device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0226] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 6 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, an abnormality detection method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0227] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0228] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0229] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0230] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0231] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0232] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0233] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0234] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An anomaly detection method, characterized in that: The method comprises: Acquire a data set to be tested of the target system, and perform variable block division on the data set to be tested to obtain a block data set to be tested; Performing feature extraction on each of the to-be-tested block data sets respectively to obtain a to-be-tested feature data set; Based on each of the to-be-tested block data sets and each of the to-be-tested feature data sets, detecting a deviation between the to-be-tested data set and a preset normal data set; When it is detected that the deviation exceeds a preset detection control limit, it is determined that an abnormality exists in the target system.

2. The method according to claim 1, characterized in that The detecting the deviation between the to-be-tested data set and a preset normal data set based on each of the to-be-tested block data sets and each of the to-be-tested feature data sets includes: Performing slow feature analysis on each of the to-be-tested block data sets and each of the to-be-tested feature data sets to obtain a plurality of detection statistics; Detecting a target distance between each of the detection statistics and a normal data distribution center corresponding to each of the detection statistics; Based on each of the target distances, the deviation between the to-be-tested data set and a preset normal data set is detected.

3. The method according to claim 2, characterized in that Before detecting the target distance between each of the detection statistics and the normal data distribution center corresponding to each of the detection statistics, the method further includes: Acquire a normal data set of the target system, and perform variable block division on the normal data set to obtain a normal block data set; Performing feature extraction on each of the normal block data sets respectively to obtain a normal feature data set; Performing slow feature analysis on each of the normal block data sets and each of the normal feature data sets respectively to obtain multiple normal data distributions; Based on the normal data distribution corresponding to each of the normal block data sets or each of the normal feature data sets, the normal data distribution center and the normal data distribution radius corresponding to each of the normal block data sets or each of the normal feature data sets are detected, and the normal data distribution radius is determined as the detection control limit.

4. The method according to claim 1, characterized in that: The feature data set to be tested includes at least one of a cumulative error feature data set to be tested and a second-order difference feature data set to be tested; the feature extraction is performed on each of the block data sets to be tested to obtain the feature data set to be tested, including at least one of the following: Extracting cumulative error features from each of the block data sets to be inspected respectively to obtain a cumulative error feature data set to be inspected; Second-order differential feature extraction is performed on each of the block data sets to be checked to obtain a second-order differential feature data set to be checked.

5. The method according to claim 4, characterized in that The to-be-tested data set includes to-be-tested data at multiple moments; the to-be-tested cumulative error feature data set includes multiple to-be-tested cumulative error features; For each block data set to be checked, cumulative error feature extraction is performed on the block data set to be checked to obtain a cumulative error feature data set to be checked, including: Detecting the data mean of each of the to-be-tested data in the to-be-tested block data set; Selecting a preset number of consecutive data to be inspected in time sequence from the block data set to be inspected as the accumulated data to be inspected; According to each of the accumulated data to be inspected and the data mean, the accumulated error characteristics of the data to be inspected are detected.

6. The method according to any one of claims 1 to 5, characterized in that: The data set to be tested includes a plurality of data subsets to be tested; the step of performing variable block division on the data set to be tested to obtain the block data set to be tested includes: By presetting a Gaussian mixture model, each of the data subsets to be inspected is classified to obtain a classification result; According to the classification result, each of the to-be-inspected data subsets is divided into a plurality of to-be-inspected block data sets, wherein the to-be-inspected data subsets belonging to the same classification are divided into the same to-be-inspected block data set.

7. An abnormality detection device, characterized in that: The device comprises: An acquisition module is used to acquire a data set to be tested of a target system, and perform variable block division on the data set to be tested to obtain a block data set to be tested; A feature extraction module is used to extract features from each of the block data sets to be inspected to obtain a feature data set to be inspected; A detection module, used for detecting the deviation between the to-be-detected data set and a preset normal data set based on each of the to-be-detected block data sets and each of the to-be-detected feature data sets; The determination module is used to determine that an abnormality exists in the target system when it is detected that the deviation exceeds a preset detection control limit.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images